IT 20 044 Examensarbete 30 hp July 2020 Network Service Mesh Solving Cloud Native IMS Networking Needs Lionel Jouin Institutionen för informationsteknologi Department of Information Technology Abstract Network Service Mesh Solving Cloud Native IMS Networking Needs Lionel Jouin Teknisk- naturvetenskaplig fakultet UTH-enheten With the growing demand for mobile networks and specially IP Multimedia subsystem (IMS), new cloud native orchestration tools Besöksadress: providing more flexibility and efficiency start to be used within Ångströmlaboratoriet Lägerhyddsvägen 1 telecommunication companies in order to improve the robustness and Hus 4, Plan 0 the reliability of these systems. However, Kubernetes, the most used among cloud native orchestration tools does not fulfill completely Postadress: all the needs and use cases in terms of networking the Box 536 751 21 Uppsala telecommunication industry meets. Network Service Mesh (NSM), a new Cloud Native Computing Foundation (CNCF) project, aiming to address Telefon: complex networking use cases in Kubernetes might solve the different 018 – 471 30 03 issues IP multimedia subsystem face. Detailed designs and Telefax: implementations using Network Service Mesh coupled with diverse 018 – 471 30 00 networking technologies are shown in this thesis with the objective of solving the networking IP multimedia subsystem requirements (e.g. Hemsida: the NAT issue and the secondary network). In addition, an analysis http://www.teknat.uu.se/student and an evaluation of Network Service Mesh is given together with a presentation of the ability of this new project to bring solutions to IP Multimedia subsystem based on a cloud native technology. Handledare: Saminathan Vijayabaskar Ämnesgranskare: Thiemo Voigt Examinator: Mats Daniels IT 20 044 Tryckt av: Reprocentralen ITC Acknowledgements This work has been conducted in collaboration with Ericsson. I want to thank the company for having provided all the information and resources I required to complete this project. Special thanks to Jerker Zetterlund for his constant support and for giving me this wonderful opportunity to work at Ericsson. To my supervisor, Saminathan Vijayabaskar, I would like to express my gratitude for his very helpful experience and kindness. I would also like to thank those who, in any way, have been involved in my thesis work. Further, I would like to thank my reviewer Thiemo Voigt at Uppsala University for his precious advices and comments for structuring and writing this document. July 3rd, 2020 Lionel Jouin Contents 1 Introduction1 1.1 Motivation and Objectives.......................2 1.2 Delimitations..............................2 1.3 Structure of the Report.........................3 2 Background4 2.1 IP Multimedia subsystem........................4 2.1.1 Network address translation issues...............5 2.1.2 Traffic separation / Secondary network............6 2.1.3 Environment..........................6 2.2 Linux..................................6 2.2.1 Namespaces...........................7 2.2.2 Container............................7 2.3 Kubernetes...............................7 2.3.1 Service.............................7 2.3.2 Container Network Interface..................8 2.4 Network Function............................8 2.4.1 Load Balancing.........................9 2.4.2 Firewall............................. 10 2.4.3 BGP / ECMP.......................... 10 2.5 Service Mesh.............................. 10 2.6 Network Service Mesh......................... 11 2.6.1 Control plane.......................... 13 2.6.2 Data plane........................... 14 2.6.3 Service Function Chaining................... 15 2.6.4 Community and future development.............. 16 2.7 Related work.............................. 17 2.7.1 NAT............................... 17 2.7.2 Alternatives........................... 17 2.7.3 Performance.......................... 18 i 3 Design 19 3.1 Ingress traffic alternatives........................ 19 3.1.1 Host shared........................... 19 3.1.2 VPN.............................. 21 3.1.3 MACVLAN / IPVLAN..................... 23 3.1.4 Overlay Network / VxLAN.................. 25 3.1.5 Load Balancing and VIP advertisement............ 27 3.2 Egress traffic alternatives........................ 28 3.2.1 Tunneling............................ 29 3.2.2 NSE delegation......................... 29 3.2.3 Connection Tracker / Port Allocation............. 30 3.2.4 Multiple NSEs......................... 32 3.2.5 Dynamic allocation by the application............. 32 3.3 Data plane / Control plane separation.................. 32 4 Implementation 36 4.1 Environment............................... 36 4.1.1 OpenStack........................... 36 4.1.2 Kubernetes........................... 37 4.1.3 Development.......................... 37 4.2 Network Service Endpoint....................... 37 4.2.1 Interface............................ 37 4.2.2 VPN.............................. 38 4.2.3 Load Balancing......................... 39 4.2.4 BGP............................... 40 4.2.5 Port allocation......................... 40 4.3 Network Function Chaining...................... 42 4.4 Network Service Client......................... 44 5 Evaluation 46 5.1 Benchmarking methodology...................... 46 5.2 Data plane performance......................... 47 5.2.1 External Connectivity..................... 47 5.2.2 Network Service Mesh Connectivity.............. 49 5.3 Security................................. 50 5.4 Scalability................................ 50 6 Conclusions and Future work 52 ii List of Figures 2.1 Reference Architecture of the IP Multimedia Core Network Subsystem5 2.2 Overview of networking in Kubernetes with NSM........... 12 2.3 Network Service Chaining example.................. 16 3.1 NSM - Ingress - Host Shared...................... 20 3.2 NSM - Ingress - VPN.......................... 22 3.3 NSM - Ingress - IPVLAN/MACVLAN................. 24 3.4 NSM - Ingress - VxLan......................... 26 3.5 NSM - Ingress - BGP and IPVS.................... 28 3.6 NSM - Egress - NAT.......................... 30 3.7 NSM - Egress - No NAT........................ 31 3.8 NSM - Data plane / Control plane separation using namespaces.... 34 4.1 LVS - Packet Flow........................... 41 5.1 External Connectivity performances.................. 48 5.2 Network Service Mesh Connectivity performances.......... 49 iii Listings 4.1 IPVS command to create a service................... 39 4.2 IPVS command to add a real server to a service............ 39 4.3 IPVS command to remove a real server from a service......... 40 4.4 IPTables command to mark TCP packets according to a destination port range................................ 42 4.5 Specification of an NSM Network service............... 42 4.6 Specification of a Network Service Endpoint deployment....... 43 4.7 Specification of a Network Service Client deployment......... 44 4.8 IPTables to source NAT outgoing traffic................ 45 iv List of symbols and abbreviations 3GPP 3rd Generation Partnership Project AS Autonomous System ASIC Application-specific integrated circuit BGP Border Gateway Protocol CIDR Classless Inter-Domain Routing CNCF Cloud Native Computing Foundation CNF Cloud Network Function CNI Container Network Interface DC Data Center DHCP Dynamic Host Configuration Protocol DNS Domain Name System DPDK Data Plane Development Kit ECMP Equal-Cost Multi-path Routing EVPN Ethernet VPN fps Frames per Second FQDN Fully Qualified Domain Name FTP File Transfer Protocol fwmark Firewall Mark GRE Generic Routing Encapsulation gRPC gRPC Remote Procedure Calls HTTP Hypertext Transfer Protocol IANA Internet Assigned Numbers Authority IETF Internet Engineering Task Force IMS IP Multimedia Subsystem IoT Internet of Things IP Internet Protocol IPAM IP Address Management IPsec Internet Protocol Security IPVS IP Virtual Server ipvsadm IPVS Administration ISP Internet Service Provider LAN Local Area Network v LB Load Balancer LVS Linux Virtual Server MAC Media Access Control memif Shared Memory Packet Interface MP-BGP Multiprotocol BGP MTAS Multimedia Telephony Application Server MTU Maximum Transmission Unit NAPT Network Address Port Translation NAT Network Address Translation NFV Network function virtualization NSE Network Service Endpoint NSM Network Service Mesh OCI Open Containers Initiative OS Operating System OSI Open Systems Interconnection OVS Open vSwitch PCIe Peripheral Component Interconnect Express pps Packets per Second QoS Quality of service RDMA Remote Direct Memory Access RFC Request for Comments SCTP Stream Control Transmission Protocol SDK Software Development Kit SDN Software-Defined Networking SDP Session Description Protocol SFC Service Function Chaining SIP Session Initiation Protocol SR-IOV Single-root input/output virtualization srv6 Segment Routing over IPv6 TCP Transmission Control Protocol Telco Telephone Company TOE TCP Offload Engine UDP User Datagram Protocol URLLC Ultra-Reliable and Low Latency Communications us Microsecond veth Virtual Ethernet Device VIP Virtual IP VLAN Virtual Local Area Network VM Virtual Machine VNF Virtual Network Function VNI VLAN/VxLAN Network Identifier VPN Virtual Private Network VPP Vector Packet Processing VxLAN Virtual eXtensible Local Area Network YAML YAML Ain’t Markup Language vi Chapter 1 Introduction In recent years, the growth of data production and consumption has never stopped increasing. One of the main motivations for the development of 5G is to manage this amount of data caused by existing technologies