Bachelor Informatica Utilizing p4 switches in con- tainer networking Ries van Zon - 11302836 June 15, 2020 Informatica | Universiteit van Amsterdam Supervisor(s): Msc Sara Shakeri, dr. Paola Grosso Signed: Signees page 2 Abstract Containers are very popular these days and there are several software tools by which a container can be created. To connect containers the most popular method is utilizing an overlay network method. A container overlay network is easy to deploy. The drawback is that with creating an overlay network each packet will get an extra layer of encapsulation. We try to find an alternative for the overlay solution by creating connectivity between docker containers using a p4 switch. A TCP connection is created between local containers. Currently, there is no TCP connection possible between two containers on a different host. We narrowed down what the problem could be. Our findings show that the problem occurs at L4. L3 and L4 (concerning TCP) connections between Docker containers on the same host are possible. For containers on different hosts (After NAT is applied) L3 connectivity is possible but the connection at L4 is refused. page 3 page 4 Contents 1 Introduction 7 1.1 Introduction.......................................7 2 Theoretical background9 2.1 Process isolation....................................9 2.1.1 Namespaces...................................9 2.2 Containers........................................ 10 2.2.1 Linux container................................. 10 2.2.2 Docker containers............................... 10 2.3 p4 switches....................................... 11 2.3.1 Defenition of a simple p4 program...................... 12 2.3.2 Deployment of a p4 program......................... 15 2.4 Overlay network.................................... 16 3 Related work 17 3.1 Connectivity of docker containers........................... 17 3.2 Usage of p4 in SDN.................................. 18 4 Method 19 4.1 Establishing connectivity between containers using a p4 switch.......... 19 4.1.1 Connecting a Docker container to a p4 switch................ 19 4.2 TCP connection between containers using a p4 switch............... 20 4.2.1 Network topologies............................... 24 4.2.2 TCP connection between containers in a single host............ 26 4.2.3 TCP connection between containers in multiple hosts........... 30 5 Discussion 33 5.1 Multiple containers on the same host......................... 33 5.2 Single container, multiple hosts............................ 34 5.3 Firewall settings.................................... 34 6 Conclusions 37 6.1 Conclusion....................................... 37 6.2 Future work....................................... 37 6.2.1 Ethics...................................... 38 Appendices 43 A Appendix A 45 5 CONTENTS page 6 CONTENTS CHAPTER 1 Introduction 1.1 Introduction Containers are very popular these days and there are several software tools by which a container can be created. Like Docker. A container is just another way of virtualization. Despite virtual machines that use separated hardware and on top of that separated operating system, in con- tainers, the virtualization is at OS level and they share the host's kernel. Therefore, comparing with Virtual Machines (VMs) the number of resources that are utilized in container technologies is much less. [8]. Another advantage of containers is that they are quick to set up and easy to deploy[8]. Containers can also run on any distribution not dependent on the hardware of the distribution. To connect containers the most popular method is utilizing overlay network methods. A container overlay network is easy to deploy. The drawback is that with creating an overlay network each packet will get an extra layer of encapsulation. This decreases performance. Since there is more and more data send through the network nowadays, it's important to minimize the overhead where possible. The challenge is to create a network of containers, with the benefits of containers but without or at least with reduced overhead. We will try to accomplish this by implementing a p4 switch. A p4 switch is a switch from which we can program the data plane. With a p4 switch, we have more control over the incoming packets and how to forward them. P4 allows building more intelligent control mechanisms with regards to packets. To implement the p4 switch we have to find a way to change the docker default settings making a container more flexible and giving us more control over a container. To do this we have to be careful not to lose the advantages of a docker container which docker provides. To answer the research question • how can we minimize the overhead in Docker container networks by using mod- ern technologies such as p4 switches? First, we will focus on establishing connectivity between containers. In the chapter 2 (background) and chapter 3 (related work), we will show how a container can be con- nected to a p4 switch. In the chapter 4 (method), we will examine several implementations with two network settings, for a docker container, to set up a TCP connection between containers. The method chapter focusses on the question: { how can we create connectivity between containers by utilizing a p4 switch? In the chapter 5 (Discussion) we will discuss and evaluate the results after which will we draw some conclusions and propose some topics for future work in the chapter 6 (Conclusion). 7 1.1. INTRODUCTION page 8 CHAPTER 1. INTRODUCTION CHAPTER 2 Theoretical background 2.1 Process isolation Around '79 the chroot command was introduced in Linux systems. The chroot command on Unix operating systems is an operation that changes the apparent root directory for the current running process and its children. When you change root to another directory you cannot access files and commands outside that directory. With the chroot command we could create the impression of an isolated process. Chroot simply modifies pathname lookups for a process and its children, prepending the new root path to any name starting with /. The current directory is not modified and relative paths can still have access to any location outside of the new root. This is seen as the start of process isolation which has led to the containers of today. 2.1.1 Namespaces Namespaces are used to isolated different sets of global resources and restrict access to an inner process to its sandbox. There are seven types of namespaces [9] of which three are of special importance concerning a container: • Network namespace (netns) • Control group namespace (cgroup) • Process ID (pid) Network namespace Network namespaces visualize the network stack. On creation, a network namespace contains only a loopback interface. Each network interface (physical or virtual) is present in exactly one namespace and can be moved between namespaces. A network namespace isolates net- work devices, ports and has a private set of ip addresses, its routing table, ARP table, and other network-related resources. Destroying a network namespace destroys any virtual interfaces within it and moves any physical interfaces within it back to the initial network namespace. Initially, all the processes share the same default network namespace from the init process. By convention when a named network namespace is created it's an object at =var=run=netns=name that can be opened and listed by the command ip netns. The network namespace does not belong to any process yet. By attaching the network namespace to the network namespace of a process we isolate the network resources of that process. Cgroups A control group (cgroup) is a collection of processes that are bound by the same criteria and associated with a set of parameters or limits. These groups can be hierarchical, meaning that each 9 2.2. CONTAINERS group inherits limits from its parent group. The kernel provides access to multiple controllers (also called subsystems) through the cgroup interface A cgroup namespace virtualizes the contents of the =proc=self=cgroup file. Processes inside a cgroup namespace are only able to view paths relative to their namespace root. PID namespace PID namespaces are nested, meaning when a new process is created it will have a PID for each namespace from its current namespace up to the initial PID namespace. Hence the initial PID namespace can see all processes, but with different PIDs than other namespaces will see processes with. 2.2 Containers 2.2.1 Linux container Linux Containers is an virtualization method for running multiple isolated Linux systems (con- tainers) on a host using a single (Linux) kernel. The Linux kernel provides the cgroups functionality that allows limitation and prioritization of resources (CPU, memory, block I/O, network, etc.) without the need for starting any virtual machines, and also namespace isolation functionality that allows complete isolation of an appli- cation's view of the operating environment, including process trees, networking, user IDs and mounted file systems. LXC combines the kernel's cgroups and support for isolated namespaces to provide an isolated environment for applications. A container is a process from which different namespaces are isolated by attaching different customized namespaces to the namespaces of the process. Each process has its own idea of its root. By default, the root of a process is set to the system root. We can change the root of a process with the chroot command. When we create a container we create it from the context of the current process where from where the command is executed. This means that by default the root of the container is set to the root of the process from which the container was created. Today's container makes use of two features of the kernel: cgroups and namespaces. Before cgroups every process resources were managed individually. Applications that had more processes running using were using more system resources. With cgroups we can create a subgroup and assign a certain amount of system resources to that subgroup. All sub-processes belonging to that subgroup can only use the resources that were assigned to their subgroup they belong too. If we list (command systemd-cgls cpu) the cgroups we see that for each container a subgroup is created.