Sessions for Docker and Kubernetes Docker History from Chroot to Docker ● https://blog.aquasec.com/a-brief-history-of-containers-from-1970s-chroot-to-docker-2016 ● https://dzone.com/articles/evolution-of-linux-containers-future ​ What is Docker ● https://docs.docker.com/engine/faq/ ● LXC or runc (Open Container Specification) + AUFS/OverlayFS + Linux Namespaces + Control Groups What are LXC and Control Groups ● https://www.upguard.com/articles/docker-vs-lxc ● https://docs.docker.com/engine/faq/ ● https://sysadmincasts.com/episodes/14-introduction-to-linux-control-groups-cgroups ● https://sysadmincasts.com/episodes/24-introduction-to-containers-on-linux-using-lxc What are Namespaces ● https://blogs.rdoproject.org/7761/hands-on-linux-sandbox-with-namespaces-and-cgroups ● Section of Docker Network Example in https://github.com/gravitational/workshop/blob/master/k8snet.m What are Docker layers (OverlayFS/AUFS) ● http://blog.programster.org/overlayfs ● https://docs.docker.com/engine/userguide/storagedriver/imagesandcontainers/ ● https://docs.docker.com/engine/userguide/storagedriver/overlayfs-driver/ Docker 101 Workshop Docker Basics VM vs Container Components of Docker ● Docker Daemon o Manages complete container lifecycle and images on docker host. ● Docker Command Line o Allows interaction with docker daemon o Allows creation and management of containers and images o Allows interaction with docker registry ● Docker Registry o “Repository” of docker “images” used in creating “containers” Docker Objects ● Docker Images o Templates of creating containers o “Layered” o Managed on local and on remote registries ● Docker Containers o Actual running instances of “images” o Combination of “namespaces”, ”control groups (cgroups)” , “capabilities”, UnionFS etc Docker built on Multiple Linux subsystems ​ ● Linux Namespaces(runc) o NET namespace creates a separate networking stack for the container, with its own routing tables and devices o PID namespace is used to assign isolated process IDs that are separate from host OS. For example, this is important if we want to send signals to a running process. o MNT namespace creates a scoped view of a filesystem using VFS. It Let’s a container to get its ​ ​ own "root" filesystem and map directories from one location on the host to the other location inside container. o UTS namespace Let’s container to get to its own hostname. o IPC namespace is used to isolate inter-process communication (e.g. message queues). o USER namespace allows container processes have different users and IDs from the host OS. ● Control Groups (cgroups) o Kernel feature that limits, accounts for, and isolates the resource usage (CPU, memory, disk I/O, network, etc.) ● Capabilities o Capabilities provide enhanced permission checks on the running process, and can limit the interface configuration, even for a root user - for example (CAP_NET_ADMIN) ● Union file system(AUFS) o Docker Engine uses UnionFS to provide the building blocks for containers. Docker Engine can use multiple UnionFS variants, including AUFS, btrfs, vfs, and DeviceMapper Docker Workshop ● Install docker o Prefer to use Ubuntu 16.04 (avoid Ubuntu 14.04 as lot of kernel updates for docker not present) o apt-get install docker.io o Note : docker.io is a Ubuntu provided package and docker-ce is the community version of docker Ubuntu package ● Let’s Check a few things about the installed docker o docker info o docker version ● Check if Docker is running as expected o docker run hello-world o The first time you run this it will download the hello-world image from docker hub and later it will use the cached image for further reruns. ● If not working set HTTP_PROXY and HTTPS_PROXY o place the above env variables in docker.service or /etc/docker/default o Prefer using systemctl instead of service ● Run commands on docker containers o docker run busybox echo “Hello World” o Let’s see processes within the container ▪ docker run busybox ps aux o Let’s see environment variables in container ▪ docker run busybox env o Let’s view the folder without any external mounts ▪ docker run busybox ls -l /home o Let’s view the folder with host directory mounted inside the container ▪ docker run -v $PWD:/home busybox ls -l /home o Let’s view the network interfaces in docker container ▪ docker run busybox ifconfig -a o Let’s setup port forward from host to container and run an http service in container ▪ docker run -p 5000:5000 library/python:3.3 python -m http.server 5000 ▪ From host ▪ curl http://localhost:5000 ​ o Let’s Run Containers as daemons ▪ docker run -d -p 5000:5000 --name=simple1 library/python:3.3 python -m http.server 5000 ▪ Essential point : As soon as the main process with container is closed , container is killed. o Let’s see container detail ▪ docker ps o Let’s see more container detail ▪ docker inspect <Container ID> o Let’s see container logs ▪ docker logs <Container ID> o Let’s attach to running container ▪ docker exec -ti mycontainer /bin/bash ▪ Please note that container should have the /bin/bash binary ▪ For security its possible that the container does not have bash or sh binary Docker Ecosystem ● Docker Engine o “Core” of docker ● Docker Machine o Creates/provisions docker “hosts” in various clouds such as Azure, AWS including vagrant ● Docker Swarm o Manages multiple docker hosts. ● Docker Compose o Helps assemble applications using multiple docker containers Docker Engine Components ● Docker Engine o ContainerD ▪ Manages container/images lifecycle ▪ Uses RunC to create containers ▪ Supports Open Container Interface format(Namespaces+CGroups+Capablities+UnionFS) ​ ​ ​ ​ ​ ​ o RunC ▪ Runtime for running containers Deep Dive on Linux Namespaces https://blog.yadutaf.fr/2013/12/22/introduction-to-linux-namespaces-part-1-uts/ Deep Dive on Control Groups http://www.fernandoalmeida.net/blog/how-to-limit-cpu-and-memory-usage-with-cgroups-on-debian-ubuntu/ Docker 102 Workshop Docker Basics – Building Images ● Build Apache Service Container via dockerfile ● Create a new directory o mkdir docker-apache ● Create a Dockerfile in the above created folder FROM alpine:3.3 RUN apk --update add apache2 && rm -rf /var/cache/apk/* RUN mkdir -p /run/apache2 EXPOSE 80 CMD httpd -D FOREGROUND ● Build the container o docker build –t docker-apache . ▪ Please note the (.) at the end of the line ● Let’s run the container o docker run –d –p 2803:80 docker-apache ● Let’s try and get a page from host o curl http://localhost:2803 Docker 103 Workshop Docker Basics – Networking Docker Built-In Network Options ● Bridge — The bridge driver creates a Linux bridge on the host that is managed by Docker. By default ​ containers on a bridge will be able to communicate with each other. External access to containers can also be configured through the bridge driver. ● Overlay — The overlay driver creates an overlay network that supports multi-host networks out of the box. ​ It uses a combination of local Linux bridges and VXLAN to overlay container-to-container communications over physical network infrastructure. ● MACVLAN — The macvlan driver uses the MACVLAN bridge mode to establish a connection between ​ container interfaces and a parent host interface (or sub-interfaces). It can be used to provide IP addresses to containers that are routable on the physical network. Additionally VLANs can be trunked to the macvlan driver to enforce Layer 2 container segmentation. ● Host — With the host driver, a container uses the networking stack of the host. There is no namespace ​ separation, and all interfaces on the host can be used directly by the container. ● None — The none driver gives a container its own networking stack and network namespace but does not ​ configure interfaces inside the container. Without additional configuration, the container is completely isolated from the host networking stack. Docker Network Model ● Explore Docker Networking o List Docker Network Options ▪ docker network ls o Inspect Network ▪ Docker network inspect <name> $ docker network ls NETWORK ID NAME DRIVER SCOPE 1befe23acd58 bridge bridge local 726ead8f4e6b host host local ef4896538cc7 none null local # Install the brctl tools $ apt-get install bridge-utils <Snip> # List the bridges on your Docker host $ brctl show bridge name bridge id STP enabled interfaces docker0 8000.0242f17f89a6 no $ ip a <Snip> 3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:f1:7f:89:a6 brd ff:ff:ff:ff:ff:ff inet 172.17.0.1/16 scope global docker0 valid_lft forever preferred_lft forever inet6 fe80::42:f1ff:fe7f:89a6/64 scope link valid_lft forever preferred_lft forever $ docker run -dt ubuntu sleep infinity 6dd93d6cdc806df6c7812b6202f6096e43d9a013e56e5e638ee4bfb4ae8779ce ​ $ brctl show bridge name bridge id STP enabled interfaces docker0 8000.0242f17f89a6 no veth3a080f $ docker network inspect bridge <Snip> "Containers": { "6dd93d6cdc806df6c7812b6202f6096e43d9a013e56e5e638ee4bfb4ae8779ce": { ​ ​ "Name": "reverent_dubinsky", "EndpointID": "dda76da5577960b30492fdf1526c7dd7924725e5d654bed57b44e1a6e85e956c", "MacAddress": "02:42:ac:11:00:02", "IPv4Address": "172.17.0.2/16", "IPv6Address": "" } }, <Snip> $ ping 172.17.0.2 64 bytes from 172.17.0.2: icmp_seq=1 ttl=64 time=0.069 ms 64 bytes from 172.17.0.2: icmp_seq=2 ttl=64 time=0.052 ms 64 bytes from 172.17.0.2: icmp_seq=3 ttl=64 time=0.050 ms 64 bytes from 172.17.0.2: icmp_seq=4 ttl=64 time=0.049 ms 64 bytes from 172.17.0.2: icmp_seq=5 ttl=64 time=0.049 ms ^C --- 172.17.0.2 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 3999ms rtt min/avg/max/mdev = 0.049/0.053/0.069/0.012 ms Docker 104 Workshop Docker Basics – Storage(revisited) What does “Docker uses Layers mean”? ● Image is combination of layers ● The running “container” basically means having a “Thin RW Layer” linked to “Image”. o All edits and changes go to the “Thin RW Layer” o Once the container is removed the changes are also gone ● Multiple containers (Thin RW Layers) point to just one “Image”, saving on-disk size.