Lesson iut.univ-paris8.fr 2018-05
Introduction to VMs & Containers
This Document: http://arnaud-nauwynck.github.io/docs/Intro-VM-Container.pdf
[email protected] Hardware → Software
"All problems in computer science can be solved by another level of indirection"
(the "fundamental theorem of software engineering")
David Wheeler
Hardware → Assembly Langage → Compiler & Langages →Shared Libraries →Frameworks
Problem 1 To Solve : Libraries Dependencies Hell
Think “Windows” … to understand “Dll Dependency Hell”
Think “Debian” … to upgrade Xserver driver
Think “NPM” / “NodeJS” … package.json node_modules and versionning
Example: Npm modules hell
Problem 2 To Solve : Configuration Conflicts / Reuse Component
foo Think “/etc/
OK bar bar.conf
How to have multiple instances ?? foo-4
Problem 4 : Resource Sharing
Hardware → OS →Virtual Machine →Containers →Orchestrators
Problem 5 : Resource Scheduling
Virtualisation – Containers History
2003 2005 2007 2008 2010 2013 2016 Qemu Intel VT-x KVM LXC dotCloud Docker released Docker Xen company DotCloud renamed on Windows VirtualBox created by Docker Inc ( Innotek Solomon Hykes 2018 → Sun 2008 Solomon → Oracle 2010) quits Docker .. what’s next?
https://en.wikipedia.org/wiki/Timeline_of_virtualization_development Qemu – KVM – VirtualBox ...
LibVirt ...
Lightweight Containers ↔ Virtual Machines
VMs Containers
https://docs.docker.com/get-started/#containers-and-virtual-machines Docker : Linux Container Then Windows & Mac
Docker – Libcontainer
Libcontainer, Open Container
https://containerd.io/
Other Container Technologies.. https://coreos.com/rkt/
Linux Isolations primitives for Containers
FileSystem
FileSystem : Class / Design-Pattern Interpretation
Facade, Delegate design-pattern : all System I/O goes to a facade API
File FileSystem
RegularFile Dir
ChRoot
ChRoot – Class/Pattern Interpretation
File FileSystem
RegularFile Dir
MountDir
1 rootDir= “/” Process (Context) Object Instance Diagram : forking a ch-rooted process Can not reach parent dir MountDir: “/” seen as “/..” for “/a/b” Dir: “/a/b/chroot-dir” MountDir: seen as “/” for process 2 CtxParent1 : Process … correspond to “/a/b/chroot-dir” (Context) ChildCtx2 : Process (Context) ChRoot – Object Diagram for ch-rooted child process
“/” for process2 Can not reach parent dir … correspond to seen as “/..” “/a/b/chroot-dir” for “/a/b” … JAILS
MountDir: “/” MountDir: seen as “/” for process 2 Dir: “/a/b/chroot-dir”
CtxParent1 : Process ChildCtx2 : Process (Context) (Context)
Mount
Mount : Class / Design-Pattern Interpretation
Adapter design-pattern : adapt a Dir sub-class to delegate to FileSystem MountDir → FileSystem
File FileSystem
RegularFile Dir
MountDir
“Mount“ usage in Docker : Volumes
Example: $ docker run -v /data:/redis-1/data --name redis-1 redis:latest
UnionFS, AuFS
Example: $ mkdir /tmp/rw && mkdir /tmp/aus & sudo mount -t aufs -o br=/tmp/rw=rw:/home/user=ro none /tmp/aus/
… will show in “/tmp/aus” the union of “/tmp/rw” and “/home/user” (kept as read-only)
UnionFS, AuFs
Read-Write overriden filesystem If found Diff Write only Diffs diff override (Copy-On-Write from Base filesystem) Virtual UnionFS Else NOT found Base Read-Only base filesystem
Example: Live linux distribution for test (no persistent files), on Boot USB disk
UnionFS : Class / Design Pattern
Decorator design-pattern ? Example: 1 layer Read-Write / Above 1 layer Read-Only decorate Read-Write over a Read-Only
Delegate / Chain-Of-Responsibility design-pattern ? Example: multiple layers … when match pattern, then delegate else bubbles up
File FileSystem
Dir
MountDir
UnionMountDir UnionFileSystem
* layers
UnionFSLayer UnionFS usage in Docker : Image Layers Stack (Dockerfile)
Dockerfile
FROM debian:latest
RUN apt-get install tomcat
RUN apt-get install apache
$ docker build -t my-image .
Loopback Device, Losetup
Loopback – Class / Design Pattern
Adapter design-pattern : DeviceFile → File
Class Diagram:
File Driver
RegularFile DeviceFile
LoopbackDriver Loopback
Namespaces (UTS, PIDs, ..)
Named 1 Resource Namespace * [name] 1 1 0..1 0..1 1 : leftNs 1 : rightNs
Name Translation Table
* Name Translation Entry LeftName rightName
Example of Namespaces in Linux: PIDs, Hostnames, ...
Reminder on TCP-IP Socket, Packet (IP:Port→IP:Port)
Level 7 Socket stream Level 7 Process1 Process2 Level 3 : TCP socket Receive/send data Level 3 : TCP socket Port=Port1 Port=Port1 IP-Src,IP-Dest,Port-Src,Port-Dest, tcpNum,tcpAck,tcpWindow,...len,data
Receive/send packets Level 1 : IP eth0… Level 1 : IP eth0… IP=IP1 IP=IP1 IP Packet format: IP-Src,IP-Dest,Port-Src,Port-Dest, len,data...
Level 0 : eth0… Level 0 : eth0… Arp=Arp1 Electric/Fiber Arp=Arp1 connection
Notice: can have Host1=Host2, IP1=IP2, ... NAT : Network Address Translation
Process1 IP-Src Port-Src
IP-Src,IP-Dest1,Port-Src,Port-Dest1 ..
NAT Translater
IP-Src,IP-Dest-NAT,Port-Src,Port-Dest-NAT ..
IP-Dest-NAT Process1 Port-Dest-NAT Example: Docker Port Export
$ docker run -p 4001:6379 --name redis-1 redis:latest $ docker run -p 4002:6379 --name redis-2 redis:latest
4001 6379 6379
4002
Container: redis-1 Container: redis-2 docker-proxy (host: a23bc6d1, IP:..) (host: b6c9aff2, IP:..)
Docker ContainerD (VETH0 Bridge..)
Linux OS (Localhost) .. Docker -p
Check connecting manually (telnet) to redis on 4001 … not on 6379 !!
execute redis commands
It works ..
Result Linux processes ..
Idem NAT … Level 7 : Http Reverse Proxy
Process1 Http Request1
Http Reverse Http Server Proxy
Http Response 200 headers k=v Http Response 200 Body {json..} headers k=v-new Body {json-new..}
DNS : Host to IP resolver
Ingress Networks.. (= Http Reverse Proxy + DNS + ..)
Next Chapter 2/3 : Docker
Next Chapter 3/3 : Kubernetes Orchestration