Cloud Applications CS 162 Lecture 23, Will Wang & David Culler Nov 19th, 2019 Containers Microservices + The Cloud Container Orchestration Summary Containers Review: what is a container? - Provides light weight operating system virtualization - A “light weight virtual machine” is a good analogy but technically untrue - A filesystem of binaries, libraries, and dependencies - Isolation from other processes and containers - via resource isolation (cgroups) - via namespace isolation - PID namespace - Network namespace - Mount namespace - etc Containers Review: what is a container? • Containers shares the host kernel, but App A App B have their own system binaries and Virtual libraries Machine Bin/Libs Bin/Libs Guest Kernel Guest Kernel { App A App B Container Hypervisor { Bin/Libs Bin/Libs Host Kernel Host Kernel Hardware Hardware Containers Docker - Features on top of the basic OS cgroup containers - Images - Portable deployment - Versioning and management - Image layering - Reproducibility - Union file system - Tools and observability (docker logs, docker exec, etc) - Container lifecycle management (docker run, docker stop, etc) Containers Images FROM ubuntu:latest A compressed collection of the libraries, - MAINTAINER Edward Elric binaries, and applications that make up a RUN apt-get update -y container RUN apt-get install -y python-pip python-dev build-essential Defined by a file called the Dockerfile - RUN pip install uwsgi flask Very similar to a script and sets up the container - COPY transmute.py ~/transmute.py environment WORKDIR ~/ - Docker than compresses the environment into a RUN echo “1 soul” > sacrifice.txt binary CMD [“python”, “transmute.py”] - Docker offers tools and repositories (container registry) to build, store, and manage these images. Containers Images - The image is divided into a sequence of layers. Each command creates a new layer on top of the previous layers. - Images with common commands will have common layers whose binaries they share. This is the union file system. Containers Command Docker CLI Docker Architecture Line Tool HTTP REST API Dockerd (docker daemon) responds to - User Level Dockerd commands from Docker CLI Management - It handles management of docker objects (images, containers, volumes, networks, etc) gRPC - Dockerd uses Containerd to manage the High Level container lifecycle Containerd Runtime - It handles image push / pull, namespace management, etc Exec - Containerd invokes a low level container Low Level runtime (runc) to create the container runc Runtime Containers Container Runtime • A library that is responsible for starting and managing containers. • Takes in a root file system for the container and a configuration of the isolation configurations • Creates the cgroup and sets resource limitations • Unshare to move to own namespaces • Sets up the root file system for the cgroup with chroot • Running commands in the cgroup • runc by Docker, rkt by CoreOS, gvisor (runsc) by Google, LXC by Google / IBM Image source: Cameron Lonsdale, runc Containers Creating a Container cgcreate -g <controllers>:<path> 1) Create the cgroup with cpu and memory controllers and path = UUID $ UUID = $(uuidgen) $ cgcreate -g cpu,memory:$UUID $ cgset -r memory.limit_in_bytes = 100000000 $UUID $ cgset -r cpu.cfs_period_us = 1000000 $UUID $ cgset -r cpu.cfs_quota_us = 2000000 $UUID $ cgexec -g cpu,memory:$UUID \ > unshare -uinpUrf —mount-proc && \ > sh -c "/bin/hostname $UUID && chroot $ROOTFS $CMD” Containers Creating a Container 2) Set resource limitations for created group $ UUID = $(uuidgen) $ cgcreate -g cpu,memory:$UUID $ cgset -r memory.limit_in_bytes = 100000000 $UUID $ cgset -r cpu.cfs_period_us = 1000000 $UUID $ cgset -r cpu.cfs_quota_us = 2000000 $UUID $ cgexec -g cpu,memory:$UUID \ > unshare -uinpUrf —mount-proc && \ > sh -c "/bin/hostname $UUID && chroot $ROOTFS $CMD” Containers Creating a Container unshare [options] [program [arguments]] 3) Unshare the indicated namespaces that were inherited from the parent process, then execute arguments $ UUID = $(uuidgen) $ cgcreate -g cpu,memory:$UUID $ cgset -r memory.limit_in_bytes = 100000000 $UUID $ cgset -r cpu.cfs_period_us = 1000000 $UUID $ cgset -r cpu.cfs_quota_us = 2000000 $UUID $ cgexec -g cpu,memory:$UUID \ > unshare -uinpUrf —mount-proc \ > sh -c "/bin/hostname $UUID && chroot $ROOTFS $CMD” Containers Creating a Container hostname [-fs] [name-of-host] 4) Change the cgroup’s hostname to the UUID $ UUID = $(uuidgen) $ cgcreate -g cpu,memory:$UUID $ cgset -r memory.limit_in_bytes = 100000000 $UUID $ cgset -r cpu.cfs_period_us = 1000000 $UUID $ cgset -r cpu.cfs_quota_us = 2000000 $UUID $ cgexec -g cpu,memory:$UUID \ > unshare -uinpUrf —mount-proc && \ > sh -c "hostname $UUID && chroot $ROOTFS $CMD” Containers chroot new-root [program [arguments]] 5) Change the cgroup's root to be the Creating a Container subdirectory at new-root, then execute arguments “chroot /Users/cs162 /bin/ls” will change the root to / Users/cs162 and then execute /bin/ls which is actually $ UUID = $(uuidgen) /Users/cs162/bin/ls $ cgcreate -g cpu,memory:$UUID $ cgset -r memory.limit_in_bytes = 100000000 $UUID $ cgset -r cpu.cfs_period_us = 1000000 $UUID $ cgset -r cpu.cfs_quota_us = 2000000 $UUID $ cgexec -g cpu,memory:$UUID \ > unshare -uinpUrf —mount-proc && \ > sh -c "/bin/hostname $UUID && chroot $ROOTFS $CMD” Containers gVisor • A new kind of low level container runtime. App / Container • In addition to creating the container, it also sandboxes System Calls the container as it runs. • Conceptually similar to a user space microkernel • A user space kernel that implements all the Linux syscalls gVisor • Intercepts all syscalls the container makes and performs them in the secure user space kernel instead of in the actual kernel Limited System Calls • Defends against privilege escalation attacks and provides stronger container isolation Host Kernel • Written in Go for memory and type safety • Great for running untrusted applications, i.e. cloud serverless applications Containers gVisor • Intercepts syscalls through ptrace • Linux syscall that allows one process to “control” another • gdb uses it to step through instructors, gVisor to intercept syscalls • Can also use kernel based virtual machines (KVM) • Experimental feature that requires hardware support • Pushes the idea of what is considered a container and what is a virtual machine Containers Microservices + The Cloud Container Orchestration Summary Microservices + The Cloud The Road Ahead • Application packed into containers • Great isolation and resource management • Dependencies packaged into the container makes them easy to deploy • Now we will see how modern “cloud native” applications are architectured and deployed Microservices + The Cloud Historically… Django Users Accounts • One big application did everything: Views • Manager users, accounts, data, etc Computation • Access to database Auth • Manager connections with clients • Perform computation • Using one big application is monolithic architecture DB Microservices + The Cloud Ingress Microservices • As the system itself became more complex, ML Model API Server it becomes harder for one big application to handle everything • Developers began to break out components of the app into separate services, or Users Accounts microservices. • Microservices are a very new variant of service oriented architecture. Batch Compute DB DB Microservices + The Cloud Service Oriented Architecture • SOA is a software design paradigm • Large applications comprised of individual loosely coupled components called services • A service is like an interface • Its implementation is a black box to users of that service • Users just need to understand the service APIs and not its implementation • i.e, a service to manage accounts might have the APIs get_account(), deposit(), withdraw() • Contact the service through its long lived abstract name and its consistent API • http://accounts-service/deposit?account=12345&amount=5&id=67890 • Complex applications strung together from a collection of services Microservices + The Cloud Microservices • A form of service oriented architecture • Loosely coupled • Standardized API contract and independent black box implementations • Long lived hostname to refer to the service • Cloud native - designed to be run in the modern cloud application • Implemented by nearly identical replicas (aside from versioning) • Microservice load balance to one of possibly many replicas • Containerizable • Replicas are packaged as containers • Resource requirements are reasonably definable • Independent - services can be independently developed, tested, deployed, and maintained Microservices + The Cloud Ingress IngressIngress Scalablility • Gives you the ability to horizontally scale ML Model API Server any of the microservices depending on their specific load. • If one service is under heavy load, you can Users Accounts simply run more replicas of that particular Users service. BatchBatch ComputeBatch DB DB ComputeCompute Microservices + The Cloud Independent Implementations • Each microservice can be specialized and implemented with the best language, framework, tools for the job. • The architecture is pluggable. A company does not even need to write all of its services. • Third party applications like MySQL, Elasticsearch, Redis are all microservices that can be easily integrated into your system. DB Microservices + The Cloud Business Organization • Each microservice is independent and can live in its own repository and be owned by a dedicated team of engineers. • Services