Introduction to Containers and Container Networking
Rohit Agarwalla, Frank Brockners BRKSDN-2115 Cisco Spark
Questions? Use Cisco Spark to communicate with the speaker after the session
How 1. Find this session in the Cisco Live Mobile App 2. Click “Join the Discussion” 3. Install Spark or go directly to the space 4. Enter messages/questions in the space
Cisco Spark spaces will be cs.co/ciscolivebot#BRKSDN-2115 available until July 3, 2017.
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public April, 26 1956 : SS IdealX leaves New York
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 4 BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 5 Containers Spawned A New Eco-System
• Today, 90+% of all cargo ships in a standard container
• Faster delivery, reduced (dis-)charging times, increased security (less loss/damage of goods)
• Average shipment cost reduced from >25% to <3%: Key enabler to globalization and just-in-time production
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 6 March 18, 1982 : chroot() introduced to BSD
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 7 March 2000: FreeBSD Jails introduced February 2004: Solaris Zones introduced
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 8 August 6, 2008: Initial release of LXC
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 9 March 13, 2013: Initial release of Docker
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 10 December 2014: Initial release of rkt
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 11 June 22, 2015: Open Container Initiative launched
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 12 Open Container Initiative
“The Open Container Initiative is an open governance structure for the express purpose of creating open industry standards around container formats and runtime.”
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 14 Cloud Native Computing Foundation “The Foundation’s mission is to create and drive the adoption of a new computing • xxx paradigm that is optimized for modern distributed systems environments capable of scaling to tens of thousands of self healing multi-tenant nodes.”
Platinum Sponsors
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 15 CNCF Hosted Projects
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 17 Container Technology
Docker
Containers on Cisco devices
Container Orchestration and Networking
Summary
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public Linux Containers
• A Linux container lets you run a Linux system within another Linux system. Zones • A container is a group of processes on a Linux machine.
• Those processes form an isolated environment.
• Inside the container, it (almost) looks like a VM.
• Outside the container, it looks like normal processes running on the machine.
• It looks like a VM, but it is more efficient: Containers = Lightweight Virtualization
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 19 Containers and Virtual Machines
App A App A’ App B Containers are isolated but share OS and where Bins/ Bins/ Bins/ VM appropriate bins/libraries Libs Libs Libs
Guest OS Guest OS Guest OS App App App App App App Container A A’ B B’ C’ C’
Hypervisor (Type 2) Bins/Libs Bins/Libs Control Container Host OS Host OS Server Server
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 20 Containers are almost like Virtual Machines
• Containers have their own network interface (and IP address) • Can be bridged, routed... just like with Xen, KVM etc.
• Containers have their own filesystem • For example a Debian host can run Fedora container (and vice-versa)
• Security: Containers are isolated from each other • Two containers can't harm (or even see) each other
• Resource Control: Containers are isolated and can have dedicated resources • Soft & hard quotas for RAM, CPU, I/O... Though…
• Apps in Containers share the kernel of the host OS (i.e. Linux guests only)
• Containers are light-weight, fast to start, allow for >10x density compared to VMs
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 21 Example - Density
• Simulating “The Internet”
• LXC container “the-internet” within which about 250 sub- containers are running, acting as BGP or OSPF routers.
• …on your Laptop!
https://github.com/nsec/the-internet
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 22 Example – Speed Let’s start 10 simple containers…
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 23 10 simple containers started.. in 4 seconds
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 24 Containers: Classes of Use-Cases
Extend Network Web-Scale Network Function Micro-Services Elements’ Virtualization Functionality
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 25 Web-scale Mirco-Service Deployments Web-Scale Spurred Interest In Container Technology Again… Micro- Services • Micro-Service design principles: 12factor.net
• Continuous Integration/Continuous Delivery: • Go from developer’s laptop, through automated test, to production, and through scaling without modification
• Alternative form of virtualization for multi-tenant services
• Scale-out: • Rapidly scale same application across hundreds or thousands of servers…and scale down as rapidly
• Cross Cloud Deployment • Move the same application across multiple clouds (public, private, or hybrid) without modification or noticeable delay
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 26 Network Containers for NFV Function Virtualization • Motivation: Containerize Virtual Network Functions • Light-weight approach to NFV • De-compose network functions into a set of containerized “component VNFs”: Compose use-case specific, optimized VNFs • Leverage alternate orchestration & lifecycle-management solutions
• Considerations • Most existing VNFs have specific kernel dependencies/requirements • Component-VNF concept still evolving – optimization vs. isolation vs. orchestration/management tradeoffs • Security/Isolation • Container & lifecycle management solutions mostly focused on micro-service deployments, less so on NFV/networking applications • NFV focused solution might also require a novel approach to current orchestration, incl. OSS/BSS
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 27 Extend Network Motivation: Containers on Network Elements Elements’ Functionality • Network OS Independence • System Modularity • Limit kernel dependencies
• Enable multiple constituents to provide for device enhancements • Cisco Development, Cisco Services, Partners, Customers, …
• Leverage existing tools/tool-chains and eco-systems • Integration of network device with server-centric tool chains
• Isolation / Security • Application-focused deployment, as opposed to base-OS focused deployment (“escape dependency hell”)
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 28 Qualities Of Containers
• Dedicated namespaces for containers Container 1 Container 2 • Hostname, Process, IPC, FileSystem, Network, User Process Process Process Process Process (subsets are possible)
• Resource Isolation between Process Process Process Process Process container and kernel (cgroups) Container Container File File File System • Shared Kernel System System
• Option: Copy-on-write file- Kernel system to allow for binary reuse between containers
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 29 Container Overview
Container 1 Container 2 Container Management P1 P2 Tools P1 P2
Namespace API/ABI Namespace Set 1 Set 2
Kernel
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 30 Namespaces: Isolate System Resources
• Partition essential kernel structures to create virtual environments. E.g., you can have multiple processes with PID 42, in different environments; E.g., you can have multiple accounts called “frank” in different environments
• Different kinds of namespaces • pid (processes) • mnt (mount points, filesystems) • net (network interfaces, routing...) • user (UIDs) • ipc (System V IPC) • uts (hostname)
• Namespace creation via the “clone()” system call with extra flags • new processes inherit the parent’s namespace • you can attach to an existing namespace (kernel >= 3.8)
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 31 PID Namespace
• Processes in a PID namespace don't see processes of the whole system
• Each pid namespace has a PID #1
• pid namespaces are actually nested Host
• A given process can have multiple PIDs • One in each namespace it belongs to • So you can easily access processes of children namespace
• Can't see/affect processes in parent/sibling namespace
Container
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 32 PID Namespace PID namespace isolates the Process ID, implemented as a hierarchy
PID namespace1 (Parent) ls /proc (Level 0) 1 2 3 4
pid:1 pid:4
P1 pid:2 pid:3 P4
ls /proc pid:1 pid:1 ls /proc 1 P2 P3 1
PID Namespace2 (Child) PID Namespace3 (Child) (Level 1) (Level 1)
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 33 Mount Namespace Each mount namespace has its own file-system layout
/proc/
/ /dev/sda1 / /dev/sda1 / /dev/sda3 /home /dev/sda2 /home /dev/sda2 /boot /dev/sda4
P1 P2 P3
Mount Mount Namespace1 Namespace2
• Mount namespaces are a “deluxe chroot()”
• A mount namespace can have its own rootfs
• Remount special filesystems, e.g. procfs (to see your processes)
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 34 Net Namespace The Net namespace isolates the networking related resources
• Each net namespace has its own… • Network interfaces Net devices: eth0 Net devices: eth1 (and its own lo/127.0.0.1) IP address: 1.1.1.1/24 IP address: 2.2.2.2/24 • IP address(es) Route Route • routing table(s) Firewall rule Firewall rule • iptables rules Sockets Sockets Proc Proc • Communication between containers: sysfs sysfs • UNIX domain sockets … … (=on the filesystem) • Pairs of veth interfaces Net Namespace1 Net Namespace2
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 35 Container Network Models
Shared Dedicated Applications inside the container appear as applications Applications inside the container appear as appliances on a subnet running natively on the host reachable from the host Examples: Nexus 3k, 9k, Cat 3k, 4k, Titanium, NCS xK Examples: ASR 1k, CSR 1kv, ISR4k, ISR 819
Container 1 Container 2 Container 1 Container 2 Network namespace: Host/TPNNS Network namespace: Host/TPNNS Network namespace: N1 Network namespace: N2 Container interfaces Container interfaces Container interfaces Container interfaces
eth0 eth1 eth2 eth0 eth1 eth2 veth0 veth1 veth2 veth0 veth1 veth2
Shared namespace: Linux Bridge Interfaces are directly mapped to container VPG Multiple bridges and virtual topologies possible Host platform Host platform Forwarding Plane Network namespace: Host/TPNNS* Network namespace: Host eth0 eth1 eth2 eth0 eth1 eth2 Physical interfaces Physical interfaces
*Third Party Network Namespace on IOS XR. See also http://www.cisco.com/c/en/us/td/docs/iosxr/AppHosting/AH_Config_Guide/AH_User_Guide_chapter_00.html
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 37 User Namespace
• Kernel-uid (kuid)/Kernel-Group-id (kgid): Original uid/gid, Global
• uid/gid: user id in user namespace, will be translated to kuid/kgid finally
• Only parent User namespace has rights to set map
kernel-uid: kernel-uid: 2000-2004 1000-1009 uid_map uid_map 10 2000 5 0 1000 10 uid: uid: 10-14 0-9
User namespace1 User namespace2
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 38 IPC Namespace IPC namespace isolates the inter-process communication resource (shared memory, semaphore, message queue)
P1 P2 P3 P4 IPC IPC namespace1 namespace2
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 39 UTS Namespace Every UTS namespace has its own UTS related information
ostype: Linux ostype: Linux Unalterable osrelease: 3.8.6 osrelease: 3.8.6 version: … version: …
hostname: cisco1 hostname: cisco2 Alterable domainname: live1 domainname: live2
UTS namespace1 UTS namespace2
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 40 Control Groups (cgroups) Generic process grouping framework (kernel >= 2.6.24)
• Control the resources in a system with many tasks – Linux itself, VMs, containers…
• Subsystems • Group CPU Scheduler • CPU Accounting Controller • Cpuset • Memory • Block IO Controller • Device Whitelist Controller • Freezer • Namespace
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 41 Manage Containers
• Requirement: Userspace container management tool set • Manage containers • Create namespace • Create private filesystem layout for container • Resources controller by cgroup
• Solution options • liblxc • libvirt lxc • libcontainer/Docker
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 42 Container Management – Workflow Example: Docker
Push App A Docker Pull Image Source Bin/Libs Registry Repository Base Libs Update Build Dockerfile App App App A B C
Bin/Libs
Docker Engine Docker Engine
Build Host Target Host
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 43 Containers as a Packaging Mechanism: Build once – Deploy Anywhere
• Containers combine content (the “RPM”) with context (the DependencyDependency Environment DependencyPackagesPackages environment the RPM was Variables Packages built for) Configuration Networking • Containers resolve RPM dependency management Application Process shortcomings Bootstrap Package • How to resolve situations Security where different RPMs require different versions of a Pseudo dependency package? terminals … • RPM built with a different Filesystem toolchain does not guarantee mounts ABI compatibility
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 44 Container World Taxonomy
microservices
• Container Tools • Docker, Rkt, repos/registries applications • micro-OSs – CoreOS, RHEL Atomic, Ubuntu Snappy, ..
• Cluster Control and Services PaaS • Scheduler/Job Monitor – Marathon, Aurora Distributed Scheduler Frameworks • Resource Managers – Mesos, Kubernetes, container / Service Orch/Mgmt Dockerswarms service Cluster Services • Distributed Key/Value/lock managers – management zookeeper, etcd, consul • Service Orchestration/Management Container Tools • Kubernetes, Mesosphere DCOS, Docker IaaS Swarm, HashiCorp Terraform, CoreOS physical & virtual Tectonic cluster nodes
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 45 Container Technology
A glimpse at Docker
Containers on Cisco devices
Container Orchestration and Networking
Summary
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public What is Docker?
• Docker is a container technology similar to Linux Containers (LXC) that… • Provides isolation for application processes from the host processes using Linux namespaces • Provides resource caps for the application using Linux cgroups • Provides industry preferred packaging model using docker images, docker index, and docker registry concepts • Provides the basis for application lifecycle management automation due to good integration with devops automation tools such as Puppet/Chef (not available in 6.1.1)
• A rich repository of certified docker base images are easily available in public as well as private docker registries to cover a variety of application use cases
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 47 When Docker over LXC?
Docker provides the following advantages over LXC containers
• Docker packaging is becoming a packaging model of choice for application packaging and delivery in the industry.
• Standard packaging format provided by docker allows easier integration with devops automation toolchains such as puppet/chef/etc.
• Docker cli provides a git like workflow for developing containerized applications which is tool-friendly
• Docker provides image management capabilities using docker image, docker index and docker registry concepts
• Public and private docker registries provide a repository of starter container images for a variety of application use cases
• Thin packaging/delivery (Docker application download or upgrade is often a thin download due to sharing of common layers among multiple containers)
• Lower foot print for the containers. Docker layers, especially, the base layers can be shared among multiple containers
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 48 Installing Docker
• Docker can be easily installed on a wide variety of platforms (Ubuntu, Mac OS X, RHEL, CentOS and many more).
• Detailed instructions are here: https://docs.docker.com/engine/installation/
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 49 Let’s pull an image from Docker Hub…
Images consist of multiple layers...
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 50 Let’s pull an image from Docker Hub…
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 51 A Docker image is made up of filesystems layered over each other.
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 52 The storage driver is responsible for presenting these layers as a single, unified file system.
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 53 When you start a container, Docker creates an empty, read-write layer on top of the stack – all changes are made in this layer.
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 54 Docker uses “copy-on- write” container layers.
If a file needs to be modified, it is copied into the read- write layer first.
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 55 This means that multiple containers can share a single copy of the image.
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 57 Now that we have an image, what do we do with it?
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 58 Let’s use the image: Start a container and run the bash shell.
/bin/bash
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 59 Interactive Terminal Name of the container Image Command to run
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 60 Let’s start 10 simple containers…
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 62 10 simple containers started.. in 4 seconds
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 63 BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 64 Creating your own images
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 65 Within my simple bash container, we try and run the “traceroute” command, but it isn’t installed:
We could install it inside my container, but as soon as we exit and start another container, traceroute is gone (as we have started a clean container from scratch).
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 66 What if I want to create a new image with the “traceroute” command already installed?
/bin/bash + traceroute
Two options: 1) Commit the container as a new image after making the changes. 2) Use a Dockerfile containing image creation instructions.
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 67 Option 1: Manually compose your container
apt-get update apt-get install inetutils-traceroute
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 68 Option 1: Commit the changes to your container
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 69 Option 1: Make your container available to others: Dockerhub
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 70 Option 1: Once pushed, the new image is available to others
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 71 The second (and recommended) method for building images is to use a Dockerfile.
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 72 Option 2: Let’s create a simple Dockerfile
• A Dockerfile is simply a text file containing instructions on how to build a Docker image – a “Makefile” for container construction.
• Each line in the Dockerfile creates a new image layer
Source Image
Command(s) to execute
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 73 Build the image
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 74 Option 2: Our –v2 Image is now available...
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 75 Option 2: Our –v2 Image is now available...... and we can run it
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 76 Container Technology
Docker
Containers on Cisco devices: IOS-XR
Container Orchestration and Networking
Summary
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public Containers in XR
• 64-bit Open Embedded Linux support Classic XR • Container support XR 6.0 • Standard Linux Toolchains • Third-Party Applications Support System System Control Admin • NCS 5500, NCS 5000, NCS1002, Control Admin XRv 9000, ASR 9000 • LXC and Docker XR 6.1.2 32 bit QNX 64 bit Linux • IOS-XR offers application hosting NPU X86 Hardware • natively on the Host OS (WRL 7) • within a Container (managed through libvirt)
• See also IOS-XR Application Hosting Configuration Guide http://www.cisco.com/c/en/us/td/docs/iosxr/AppHosting/AH_Config_Guide.html
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 78 Third Party Network Name Space (TPNNS)
RPMs
• Secure/Isolated environment with Control Plane Admin Apps Apps 3rd-Party read-access to all interfaces Plane NNS outside XR CLI Mgmt Mgmt Internal Gig Gig • Available to processes in the XR IPC Interfaces TenGig TenGig containers or Third Party containers HunGig HunGig • Routing handled by XR TPNNS 64-bit Host OS Routing Processor
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 79 Third Party Network Name Space
RP/0/RP0/CPU0:ios#run Wed Oct 28 18:45:56.168 IST
[XR-vm_node0_RP0_CPU0:~]$ ip netns exec tpnns bash [XR-vm_node0_RP0_CPU0:~]$ ifconfig Gi0_0_0_0 Link encap:Ethernet HWaddr 52:46:04:87:19:3c inet addr:192.164.168.10 Mask:255.255.255.0 inet6 addr: fe80::5046:4ff:fe87:193c/64 Scope:Link UP RUNNING NOARP MULTICAST MTU:1514 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:3 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:210 (210.0 B)
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 80 Let’s leverage our learnings on Docker in IOS-XR
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 82 IOS XR Docker Architecture – IOS XR 6.1.1
User orchestrates docker containers in XR bash shell
XR LXC Bash Docker Container 1 Docker Container n
Docker Client Binary 1.10 App 1 Binaries … App 2 Binaries
Loopback1
Shared Network Namespace (global-vrf) XR LXC only Storage Volume Docker Storage Pool (upto 1.5 to 2G) libdevmapper/libcontainer
Remote REST API
XR Router
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 84 Linux Build Machine: Compose your App
• Create Dockerfile on a Linux Build Host describing your App
Linux build host BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 85 Linux Build Machine: Compose your App
• Build and image Find existing docker images (initially this will show existing images) • Upload to your repository of choice docker images Using the Dockerfile created earlier build the docker image in the directory (custom or Dockerhub) containing created dockerfile
docker build –t myimage . docker images # should now show myimage
Push docker image to a customer repository
docker push
Alternatively, one can just export the docker image as a tarball. This requires creating a docker container.
docker run --net=host --name=mycont --it myimage /bin/bash docker export --output “myimage”
Linux build host BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 86 XR Router: Install your App using the tarball
• Download image tarball Find existing docker images (initially this will show no images) docker images • User downloads docker image in bash Create a docker image using a tarfile (do “docker export --output “myimage”
up Remove a docker image (must stop and then remove all running containers using that image)
docker rmi myimage docker images “ should show myimage is removed”
XR Router
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 87 XR Router: Install your App using the Docker Registry
• User downloads Find existing docker images (initially this will show no images)
docker image from docker images
registry in bash shell To pull a docker image from a customer’s private registry, download and install ca.crt for the created using bash customer private registry
command mkdir –p /etc/docker/certs.d/
docker pull
Remove a docker image (must stop and then remove all running containers using that image)
docker rmi myimage docker images “ should show myimage is gone”
XR Router
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 88 XR Router: Start your Containers
Find existing docker containers (with their container-id/container-name) • User orchestrates/ manages docker docker ps -a containers in bash Create and start a new docker container (creates, starts and attaches to the container bash)
shell created using docker run --net=host --name=mycont --restart “always” --it myimage bash command /bin/bash Alternatively, create but do not start a new docker container
docker create --net=host --name=mycont --restart “always” --itd myimage /bin/bash
Start existing stopped docker container
docker start -i
Run a new process (e.g., bash shell) in an existing running docker container
docker exec -it|-d
XR Router
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 89 XR Router: Stop and Remove your Containers
• User orchestrates/ Find existing docker containers (with their container-id/container-name) manages docker docker ps -a containers in bash shell created using bash command Stop existing running docker container docker stop
Remove stopped docker container
docker rm
Check that the docker container is completely removed
docker ps -a
XR Router
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 90 XR Router: Debug your Containers
• User debugs docker Check the state of existing docker containers
containers in bash docker ps -a shell created using
bash command Capture docker logs to capture activity that has already occurred
docker logs --details
Capture docker logs to capture future docker activity
docker logs --follow --details
To debug what is happening in a running container, run another bash shell in the existing docker container
docker exec --it
XR Router
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 91 Containers on IOS-XR - Resources
• Cisco IOS XR Application Hosting Configuration Guide for NCS 5000 Series Routers: http://www.cisco.com/c/en/us/td/docs/iosxr/ncs5000/app-hosting/b-application- hosting-configuration-guide-ncs5000/b-application-hosting-configuration-guide- ncs5000_chapter_011.html
• Using Puppet with IOS-XR 6.1.1 https://xrdocs.github.io/application-hosting/tutorials/2016-08-22-using-puppet- with-iosxr-6-1-1
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 92 Container Technology
Docker Containers on Cisco devices: Guestshell – IOS-XE/NX-OS Container Orchestration and Networking
Summary
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public Technology Spectrum
Native TCL/Python Application hosting Guest Native OS Scripting (“agents/plugins) Shell Access
Fully controlled Uncontrolled Environment Environment
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 94 Guest Shell Application Linux Shell Environment
• Maintain NX-OS/IOS-XE system integrity • Isolated User Space • Fault Isolation Linux • Resource Isolation applications • Integrate into your Linux workflow Guestshell • On-box rapid prototyping • Device-level API Integration Open Application Container • Scripting (Python) API • Linux Commands Network OS
• Integrated with NX-OS/IOS-XE
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 95 Entering the Guest Shell Console
Enter Guest Shell
Exit Guest Shell
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 96 Guest Shell Connectivity
default vrf used
Use chvrf to switch to a specific vrf
chvrf
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 97 Run host OS CLI commands dohost
Return code Command output
Return code and output for each command passed in
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 98 Running Guest Shell commands from Host OS
run guestshell
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 99 Package Management – RPM: Install
yum install
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 102 Front Panel Interfaces in Guestshell
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 105 Playing with Front Panel Interfaces
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 106 Python Code Samples https://github.com/CiscoDevNet/python_code_samples_network
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 107 Integration With DevOps Tools
Puppet Master
REST
Puppet Plugin Puppet Agent Puppet Plugin Puppet Agent
Puppet Agent Plugin Container Switch Compute Switch Compute Node Node onePK API • Puppet agent hosted in a container Network OS Chef plugin works in a similar way
• Leverages configuration API to implement configuration change
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 123 Container Technology
Docker
Containers on Cisco devices
Container Orchestration and Networking
Summary
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 131 What is Docker Swarm mode ?
• Cluster management integrated with Docker Engine
• Multi-host networking Container Docker Engine • Declarative service model Docker Host (s) Swarm Manager Swarm Manager DockerDocker Host Engine (s) Swarm(Leader)(Leader) Manager • Routing Mesh (Leader) Swarm Worker (s) • Multi-Container Deployment with Raft Consensus Group Docker Stacks Gossip Network
docker swarm init docker swarm join
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 132 Container Network Model (CNM)
• Proposed by Docker to provide networking abstractions/API for container networking Docker Container Docker Container • Sandbox contains configuration of a Network Network container's network stack (Linux Sandbox Sandbox network namespace) Endpoint Endpoint • An endpoint is container's interface Blue Green Network into a network (veth pair) Network • A network is collection of endpoints that can communicate with each other (Linux Bridge, VLAN) • A container can belong to multiple endpoints (and therefore multiple networks)
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 133 Docker Networking
• Bridge network driver (--driver=bridge)
• None network driver (--driver=none)
• Host network driver (--driver=host)
• Overlay network driver (--driver=overlay) – Multi-Host using VXLAN
• MACVLAN network driver (--driver=macvlan)
• Remote drivers – compatible with CNM • Contiv, Weave, Calico, Kuryr
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 134 Docker Networking Internals Overlay Networking ToR
eth0 Bridge Networking eth0 iptables : iptables : NAT/port-mapping NAT/port-mapping eth0 docker_gw docker_gw iptables : iptables : Gossip NAT/port-mapping NAT/port-mapping veth pairs veth pairs eth0 eth0 eth0 eth0 C1 C2 C4 Docker0 isolatedbridge C3 eth1 eth1 eth1 eth1 veth pairs veth pairs eth0 eth0 eth0 eth0 ov-net1 ov-net2 ov-net1 ov-net2 C1 C2 C3 C4 VTEP exists in the Docker Host’s network namespace
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 135 Docker Service, Tasks and Containers
S Front End Service Uses IPVS Nginx:latest, replicas = 3
C Front End Container C Front End Container C Front End Container nginx:latest nginx:latest nginx:latest
T Front End Task T Front End Task T Front End Task Nginx.1 Nginx.1 Nginx.1
docker service create --replicas 3 --name frontend --network mynet frontend:latest
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 136 Ingress Networks and Routing Mesh
ToR 8080 8080 eth0 eth0 iptables : NAT/port-mapping iptables : NAT/port-mapping
docker_gw docker_gw Gossip veth pairs veth pairs eth0 eth0 eth0 eth0 C2 eth2 Ingress C1 Ingress eth1 eth1 eth1 eth1
Ingress ov-net1 ov-net1 Ingress Ingress
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 137 Routing Mesh
mynet 8080 8080 8080
frontend frontend
Docker Engine Docker Engine Docker Engine
Swarm Worker (s) Swarm Worker (s) Swarm Worker (s)
docker service create --replicas 2 --name frontend --network mynet -p 8080:80 frontend:latest
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 138 What is Kubernetes ?
• Pod serves as core unit of management
• Cluster Networking Container Pod • Replica Sets Kubernetes Master Pod Node(s) • Services Master(s) Kubelet Kube-Proxy Worker Node(s) • Deployments
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 139 Container Network Interface (CNI)
• Proposed by CoreOS as part of appc specification Container Kubernetes, Rocket…
• Common interface between container Network run time and network plugin namespace Container Network Interface • Gives driver freedom to manipulate network namespace Plugins • Network described by JSON config Driver plumbing • Plugins support two commands: - Add Container to Network - Remove Container from Network
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 140 Kubernetes Networking
• Highly-coupled container-to-container communications
• Pod-to-Pod communications • Cilium, Contiv, Contrail, Flannel, Google Compute Engine, Nuage VCS, OVN, Calico, Romana, Weave, CNI-Genie
• Pod-to-Service communications • Cluster IP
• External-to-Internal communications • Node Port • Load Balancer • External IP
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 141 Kubernetes Networking Internals cbro
Container to Container C C 10.0.2.4 C1 Pod C2 C C --net=container:infra --net=container:infra 10.0.1.3 Node --ipc=container:infra --ipc=container:infra Pod
cbro cbro
Infra C C C C 10.0.1.3 10.0.1.4 10.0.3.3 Pod Pod Pod
Communicate using localhost static ports Node Node Shared namespace – IP address, IPC
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 142 Kubernetes Service, Pods, Replication Controllers, Labels
S Front End Service Manages endpoints app=webapp,role=frontend, version=v1
P Front End v1 Pod P Front End v1 Pod P Front End v2 Pod app=webapp, role=frontend, app=webapp, role=frontend, app=webapp, role=frontend, version=v1 version=v1 version=v2
R Front End v1 Controller R Front End v2 Controller
Desired Count = 2 Desired Count = 1
app=webapp,role=frontend,version=v1 app=webapp,role=frontend,version=v2
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 143 Pod to Service Communication
Service IP gets translated to Pod IP and vice versa
eth0 eth0
iptables iptables
cbr0 cbr0 vethxx vethxx
eth0 eth0 C C C C 10.0.3.3 10.0.4.3 Pod Pod Node Node
Service IP is a virtual IP implemented in IP Tables (connection tracking)
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 144 Performance Suffers Using Overlays Host 2 Host 1 Host 2 Host 1 VM1 VM2 VM1 VM2
C1 Cn C1 Cn C1 Cn C1 Cn
Physical Network Hypervisor Hypervisor Overlay Network Overlay Network Overlay Network Overlay Network - VXLAN - VXLAN - VXLAN - VXLAN Virtual Switching or Overlay Network Overlay Network - VXLAN Overlay Network - VXLAN
Physical Network Hypervisor Hypervisor
Physical Network
Encap over encap over encap – Can not leverage performance and performance suffers security by natively integrating with HW
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 145 What is Contiv Any Infrastructure
ACI integration Application Intent Rich Policy Container,VM,BM LDAP/RBAC Connectvity
DevOps IT Admin
Any Platform Any Networking
The Most Powerful Container Networking Fabric 100% Open Source Rich Policy Model L2, L3, Overlay or ACI
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public Introducing Contiv 1.0 What’s New: Commercially Supported Contiv
LDAP+ All New User Kubernetes RBAC Experience 1.4+ Support Cisco Advance and Workflow Services
1 Cisco Technical Docker 1.12+ OpenShift 1.4+ Simple 1-click Services Solutions Support Integration Install Support
Open Source at github.com/contiv © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public Cisco ACI + Contiv (ACI mode)
Container Contiv Master Management Contiv APIC Plugin Docker Solution Highlights
OpenShift • Docker containers/OpenShift pods data plane connectivity is provided by ACI fabric
• ACI policies can be extended across physical, virtual machines, and Docker containers/OpenShift Pods
OVS Contiv Plugin
HYPERVISOR HYPERVISOR HYPERVISOR Container/Pod Host
Unified Policy Automation and Enforcement Across Physical, Virtual, and Containers
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 148 Getting More Information / Getting Started
http://contiv.io/ http://blogs.cisco.com/tag/contiv https://contiv.slack.com/
https://hub.docker.com/u/contiv/ https://github.com/contiv http://cs.co/SBxK8sContiv
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 150 Docker+Cisco Partnership
Joint Engineering, Contiv Docker Datacenter Docker Datacenter Sales and Marketing Network Plugin On FlexPod CVD On Cisco UCS
Stronger Together + Open Source community and technology partners to build solutions
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 151 Summary
• Containers – a lightweight tool for virtualization, rapid development and deployment • Web-Scale MicroServices / NFV / Custom Apps on Network-Device • Efficient tool chains for containers drove acceptance
• Containers on routers and switches • Fast/Custom feature delivery: 3rd-party integration tools (Puppet, Chef, …), Guest-Shell, Your stuff…
• Container networking • CNM and CNI models • Contiv – Most powerful container networking and policy fabric
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 152 Complete Your Online Session Evaluation
• Give us your feedback to be entered into a Daily Survey Drawing. A daily winner will receive a $750 gift card. • Complete your session surveys through the Cisco Live mobile app or on www.CiscoLive.com/us.
Don’t forget: Cisco Live sessions will be available for viewing on demand after the event at www.CiscoLive.com/Online.
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public Continue Your Education
• Demos in the Cisco campus
• Walk-in Self-Paced Labs
• Lunch & Learn
• Meet the Engineer 1:1 meetings
• Related sessions
BRKSDN-2115 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 154 Thank you