<<

Cisco Container Platform for Infrastructure Teams

Justin Barksdale Technical Solutions Architect @3Pings

BRKCLD-2005 Cisco Webex Teams

Questions? Use Cisco Webex Teams to chat with the speaker after the session How 1 Find this session in the Cisco Events Mobile App 2 Click “Join the Discussion” 3 Install Webex Teams or go directly to the team space 4 Enter /questions in the team space

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 3 Deploying production Kubernetes clusters does not have to be hard.

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 5 Agenda

• Containers 101 ()

• Kubernetes

• CCP Architecture

• Container Networking • ACI-CNI

• Hyperflex-CSI

• Cloud

• HX-AP

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 6 Agenda

• Containers 101 (Docker)

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 7 Container Images

Application Code

Container Image

Application Dependencies

Think of Container Images as “golden” templates from which we deploy container instances

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 8 Dockerfile #Base Image FROM nginx:mainline-alpine

# Remove Base conf.d file RUN rm /etc/nginx/conf.d/*

# Add new conf.d file ADD hello.conf /etc/nginx/conf.d/

# Add the index ADD index.html /usr/share/nginx/html/

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 9 Docker Image

~ $ docker build -t 3pings/nginx-hello . Sending build context to Docker daemon 214.5kB Step 1/4 : FROM nginx:mainline-alpine mainline-alpine: Pulling from library/nginx 4167d3e14976: Pull complete db94a93dfca0: Pull complete Digest: sha256:9e81b8f9cef5a095f892183688798a5b2c368663276aa0f2be4b1cd283ace53d Status: Downloaded newer image for nginx:mainline-alpine ---> 48c8a7c47625 Step 2/4 : RUN rm /etc/nginx/conf.d/* ---> Running in 88f85486bb0f Removing intermediate container 88f85486bb0f ---> bd8801ea4219 Step 3/4 : ADD hello.conf /etc/nginx/conf.d/ ---> 8f8b37538099 Step 4/4 : ADD index.html /usr/share/nginx/html/ ---> c487928011dc Successfully built c487928011dc Successfully tagged 3pings/nginx-hello:latest

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 10 Container

~ $ docker history 3pings/nginx-hello IMAGE CREATED CREATED BY SIZE COMMENT c487928011dc 23 seconds ago /bin/sh -c #(nop) ADD file:3ed87b7f3e1501306… 7.19kB 8f8b37538099 23 seconds ago /bin/sh -c #(nop) ADD file:28b8354e8f97bc7e2… 373B bd8801ea4219 24 seconds ago /bin/sh -c rm /etc/nginx/conf.d/* 0B 48c8a7c47625 2 days ago /bin/sh -c #(nop) CMD ["nginx" "-g" "daemon… 0B

~ $ docker run -P -d 3pings/nginx-hello 82dae961268c30c91fa81978bc79c5da436e6e1041843b7b1fd7ce16910ee8d0

~ $ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 82dae961268c 3pings/nginx-hello "nginx -g 'daemon of…" 7 seconds ago Up 6 seconds 0.0.0.0:32770->80/tcp determined_tu

82dae961268c Writeable Container Layer R/W c487928011dc image layer: ADD index.html /usr/share/nginx/html/ 8f8b37538099 image layer: ADD hello.conf /etc/nginx/conf.d/ Image bd8801ea4219 image layer: RUN rm /etc/nginx/conf.d/* 48c8a7c47625 base image: nginx:mainline-alpine

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 11 Agenda

• Containers 101 (Docker)

• Kubernetes

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 12 Comparing Operational Models

10.1.1.5 10.1.1.6 10.1.1.5 10.1.1.6 Nginx Mysql Container: Container: Tomcat Java Nginx Mysql Container: Container: Ubuntu CentOS Tomcat Java VM VM Pod Pod

ESXi Host Kubernetes Node

VMware vSphere Kubernetes

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 13 Single Docker Host

Container: Container: Nginx Mysql Container: Container: Tomcat Java Container Runtime

Host OS Host Hardware

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 14 The need for Orchestration

Container: Container: Container: Container: Container: Container: Nginx Mysql Nginx Mysql Nginx Mysql Container: Container: Container: Container: Container: Container: Tomcat Java Tomcat Java Tomcat Java Container Runtime Container Runtime Container Runtime

Host OS Host OS Host OS Host Hardware Host Hardware Host Hardware

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 15 Orchestration Wars Circa 2015-2016

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 16 Docker vs Kubernetes

Packaging an Application Running an Application

Day 1 Day 2 and Beyond

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 17 What is Kubernetes

Kubernetes provides an API for managing application

• Service discovery and load balancing • Storage orchestration • Automated rollouts and rollbacks • Automatic bin packing • Self-healing • Secret and configuration management

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 18 Kubernetes does not:

• Deploy Source Code • Dictate Logging and Monitoring or alerting solutions • Come with networking • Provide Middleware

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 19 Kubernetes Architecture

Worker Node Kubelet Kube Proxy Kubectl Container Runtime

Master Node Pod Pod Pod

K8s API Server Dashboard Worker Node Scheduler Kubelet Kube Proxy Container Runtime REST etcd Pod Pod Pod API

Controllers Worker Node Kubelet Kube Proxy Container Runtime • Kubectl: CLI-based remote management of Pod Pod Pod K8s cluster • K8s Dashboard: Native K8s UI

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 20 Agenda

• Containers 101 (Docker)

• Kubernetes

• CCP Architecture

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 21 Development

Orchestrator Container Runtime Host

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 22 Production Kubernetes is more than just an Orchestrator Development Production

CI/CD Repository Service Mesh Addons* Logging Monitoring Load Balancing CCP CNI Orchestrator Container Runtime Orchestrator OS Container Runtime IaaS Host Storage

* Addons are available at no additional charge but are not required to be installed. They are supported as part of the platform if they are installed

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 23 Software Layering and CCP Security Scope

End-user Application responsibility

End-user CCP Application Addons Addons Applications CCP packaging & Kubernetes, Docker, Container Kubernetes, Docker, Container Security infra plugins infra plugins responsibility

VMs, Instances, Node OS VMs, Instances, Node OS

Physical Infra , infra e.g. Hypervisor, Virtualization infra e.g. separate vSphere vSphere setup + responsibility Physical Compute, Network, Physical Compute, Network, Storage Storage Control Plane Cluster Tenant Cluster BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 24 Cisco Container Platform

Control Plane Cluster Tenant Clusters

Pod Pod

Pod Pod Ops Ops Cluster 1 Cluster Cluster 2 Cluster Cluster 1 Cluster Cluster 2 Cluster Workloads Pod Workloads Pod HX Connect Cluster/ Operations Automation Machine Orchestration Controllers Cluster 1 Kubernetes Cluster 2 Kubernetes

M M M M Control Plane Kubernetes VM VM VM VM VM VM

M VM VM VM VM VM VM VM

Storage (HyperFlex / VMware) Hypervisor Layer (HyperFlex / VMware)

Compute Hardware (UCS)

Networking (Nexus 9K)

Kubernetes Fluentd Kibana Hyperflex CNI Istio

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 25 Not all workloads created equal!

CPU Intensive Memory GPU Intensive Intensive

• Financial • High paging • 3D Modelling applications Rendering work applications • Apache • In-memory Spark databases • AI / ML • Encoders / Applications decoders with Tensorflow

Kubernetes can manage different types of workloads through tag based node pools

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 26 Node Pools in Cisco Container Platform

• Machines sizes can be different between pools (high CPU or GPU or high memory) • Individual pool can be separately managed (change size, delete) • Planning to add node pool for Kubernetes masters with multi- master support

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 27 Multi-GPU as a Service

Kubernetes Cluster 1 Kubernetes Cluster N Non-GPU Node Pool GPU Node Pool GPU Node Pool … Worker VM Worker VM Worker VM Worker VM Worker VM

HYPERVISOR HYPERVISOR

UCS Server UCS Server GPU GPU GPU

Cisco Container Platform v4.x • Automate AI/ML workload similar to any K8s workload, CCP provides ”Multi-GPU as a Service” • Multiple GPU per worker node • Optimized GPU selection for UCS Servers • GPU passthrough (native performance), • Auto provisioning of Nvidia CUDA drivers, container runtime, NVIDIA device plugin BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 28 Agenda

• Containers 101 (Docker)

• Kubernetes

• CCP Architecture

• Container Networking

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 29 Cisco Container Platform CNI Options

ACI CNI Contiv

Network Policy • K8s network policy • K8s network policy • K8s network policy • ACI policy (EPGs + Contracts) for K8s network policy Underlay Network Integration • Underlay integration with ACI fabric • Policy extends beyond single K8s cluster across VMs, Bare Metal, Multi- clusters Load Balancer Integration • Hardware L3 Load • Software metalLB L3 Load • Software metalLB L3 Load Balancer integrated with Balancer Balancer ACI CNI to provide optimal data path Istio Integration • Istio integration • Istio integration • Istio Integration

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 30 Network Policy Support (all CNIs)

Ingress Example YAML file From kind: NetworkPolicy • ipBlock apiVersion: networking.k8s.io/v1 • podSelector metadata: • namespaceSelector name: -allow • Wildcards - {}, [] spec: Egress podSelector: To matchLabels: • ipBlock app: bookstore • Wildcards - {}, [] role: api ingress: - from: Example Ingress Network Policy - podSelector: matchLabels: Default app: bookstore app: app: bookstore bookstore role: api

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 31 Contiv-VPP Architecture

 Option for complete container networking solution entirely from user space. - enables rapid upgrades, highly available, no kernel tax  Option to replace all eth/kernel interfaces with high performance/scale memif/users space interfaces - high shared memory performance / scale  VPP TCP stack for higher performance (bypass kernel host stack and use VPP TCP stack) - envoy sidecar high performance / scale  Fully supports legacy apps that use the kernel host stack in the same architecture - enhanced monitoring / debugging for existing apps

K8s Master

High Performance Cloud- Contiv Cloud- High Performance Legacy Netmaster Legacy Apps Apps Native VNFs Native VNFs Apps Apps

PodPod PodPod PodPod PodPod PodPod PodPod Pod Pod Pod Kubelet Kubelet Pod Pod Pod Istio Envoy App App VNF Contiv VNF App App Istio Envoy Etcd

VPP VPP Kernel Host stack CNI K8s policy & state CNI Kernel Host stack TCP CRI CRI TCP memif memif Stack tapv2 distribution tapv2 Stack

VPP Agent Agent VPP Contiv vSwitch … Contiv vSwitch IPv4/IPv6/SRv6 Network BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 32 Agenda

• Containers 101 (Docker)

• Kubernetes

• CCP Architecture

• Container Networking • ACI-CNI

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 33 ACI CNI Solution Overview

Network Policy Kubernetes Technical Description

• Network policies of Kubernetes supported using standard upstream format but enforced through OpFlex / OVS using APIC Host Protection Profiles ACI Policies • Kubernetes app configurations can be moved without modification to/from ACI and non-ACI environments • Embedded fabric and virtual switch load balancing • PBR in fabric for external service load balancing • OVS used for internal service load balancing • VMM Domain for Kubernetes • Stats per namespace, deployment, service, pod OpFlex OVS OpFlex OVS • Physical to container correlation Node Node

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 34 Why Use ACI CNI

Fast, easy, Turnkey solution for Flexible policy: Hardware-accelerated: secure and node and container Native platform Integrated load connectivity policy API and ACI balancing scalable policies networking for your Application Container Platform

Visibility: Live statistics in APIC Enhanced Multitenancy and per container and health metrics unified networking for containers, VMs, bare metal

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 35 ACI allows flexible POD to EPG mapping K8s Network Policy

Cluster Isolation Namespace Isolation Deployment Isolation

Kube-default-EPG namespace-PROD-EPG Frontend-EPG API-Gateway-EPG

POD POD POD POD POD POD POD POD POD Contract POD POD POD POD POD POD POD POD POD

Contract Contract Contract POD POD namespace-QA-EPG Backend-EPG Monitoring-EPG POD POD POD POD POD POD POD POD POD POD POD Contract POD POD POD POD POD

• Default behavior: single EPG for • Each namespace mapped to an • Each deployment mapped to an EPG entire cluster user PODs EPG • Contracts control traffic between • No need for internal contracts • Contracts for inter-namespace microservice tiers traffic are required BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 36 Container to Non-Container Communications

• In production environments certain services like high performance databases will be running as VMs or Bare Metal Servers • This calls for the ability to easily provide communication between Kubernetes PODs and VMs/Bare Metal endpoints • Simply deploy a contract between your EPGs, ACI will do the rest! • This works for any VMM domain and Physical Domains, for example you can have a Container Domain using VXLAN speaking with a Microsoft SCVMM Domain using VLAN.

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 37 Lab Architecture

pod1 pod2

pod3 pod4

Control Plane Tenant Master Tenant Worker (10.139.13.50) Ubuntu Ubuntu Ubuntu

Hyperflex

vmwareESXi

ACI Fabric ACI Fabric Nexus 93180 LEAF Nexus 93180 LEAF

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 38 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 39 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 40 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 41 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 42 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 43 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 44 ~9 minutes

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 45 Demo Cisco Container Platform

Control Plane Cluster Tenant Clusters

Pod Pod

Pod Pod Ops Ops Cluster 1 Cluster Cluster 2 Cluster Cluster 1 Cluster Cluster 2 Cluster Workloads Pod Workloads Pod HX Connect Cluster/ Operations Automation Machine Orchestration Controllers Cluster 1 Kubernetes Cluster 2 Kubernetes

M M M M Control Plane Kubernetes VM VM VM VM VM VM

M VM VM VM VM VM VM VM

Storage (HyperFlex / VMware) Hypervisor Layer (HyperFlex / VMware)

Compute Hardware (UCS)

Networking (Nexus 9K)

Kubernetes Fluentd Prometheus Kibana Hyperflex CNI Istio

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 47 Livewall Architecture (app profile)

livewall-weather

pod

livewall-incidents

pod livewall-frontend livewall-api mysql

pod pod VM frontend api mysql-clemea (EPG) (EPG) (EPG)

livewall-events

pod

collector (EPG)

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 48 Agenda

• Containers 101 (Docker)

• Kubernetes

• CCP Architecture

• Container Networking • ACI-CNI

• Hyperflex-CSI

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 49 HyperFlex Storage – Controller VM

running on HyperFlex node which assumes control of local disks through PCI pass-through

CONTROLLER CONTROLLER CONTROLLER VM VM VM HYPERVISOR HYPERVISOR HYPERVISOR disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 50 HyperFlex Storage – Controller VM

• Aggregates available storage into cluster-wide datastores configurable by the HyperFlex admin

CONTROLLER CONTROLLER CONTROLLER VM VM VM HYPERVISOR HYPERVISOR HYPERVISOR disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk DATASTORE(s)

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 51 HyperFlex Storage - IOVisor

• Runs as software VIB inside ESXi and works in conjunction with HyperFlex Controller VMs across entire cluster through high-speed data network to provide NFS target for ESXi hosts

CONTROLLER CONTROLLER CONTROLLER VM VM VM IOVISOR IOVISOR HYPERVISOR HYPERVISOR IOVISOR HYPERVISOR disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk disk DATASTORE(s)

High Speed Data Network

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 52 HyperFlex Storage - IOVisor

• HyperFlex provides NFS datastores to vSphere for storing Kubernetes Node VM “vmdk”

Kubernetes Cluster

Master Node Worker Node Worker Node

CONTROLLER CONTROLLER HYPERVISOR CONTROLLER VM HYPERVISOR HYPERVISOR VM IOVISOR VM IOVISOR IOVISOR

VMDK File VMDK File DATASTORE VMDK File

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 53 HyperFlex Storage for Kubernetes Node VMs

• On the back-end, the “vmdk” files are segmented into blocks and distributed as evenly as possible across all HyperFlex nodes in the cluster

Kubernetes Cluster

Master Node Worker Node Worker Node

CONTROLLER CONTROLLER HYPERVISOR CONTROLLER VM HYPERVISOR HYPERVISOR VM IOVISOR VM IOVISOR IOVISOR

VMDK File VMDK File DATASTORE VMDK File

A C B B A C C B A

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 54 HyperFlex Storage for Kubernetes Node VMs

• The “vmdk” blocks are synchronously replicated within the cluster based on the HyperFlex “Replication Factor”

Kubernetes Cluster

Master Node Worker Node Worker Node

CONTROLLER CONTROLLER HYPERVISOR CONTROLLER VM HYPERVISOR HYPERVISOR VM IOVISOR VM IOVISOR IOVISOR

VMDK File VMDK File DATASTORE VMDK File

A C B B A C C B A B2 A2 A3 A3 C3 B2 A2 C2 C2 C2 B2 C3 C3 B3 A2 B3 A3 B3 Based on cluster-wide Replication Factor RF3 = three copies of data (recommended) BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 55 HyperFlex Storage for Kubernetes Node VMs

• If a node VM were to move via vMotion, it retains access to it’s “vmdk” blocks

Kubernetes Cluster

vMotion MasterMaster VM WorkerWorker 1 VM Worker Node

CONTROLLER CONTROLLER HYPERVISOR CONTROLLER VM HYPERVISOR HYPERVISOR VM IOVISOR VM IOVISOR IOVISOR

VMDK File VMDK File DATASTORE VMDK File

A C B B A C C B A B2 A2 A3 A3 C3 B2 A2 C2 C2 C2 B2 C3 C3 B3 A2 B3 A3 B3

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 56 HyperFlex Storage for Kubernetes Node VMs

• If a host were to fail, node VMs can be restarted and “vmdk” block copies can be leveraged

Kubernetes Cluster

Restart by VMware HA MasterMaster VM WorkerWorker 1 VM Worker Node

CONTROLLER CONTROLLER HYPERVISOR CONTROLLER VM HYPERVISOR HYPERVISOR VM IOVISOR VM IOVISOR IOVISOR

VMDK File VMDK File DATASTORE VMDK File

A C B B A C C B A B2 A2 A3 A3 C3 B2 A2 C2 C2 C2 B2 C3 C3 B3 A2 B3 A3 B3

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 57 HyperFlex CSI Plugin Components

Kubernetes Cluster

Kubernetes Worker Kubernetes Worker Kubernetes Master

csi-nodeplugin-hxcsi(Daemon Set) csi-nodeplugin-hxcsi(Daemon Set) csi-provisioner-hxcsi(Stateful Set) 3 HX HX HX Provisioner Node Driver Registrar Node Driver Registrar Container Container Container Controller Container Container Container 2 2 Pods App App App HyperFlex Pod1 Pod2 Pod3 etcd Pods Pods Storage Class Persistent Persistent Persistent 1• HyperFlexVolume API: Claim automaticallyVolume deployed Claim as part ofVolume HyperFlex Claim cluster API Server 2• HyperFlex NodeKubelet Container: deployed as DaemonSet, instanceKubelet running on each Controller KubernetesSoftware worker iSCSI node Initiator Software iSCSI Initiator 3• HyperFlex Controller Container: deployed as StatefulSet with single replica, one instance per Kubernetes cluster 1 iSCSI iSCSI iSCSI HyperFlex API LUN LUN LUN

HyperFlex Datastore

Cisco HyperFlex

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 58 HyperFlex CSI Plugin Components

Kubernetes Cluster

Kubernetes Worker Kubernetes Worker Kubernetes Master csi-nodeplugin-hxcsi(Daemon Set) csi-nodeplugin-hxcsi(Daemon Set) csi-provisioner-hxcsi(Stateful Set)

HX HX HX Provisioner Node Driver Registrar Node Driver Registrar Container Container Container Controller Container Container Container Pods App App App HyperFlex Pod1 Pod2 Pod3 etcd Pods Pods Storage Class Persistent Persistent Persistent Volume Claim Volume Claim Volume Claim API Server Kubelet Kubelet Controller Software iSCSI Initiator Software iSCSI Initiator

iSCSI iSCSI iSCSI HyperFlex API LUN LUN LUN

HyperFlex Internal Datastore

Cisco HyperFlex

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 59 Cisco Container Platform Kubernetes-as-a-Service

Deploy 1 Tenant Clusters

Control Plane Upstream Cluster Kubernetes Clusters Automatically Deploys & 2 Configures HyperFlex CSI Integration

Cisco HyperFlex

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 60 Message Board Application

message_board:v1

pod

82dae961268c Writeable Container Layer R/W c487928011dc image layer: ADD index.html /usr/share/nginx/html/ 8f8b37538099 image layer: ADD hello.conf /etc/nginx/conf.d/ Image bd8801ea4219 image layer: RUN rm /etc/nginx/conf.d/* 48c8a7c47625 base image: nginx:mainline-alpine

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 61 Message Board Application

message_board:v1 message_board:v2

pod pod

82dae961268c Writeable Container Layer R/W c487928011dc image layer: ADD index.html /usr/share/nginx/html/ 8f8b37538099 image layer: ADD hello.conf /etc/nginx/conf.d/ Image bd8801ea4219 image layer: RUN rm /etc/nginx/conf.d/* 48c8a7c47625 base image: nginx:mainline-alpine

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 62 Message Board Application

message_board:v1 message_board:v2

pod pod

57efu621436f Writeable Container Layer R/W c487928011dc image layer: ADD index.html /usr/share/nginx/html/ 8f8b37538099 image layer: ADD hello.conf /etc/nginx/conf.d/ Image bd8801ea4219 image layer: RUN rm /etc/nginx/conf.d/* 48c8a7c47625 base image: nginx:mainline-alpine

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 63

Agenda

• Containers 101 (Docker)

• Kubernetes

• CCP Architecture

• Container Networking • ACI-CNI

• Hyperflex-CSI

• Cloud

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 65 Cisco Hybrid Solution for Kubernetes on AWS

On-premises Consistent, production-grade environment experience

AppDynamics

CloudCenter Suite Amazon ECR

Cisco Container AWS Identity and Access Amazon Elastic Kubernetes Platform (CCP) Management (IAM) Service (EKS)

Cisco HyperFlex/UCS or other Stealthwatch Cloud Amazon EC2 / Amazon EBS

Cisco Nexus9K/ACI or other Amazon VPC Cisco CSR1000v

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 66 Cisco hybrid architecture for Microsoft Azure

On-premises Consistent, production-grade environment experience

AppDynamics

CloudCenter Suite Azure Marketplace

Cisco Container Identity and Access Azure Kubernetes Platform Management (AD) Service (AKS)

Stealthwatch Cloud HyperFlex | UCS SD-WAN | CSR 1000v Nexus9K | ACI

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 67 Cisco hybrid architecture for Cloud Coming soon

On-premises Consistent, production-grade environment experience

AppDynamics BigQuery Cloud SQL Pub/Sub Big Table Cloud Storage CloudCenter Suite Cloud

Cisco Container Identity and Access Google Kubernetes Platform Management (AD) Engine(GKE)

Stealthwatch Cloud HyperFlex | UCS SD-WAN | CSR 1000v Nexus9K | ACI

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 68 Agenda

• Containers 101 (Docker)

• Kubernetes

• CCP Architecture

• Container Networking • ACI-CNI

• Hyperflex-CSI

• Cloud

• HX-AP

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 69 HyperFlex Application Platform for Containers Turnkey Platform for Application Modernization

Native Kubernetes (100% Upstream) Production grade: hardened, secure, reliable Enterprise ready Full Stack Kubernetes Integrated Management Networking | Storage | Management Monitoring, Telemetry Multi-Tenancy Add-ons Logging | Monitoring | Registry | Service Mesh Lifecycle Operations Physical Infra, VMs, Containers HyperFlex HCI Datacenter & Edge

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 70 HyperFlex Application Platform (HXAP) for Containers

Intersight based “Full-Stack” Management, Intersight SaaS or Monitoring, Telemetry & Multi-Tenancy for On-prem Kubernetes Clusters

Curated, Hardened & Supported Kubernetes with out-of-box add-ons including CSI, CNI, L4/L7 Load Balancer, Service Mesh, Logging/Monitoring, K8s nodes K8s nodes K8s nodes Registry etc. Auth/RBAC integration K8s nodes K8s nodes K8s nodes K8sKubernetes nodes K8sKubernetes nodes K8sKubernetes nodes nodes nodes nodes Enterprise grade compute virtualization with host clustering, resource scheduling, HyperFlex Application HyperFlex Application HyperFlex Application orchestration, HA, live migration etc. Platform Platform Platform Industry leading storage virtualization , distributed persistent storage& data services provided by HX Data Platform

Hyperconvered Infra provided by scalable HyperFlex Cluster HyperFlex Cluster HyperFlex Cluster HX cluster with UCS-FI or Nexus 9K

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 71 HyperFlex Application Platform (HXAP) Architecture

These VMs would run Kubernetes Nodes forming Tenant Kubernetes K8s Tenant cluster 1 K8s Tenant cluster 2 Clusters

Master Worker Worker Worker Master Worker Worker Worker node VM node VM node VM node VM node VM node VM node VM node VM VMs spun up on HXAP cluster with ...... highly available & resilient VM compute & storage resources

HXAP Cluster HXAP compute cluster and HXDP storage cluster together form HXAP HXAP Compute cluster Cluster for HXAP VMs

HXDP Storage cluster HXAP Infra on every node provides highly available compute cluster

HXDP HXDP HXDP HXAP Infra HXAP Infra HXAP Infra Controller Controller Controller HXDP Controller VM on every node provides highly available storage / KVM Linux / KVM Linux / KVM cluster

Each host runs Linux / KVM

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 72 HyperFlex Application Platform (HXAP) Management

Server & Fabric management • Policy driven Multi-site operations K8s Tenant Cluster mgmt. • AI based Recommendation Engine Master/Worker golden Image, • Connected TAC, HCL, Advisories Install/Upgrade, Auto-scaling, Node • SaaS or On-Prem pools, Version match w/ upstream, HXAP Cluster Logging/Monitoring, Interoperability, management Security compliance, RBAC etc. Kubernetes Node VM management K8s-Node (Master/Worker) VM mgmt. K8s Cluster management Tenancy groups for Isolation, Quota & Auth. VM operations including CRUD, Placement, Affinity/Anti-Affinity, HA, K8s Tenant Cluster 1 K8s Tenant Cluster 2 Clones, Monitoring/Telemetry etc.

Master Worker Worker Worker Master Worker Worker Worker ... node VM node VM node VM node VM node VM node VM node VM node VM ... HXAP mgmt including Compute/Storage cluster Install, Upgrade, Expand, Monitor, HXAP Cluster Capacity/Perf reporting etc.

Physical Infra mgmt including BIOS/Boot policies, Firmware, HCL, Networking, security patches etc.

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 73 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 74 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 75 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 76 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 77 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 78 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 80 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 82 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 83 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 84 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 85 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 86 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 88 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 89 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 90 BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 91 Cisco Container Platform

Native Kubernetes (100% Upstream) Direct updates and best practices from open source community

Hybrid Cloud Optimized E.g: Google, AWS…

Integrated Networking | Management | Security | Analytics

Turnkey Solution For Production-Grade Container Flexible Deployment Model Environments VM | Bare metal  HX, ACI | Public cloud

Easy to acquire, deploy and manage | Open and consistent | Extensible platform | World-class advisory and support

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 92 Deploying production Kubernetes clusters does not have to be hard.

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 93 Complete your online session • Please complete your session survey survey after each session. Your feedback is very important.

• Complete a minimum of 4 session surveys and the Overall Conference survey (starting on Thursday) to receive your Cisco Live t-shirt.

• All surveys can be taken in the Cisco Events Mobile App or by logging in to the Content Catalog on ciscolive.com/emea.

Cisco Live sessions will be available for viewing on demand after the event at ciscolive.com.

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 94 Continue your education

Demos in the Walk-in Cisco campus self-paced labs

Meet the engineer Related sessions 1:1 meetings

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 95 Thank you Call to Action

• Learn more about Kubernetes and how it can help you

• It’s easy to get started…dive right in! • Online FREE Kubernetes “playgrounds” available, for example: • https://www.katacoda.com/courses/kubernetes/playground

• Treasure-trove of information available, the native Kubernetes docs are a pretty good place to start… • https://kubernetes.io/

• Be active in the Kubernetes communities, help shape the future!

• Access Cisco DEVNET • https://developer.cisco.com

• Ivan Kovacevic – Walk In Lab LABCLD-2099 “Kubernetes made easy with CCP”

BRKCLD-2005 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 97