Spice up Your Workloads with Kata Containers

Spice Up Your Workloads With Kata Containers

Ricardo Aravena branch.io @raravena80

@raravena80 Who Am I?

Work @ Branch Metrics Cloud Ops Kata Containers contributor Slack: rico

@raravena80 Containers @ Branch

DB & Big data apps 5 Standalone Runtime 4 Docker/containerd/runC Mesos qa/dev/prod Data pipelines 3 K8s in qa/dev/prod Spark/Flink/HDFS/Druid API services - Lots of 2 different languages Where? 1 Everywhere

@raravena80 https://branch.io/ Outline

● What Kata ● Why

● API Services ● Credentials Store Workloads ● Databases ● Big Data

● Containerd ● Docker Kubernetes ● CRI-O ● Serverless

● Hotplug ● GPUs Future ● Hypervisors ● Blockchain ● GPUs ● Blockchain @raravena80 Traditional Containers Container A Container B Container C

App A App B App C

Middleware A Middleware B Middleware C

Linux* Kernel Virtual Machine Linux Kernel Server Hardware @raravena80 https://katacontainers.io Speed vs Security

Isolation

Speed

@raravena80 https://katacontainers.io Kata Containers

Container A Container B Container C

App A App B App C

Middleware A Middleware B Middleware C

Linux Kernel A Linux Kernel B Linux Kernel C

Virtual Machine Virtual Machine Virtual Machine Linux* Kernel Server Hardware @raravena80 https://katacontainers.io How does it work? http://bit.ly/kata-containers-2-pager

Kata Containers Traditional Containers Each container or container pod isolated in its Prone to exploits own lightweight VM Requirements

● Bare Metal Machine ● Nested Virtualization

OS ● Linux

● GCP - Nested Public Cloud ● Azure - Nested ● AWS - i3.metal instances

Private Cloud ● Openstack ● Bare-metal & Nested Virt Providers

Platform ● Kubernetes - CRIO/Containerd ● Docker

@raravena80 https://katacontainers.io Installation

Requirements ● $ kata-runtime kata-check

● Kata Deploy @egernst Kubernetes ● github.com/egernst/kata-deploy

Docker ● Kata Installation guide

@raravena80 https://katacontainers.io Kata Workloads

APIs or Microservices

Credentials Store

Databases

Kubernetes

Big Data & Analytics

@raravena80 Workloads

APIs or Microservices

@raravena80 Microservices

● React.js Backend for web apps ● Angular ● Vue.js

● Rails (Ruby) ● Django (Python) MVCs ● Play! Framework (Java, Scala) ● Laravel (PHP)

● gRPC or Rest with GoKit High performance APIs ● go-micro ● gin-gonic

@raravena80 Microservices

OR µService

Load Database Balancer µService

User

µService

@raravena80 Microservices

µService 1 µService 2

Load Load Balancer Balancer

User µService 1 µService 2

@raravena80 Workloads

Databases

@raravena80 Databases

What? ● Relational ● NoSQL ● Containerized

Why? ● Data Protection ● Compliance ● Memory Management

Advantages ● Easy to set up ● Upgrades ● Better resource utilization

@raravena80 Databases

● MySQL Relational DBs ● PostgreSQL ● CockroachDB

● Redis NoSQL DBs ● CouchDB ● MongoDB

● Cassandra Cluster KV DBs ● Scylla ● Hbase

@raravena80 https://landscape.cncf.io/grouping=landscape&landscape=database-and-data-warehouse Relational DBs

Master MySQL Master Disk

µService

MySQL Slave Slave Disk

@raravena80 DB Cluster

Cassandra - Node 5 OR

Cassandra - Node 6 Cassandra - Node 4 OR

Disk 6 Disk 4 Disk 5

Cassandra - Node 1 Cassandra - Node 3

Cassandra - Node 2 Disk 1 Disk 3

Disk 2 @raravena80 https://github.com/scylladb/scylla https://cassandra.apache.org/ Workloads

Credentials Store

@raravena80 Credentials Store

Why? ● Centralized credentials ● Microservices auth ● Compliance

Encryption ● In transit ● At rest

Solutions ● Vault (Hashicorp) ● KeyWhiz (Square) ● Knox (Pinterest) ● Confidant (Lyft) ● SPIFEE/SPIRE

@raravena80 Vault

What? ● Manage secrets ● Audit logs ● Microservice Auth

Architecture ● Redundant ● Active/Standby ● Client Server

Features ● Multiple Storage Options ● API Keys ● Employee credentials

@raravena80 Credentials Store

OR Storage Vault Standby TLS

User

TLS TLS Vault Active

TLS

Vault Standby @raravena80 Demo

@raravena80 Demo

Vault cluster in Kata

Create Credentials

Retrieve Credentials

Security in VM

@raravena80 Demo

Vault with Consul

https://github.com/raravena80/oss-vaultsa

Vault Operator

https://github.com/kubevault/operator

@raravena80 Workloads

Kubernetes

@raravena80 Kubernetes

● CRIO How? ● Containerd

● Bare Metal Requirements ● Nested Virtualization ● QEMU

● Standalone Microservices ● Service Mesh ○ Istio ○ Conduit

@raravena80 Kubernetes

Istio Pod Leader Kubelet

Svc Pod CRI

Svc Pod

Kubelet

CRI Svc Pod

K8 Masters Svc Pod

@raravena80 Kubernetes

Istio Pod

Kubelet

Svc Pod CRI Load

Svc Pod Balancer User

Svc Pod

Kubelet

CRI Svc Pod

Svc Pod

@raravena80 Kubernetes CRIO

apiVersion: v1 kind: Pod metadata: name: nginx annotations: io.kubernetes.cri-o.TrustedSandbox: "false" labels: env: test spec: containers: - name: nginx image: nginx imagePullPolicy: IfNotPresent nodeSelector: kata-runtime: "true"

@raravena80 http://cri-o.io Kubernetes Containerd

apiVersion: v1 kind: Pod metadata: name: nginx annotations: io.kubernetes.cri.untrusted-workload: "true" labels: env: test spec: containers: - name: nginx image: nginx imagePullPolicy: IfNotPresent nodeSelector: kata-runtime: "true"

@raravena80 https://containerd.io/ Demo

@raravena80 Demo

K8s Cluster

Run Stateful App with Containerd Run Trusted With Untrusted Workloads

Add Sample Data

@raravena80 Workloads

Big Data & Analytics

@raravena80 Big Data & Analytics

Why? ● Containerized workloads ● Tie to Kubernetes Experimental

Security ● Data In transit ● Data at rest

Applications ● Spark ● Flink ● Hadoop

Workloads ● Streaming ● Batch

@raravena80 Spark & Flink Streaming

What? ● Real-time processing ● Large amounts of data Experimental

Workloads ● Ingest from Kafka ● Process and change data

Storage ● Object Storage ● Columnar Storage (Druid/Vertica)

@raravena80 Spark & Flink Streaming

Spark Master

Kubelet

Spark Worker Experimental CRI Stream

Spark Worker

Kubelet

CRI Spark Worker

Spark Worker

@raravena80 https://flink.apache.org/ https://spark.apache.org/ Apache Kafka

What? ● Streaming Platform ● Real-time Processing Experimental ● Geo-replication

Architecture ● Distributed Cluster / Data Partition ● Producers ● Consumers

Use Cases ● Website Activity Traffic ● Events Tracking ● Metrics

@raravena80 https://kafka.apache.org/ Big Data & Analytics

Fast Storage Kubelet Kafka Node CRI

Experimental Node 1

Publish

Fast Storage Kubelet Kafka Node CRI

Node 2 Subscribe

Fast Storage Kubelet Kafka Node CRI

Node 3

@raravena80 https://kafka.apache.org/ Apache Druid

What? ● High Performance ● Distributed Data Store Experimental ● Columnar Based

Architecture ● Different Node Types ● Data Indexing in Hadoop or Local ● Fast Real Time Queries

Features ● Different Storage Options ● Data Compression ● Different Data Formats (parquet, avro, etc)

@raravena80 http://druid.io Big Data & Analytics

Kubelet Overlord CRI

Overlord Node Kubelet Middle Manager Experimental CRI

Kubelet Coordinator CRI Indexing

Coordinator Node Middle Manager Nodes

Kubelet Broker Kubelet Historical Fast Storage User CRI CRI Indexing

Broker Nodes Historical Nodes

@raravena80 http://druid.io Future

@raravena80 Kata Containers

Hypervisors ● Hyper-V ● VMware ● Xen

Public Cloud ● AWS Nested Virtualization ● More Bare Metal Offerings ● GKE?, AKS?, ACS?

Features ● VM Hotplug CPUs, Memory, Network ● Faster Shim Implementation

@raravena80 http://druid.io Workloads

NFV ● Multus ● https://github.com/intel/multus-cni

Edge Computing ● 5G ● IoT

AI/HPC ● Tensorflow / Kubeflow ● GPUs ● ML model training

@raravena80 http://druid.io Resources

Kata Containers ● https://katacontainers.io

Microservices ● http://microservices.io/

Hashicorp ● https://github.com/hashicorp

CNCF ● https://github.com/cncf/landscape

Kafka ● https://kafka.apache.org/

Druid ● http://druid.io

@raravena80 Thank You!

@raravena80