Trusted Docker Containers and Trusted VMs in OpenStack Raghu Yeluri Abhishek Gupta Outline o Context: Docker Security – Top Customer Asks o Intel’s Focus: Trusted Docker Containers o Who Verifies Trust ? o Reference Architecture with OpenStack o Demo o Availability o Call to Action Docker Overview in a Slide.. Docker Hub Lightweight, open source engine for creating, deploying containers Provides work flow for running, building and containerizing apps. Separates apps from where they run.; Enables Micro-services; scale by composition. Underlying building blocks: Linux kernel's namespaces (isolation) + cgroups (resource control) + .. Components of Docker Docker Engine – Runtime for running, building Docker containers. Docker Repositories(Hub) - SaaS service for sharing/managing images Docker Images (layers) Images hold Apps. Shareable snapshot of software. Container is a running instance of image. Orchestration: OpenStack, Docker Swarm, Kubernetes, Mesos, Fleet, Project Docker Layers Atomic, Lattice… Docker Security – 5 key Customer Asks 1. How do you know that the Docker Host Integrity is there? o Do you trust the Docker daemon? o Do you trust the Docker host has booted with Integrity? 2. How do you verify Docker Container Integrity o Who wrote the Docker image? Do you trust the image? Did the right Image get launched? 3. Runtime Protection of Docker Engine & Enhanced Isolation o How can Intel help with runtime Integrity? 4. Enterprise Security Features – Compliance, Manageability, Identity authentication.. Etc. 5. OpenStack as a single Control Plane for Trusted VMs and Trusted Docker Containers.. Intel’s Focus: Enable Hardware-based Integrity Assurance for Docker Containers – Trusted Docker Containers Trusted Docker Containers – 3 focus areas o Launch Integrity of Docker Host o Runtime Integrity of Docker Host o Integrity of Docker Images Today’s Focus: Integrity of Docker Host, and how to use it in OpenStack. Trusted VMs - Summary o Launch VMs on Servers that have demonstrated Boot Integrity – Platform Trust Apps Measurements done at the time of Server boot) o Measured Launch of Boot Process/Components vFW VM-2 with Intel TXT. Host OS/Hypervisor o Trust Chain: HW->FW->BIOS->OS/VMM Kernel, Initrd o Tboot What is measured at launch: HW w/ Intel TXT/TPM o Current: F/W, Core BIOS, OS/VMM Kernel, Initrd Measurements match! System trusted o Ext measurements: An7OS/FS modules Trust Boundary o Schedulers/Orchestrators Policy Manager use Trust to launch/create/Migrate VMs. App App o Measurements done at the time of boot Extend Chain of Trust to VMs. (Server boot and VM Launch) o Measure & Attest VM Images prior to Launch. VM-1 VM-2 Trust Boundary o Encrypt VM Images and decrypt based on Platform Trust (Tenant-Controls the Keys) vRTM Host OS/Hypervisor Kernel, Initrd++ o Boundary Control of VMs– Control where your Tboot Trusted VMs are launching and migrating. HW w/ Intel TXT/TPM Measurements match! System & VMs Trusted Will enable the same model and use-cases for Trusted Docker Containers Trusted Docker Containers - 1 Ensure Docker Containers are launched on Trusted Docker Hosts Container B container A Container C e.g. e.g. Apache o Boot-time integrity of the Docker Host e.g. Nginx Apache v2 Shared Bin/Libs o Measured Launch of Boot Process & components with Intel TXT. Docker Daemon o Docker daemon and associated Host OS component added to TCB and Measured. TBOOT TPM HW w/ Intel TXT o Chain of Trust: H/w->FW->BIOS->OS- >Docker Engine Docker Host Platform Integrity o Remote attestation using an Attestation Authority* Trusted Docker Containers - 2 Ensure that Docker Images are not tampered } container Container prior to Launch - Container A B o Launch time integrity of Docker Images C e.g. e.g. e.g. Nginx Apache Apache v2 o Chain of Trust: H/w->FW->BIOS->OS->Docker Engine -> Docker container Shared Bin/Libs layers (apache, Ubuntu14.04, ubuntu14,…, base) o Docker daemon modification: prior to container launch, measure and verify Agents Docker Daemon Docker image (and parent layer graph recursively) Host OS Boundary Control/Geo-Tagging applies equally TBOOT to Docker Containers as well - Compliance TPM HW w/ Intel TXT Needs. Docker Host & Container Launch o Orchestrator determines location/boundary for launching Integrity Docker Images. Exploring: Docker Image encryption & Trust- based Retrieval of Keys – Sensitive Container Images (VNFs, PCI-DSS/HIPPA Containers.. etc) What is measured for Trusted Docker Containers Apache Patch v2 Trusted launch of containerized application Apache Patch v1 Docker Daemon Apache • container management engine (e.g. Docker engine) Ubuntu14.04 • Measurement Agents Ubuntu Containerized application Initrd++ (includes a measurement agent) layers (e.g. Docker image layers) Bootloader, Tboot and OS Kernel launch Bios ChainofTrustextendedtoapplication ACM signed by manufacturer Intel® TXT + TPM What is measured – the details PCR0 + SINIT System Hash + … Power ON PCR0 PCR0 PCR0+ ENTERACCS: PCR17 PCR18 PCR19 PCR19+ LockConfig SENTER ACM Measure UCode Init TXT Lock SINIT Tboot-xm Validates, SMM & Non- Load uCode SINIT Validates, & Mem, TXT & Measures Measures Measures other Critical SINIT & Validates Measures Launch Measures Load Memory OS Kernel Docker BIOS Init Trusted Code OS code SINIT TBOOT OS BIOS ACM SMM Config Initrd++ Engine, other Code Code BIOS OS Option ROMs & other X non-critical modules Measurement Phase 1 (H/W + BIOS) Measurement Phase II (TBOOT, OS, Docker • uCode evals BIOS ACM Engine…) • BIOS ACM (evals BIOS init code) • Boot loader • BIOS • uCode (evals SINIT ACM) • BIOS Option ROMs • SINIT ACM (measures OS Kernel, initrd •Tboot-xm(agent in initrd) measures DockerEngine, other components Source: Intel Who Verifies the Docker Host Trust? Scheduler/Cluster Manager/Policy Manager… Image Docker Principles Of Operation Agents Registry Engine OS, Initrd++ o Cluster Manager determines best TPM v1.2 hosts in the cluster, based on Trusted utilization, type, location Scheduler/Cluster Manager Host compliance.. etc. Trust Filter Docker o (for this host list) Cluster Manager Agents Engine verifies Host Integrity with the OS/initrd+ Examples Attestation Authority. TPM v1.2 Attestation • OpenStack Traffic o Attestation Authority responds with • Docker Trusted Swarm Host Attestation Reports for the Hosts • Kubernetes Remote Docker o • Attestation Agents Cluster Manager picks best Server Mesos API Engine • Fleet Attestation OS,Initrd+ that has the Integrity and Authority instantiates Containers. TPM v1.2 Trust Not Verified. Trusted Docker Containers & VMs with OpenStack 5 Nova + Docker Nova Docker Agents +Agents Engine Engine Glance OS, Initrd++ OS, Initrd++ Nova Scheduler TPM v1.2 TPM v1.2 2 Trusted Trusted API Server ImageProp Filter Host Host Location Trust 3 Filter Filter 1 4 VM1 VM2 5 Nova + Nova + Remote Qemu Agents Qemu Trust VM launch Attestation Agents API Attestation OS OS Horizon Trusted Container Authority TPMTPM v1.2 v1.2 TPMTPM v1.2v1.2 Launch (OAT) Trusted Trust Not Verified. Host Horizon/API Server : Initiate Launch of Image (with 1 4 Attestation Authority: Challenges Host to Attest. Hypervisor_Type Property) Provides Signed Attestation Report to Scheduler to use. – Identifies Trusted Host for VMs or Docker Containers. 2 Nova Scheduler: ImageProp Filter excludes Hosts that don’t met Image Hypervisor Type. Nova Compute: Download Glance Image and Launch. 5 For Docker Images: Nova uses DockerDriver to download, and loaded 3 Nova Scheduler: Runs Trust/Location Filter to identify Trusted Host (for VM or Docker Container) to Docker File system with Docker load Command. [ Changes needed in OpenStack Infrastructure OpenStack changes Docker Specific changes 1. Add hypervisor_type property to images For Docker Image Integrity: Value=qemu for VM images o Modified Docker daemon to intercept Value=docker for docker images container launch request and call 2. Activate ImageProperties filter measurement agent before launch filters out hosts that don’t match Value from Image o Hypervisor Type Manifest/trust-policy created and associated with each Docker layer 3. Activate Trust filter in openstack scheduler and trust properties in images 4. Configure Nova-compute to use docker driver. DEFAULT] compute_driver = Infrastructure related changes novadocker.virt.docker.DockerDriver o TXT/TPM hardware; Steps at: https://wiki.openstack.org/wiki/Docker) o TXT/TPM activation on the clusters o Attestation Server is setup Demo Summary & Call to Action o Intel’s focus: Enable Hardware-based Integrity assurance for Docker Containers – Trusted Docker Containers o Enabling the same model as we have done for VMs. o Intel TXT and Attestation Software becomes the foundation for asserting Docker Host Integrity.. o Intel iKGT (Kernel Guard Technology) can help in runtime integrity protection of the Linux Kernel. o OpenStack can launch VMs and Containers with the extensions that are already mainstream (Trusted Compute Pools) o Get engaged, get started with Trusted VMs and OpenStack. Extensions to OpenStack for Trusted Docker containers, will be available in Q3 timeframe. o iKGT is available now on 01.org. Download it and try it out. Q & A.