Current Events in Container Storage
Total Page:16
File Type:pdf, Size:1020Kb
Current Events in Container Storage Keith Hudgins Docker 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 1 Container Community Orgs ▪ Cloud Native Computing Foundation (CNCF) ▪ https://www.cncf.io/ ▪ Open Container Initiative (OCI) ▪ https://www.opencontainers.org/ ▪ Both are part of the Linux Foundation ▪ https://www.linuxfoundation.org/ 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 2 Container Runtimes ▪ Docker ▪ Default for most container installs, widest user base ▪ 20 million+ Docker Community Engine installs alone ▪ Containerd ▪ Fully graduated CNCF project (as of Feb 2019) ▪ Windows and Linux container runtime ▪ Upstream of Docker (for Linux, anyway) ▪ CoreOS (rkt) ▪ CoreOS acquired by RedHat May 2018 ▪ rkt archived by CNCF as of 8/16/2019 ▪ CRI-O ▪ Default runtime for RH OpenShift as of 4.0, June 2019 ▪ Intended to be Kubernetes-native runtime ▪ Fully open-source ▪ Upstream of these projects* are CNCF efforts. *Docker has non-CNCF upstream 3 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. Components as well Container Storage Lifecycle 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 4 Container Storage Lifecycle (cont) ▪ Containers use storage in 3 primary contexts: ▪ Raw container image ▪ At rest, either as a file on disk or object storage ▪ At runtime ▪ One of several graph drivers ▪ These are NOT persistent ▪ Persistent storage ▪ Volumes attached to containers for persistent data ▪ Most of this talk will focus on this type of storage 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 5 Graph Driver Overview ▪ Disk image of container at runtime ▪ Most graph drivers are Copy-On-Write (COW) or UnionFS based ▪ For Docker/Containerd, refer to support matrix for which driver to use ▪ https://docs.docker.com/storage/storagedriver/select-storage-driver/ ▪ Intended for images that don’t write to disk much ▪ Can have performance issues if too many writes ▪ Drivers include Overlay2, AUFS, Btrfs, Zfs, LCFS (Experimental) 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 6 Graph Driver Deep Links ▪ https://docs.docker.com/storage/storagedriver/ ▪ https://blog.mobyproject.org/where-are-containerds-graph-drivers-145fc9b7255 ▪ https://integratedcode.us/2016/08/30/storage-drivers-in-docker-a-deep-dive/ ▪ https://github.com/portworx/lcfs ▪ https://docs.docker.com/engine/extend/plugins_graphdriver/ 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 7 What You’ve All Been Waiting For... Persistent storage is, continually, one of the hardest problems to solve in the container space. 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 8 In the Beginning…. OS Containers are NOT NEW. chroot was released in 1979 But when Docker introduced immutable containers in 2013, Everything changed for storage. 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 9 Immutable Containers Each instanced container is meant to be a pluggable component of an application stack. In production, a container tends to spin up and down, as well as move between nodes rapidly depending on orchestration. Not every container will need persistence. But, your data needs to go somewhere! 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 10 Just a quick timeline 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 11 So, how about that data? Prior to pluggable, dynamic volume management: ▪ Data was stored in a bind-mounted volume ▪ Volume was just a directory on disk ▪ Production typically used SAN or NAS directly mounted on the host OS 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 12 Modern container storage For persistent storage, there are two primary interfaces: ▪ Docker Volume Driver ▪ Container Storage Interface ▪ For Kubernetes, there’s also: ▪ Native in-tree plugins ▪ FlexVolume ▪ Both are deprecated-ish in favor of CSI 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 13 Docker Volume Driver Overview ▪ Standard persistence interface for Docker Swarm and Mesosphere ▪ Runs as a container on each host node ▪ Each container implements 2 interfaces: ▪ Orchestrator-facing REST API service ▪ Storage-facing interface ▪ Finally, requires a unique description file (manifest.json) to request privileges and declare requirements for install. ▪ Supports all platforms Docker runs on (Linux, Windows*, i386, ARM) ▪ You will need native drivers for each supported platform ▪ Windows support uses the older, v1 driver API 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 14 Docker Volume Driver Key Items Interface is very stable ▪ No changes since 2017 ▪ No cross-node knowledge ▪ Can cause problems with scaling ▪ Every volume needs to be exposed to every node in each cluster ▪ Docker EE tracks volumes out-of-band based on Volume ID Deep Links https://docs.docker.com/engine/extend/plugins_volume/ https://docs.docker.com/registry/spec/manifest-v2-2/ http://mesos.apache.org/documentation/latest/isolators/docker-volume/ 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 15 Container Storage Interface Emerging standard, becoming the default ▪ 1.0 spec released in late 2018 ▪ Supported by Kubernetes 1.13+, Mesos, and Cloud Foundry ▪ Initially supports basic CRUD operations on volumes ▪ Expanded feature support coming via microservices-like controllers 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 16 What does a CSI Driver Look Like? CSI drivers need 3 fundamental features: ▪ Node service ▪ Controller service* ▪ Identity service All of these communicate via gRPC - https://grpc.io/ * See next slide for caveats 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 17 Breaking things down Controller service is for: ▪ Volume Operations ▪ Create, Delete, List volumes ▪ “Publish” operations ▪ This designates a volume to a given node ▪ Required for scheduling volumes at runtime ▪ If you don’t do volume operations, you can skip the controller service (See iSCSI, NFS drivers) 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 18 Breaking things down 2 Node service is for: ▪ Node operations ▪ Mount/unmount volumes on a node ▪ “Staging” operations ▪ Basically prep ops on a new volume ▪ Create filesystem, etc ▪ Also used in multiple mount scenarios 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 19 Breaking things down 3 Identity service is for: ▪ API grease ▪ GetCapability and GetPluginInfo requests ▪ Should be implemented on both Node and Controller 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 20 So I need two different containers for CSI? Not necessarily, but probably. You can absolutely put Node and Controller ops in the same container if you wish BUT…. Check out what’s new and interesting: 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 21 Proposed CSI Features ▪ Volume Snapshots ▪ Implemented as a separate service (own container) ▪ Dynamic Volume Resize operations ▪ Likewise separate service ▪ Raw Block Volume Support ▪ Windows CSI Support Deep Links https://github.com/kubernetes/enhancements/blob/master/keps/sig-windows/20190714-windows-csi-support.md https://kubernetes-csi.github.io/docs/features.html 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 22 How to Deploy a CSI Driver ▪ This is a possible topology ▪ Driver is deployed via kube application yaml definition ▪ Or Helm Chart ▪ Or Docker Compose ▪ Or…… ▪ Really recommend standard application yaml - it’s universally supported 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 23 CSI Further Reading ▪ https://github.com/kubernetes/community/tree/master/sig-storage ▪ https://medium.com/google-cloud/understanding-the-container-storage-interface-csi-ddbeb966a3b ▪ https://arslan.io/2018/06/21/how-to-write-a-container-storage-interface-csi-plugin/ ▪ https://kubernetes-csi.github.io/docs/ Also, Special Thanks to Deep Debroy and Anusha Ragunathan at Docker for reviewing this talk and keeping me accurate! 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 24 Questions? Please take a moment to rate this session. Your feedback matters to us. 2019 Storage Developer Conference. © Docker, Inc. All Rights Reserved. 25.