http://www.sylabs.io @SylabsIO @SingularityApp Singularity Container Workflows for Compute. Gregory M. Kurtzer CEO, Sylabs Inc. @gmkurtzer INTRODUCTIONS… Gregory M. Kurtzer CEO and Founder, Sylabs Inc. Previously spent ~20 years at LBNL/DOE as the HPC Systems Architect. I’m also known for founding various open source projects like Warewulf, CentOS Linux, and most recently Singularity! APPLICATION CONTAINERIZATION 101 Host 1 Host 2 An environment can be The container image built on one host, can be copied to encapsulated, and SCP, HTTP, another host, and packaged up into a FTP, Archive applications can be container image. executed directly as if they are running native. Applications, Libraries, Services Container Apps, libs, services You can additionally isolate Kernel or integrate the container Kernel environment on the host as CPU Memory Devices CPU Memory Devices the need necessitates. DESIGNED FOR SECURITY, MOBILITY, AND PERFORMANCE Runtime Singularity is differentiated by two Engine primary categories: • Container Format: Sylabs created an image format to encapsulate OCI and Docker based containers, which is single file based, cryptographically signed, trusted, and immutable. • Runtime Engine: Standardizing on existing POSIX security practices, Singularity improves performance, integration, ease of use and reduces attack surfaces while enabling HPC and the growing need for compute based orchestration. Environment Format SINGULARITY IMAGE FORMAT (SIF) Global Header ”Building a container can be done in only 52 lines of Descriptors code!” – Liz Rice, Container Camp 2016 Recipe Definition Labels Environment SIF is a unique, single file, container encapsulation Immutable Runtime format! Container Image CRYPTOGRAPHICALLY SIGNED Signature Block SIF is to containers what RPM and DEB is to source code! Writable Overlay A NEW DELIVERY PARADIGM FOR SOFTWARE Network Registry Root Owned Container SIF encapsulates OCI and Docker containers Daemon into a single file adding benefits such as: Presentation Layer sha256:94061.. sha256:aa74a... sha256:becac… … • Guaranteed immutable and reproducible sha256:94ed0.. Host Operating System • Easy to move, share, archive, etc.. • POSIX compatible • Encapsulates the entire application and environment stack • Cryptographic signatures and validation • No layers or dependencies • No tarballs, SIF is the runtime format TRUST • Encryption (with in-kernel description) coming soon Singularity Container SIF PERFORMANCE Invocation performance over shared storage Objectives: 1.Measure scaling of python startup and import speed with increasing numbers of concurrent python interpreters 2.Compare scaling of a standard python installation with an identical containerized installation Note: Underlying file system is NFS, max jobs was 5120 over 320 nodes, graph is logarithmic on both axis. DR. WOLFGANG RESCH HTTPS://GITHUB.COM/WRESCH/PYTHON_IMPORT_PROBLEM ABSOLUTE TRUST OF ALL WORKLOADS $ singularity sign container.sif Singularity provides absolute trust and $ singularity push container.sif library://user/container accountability Execution of containers can be limited to only valid keys, and/or key fingerprints If a malicious user is found, keys are revoked from the Sylabs $ singularity pull container.sif library://user/container $ singularity verify container.sif KeyStore, limiting exposure Data integrity checked, authentic and signed by: Gregory Kurtzer [email protected], KeyID F4EIAL82E… EXTREME MOBILITY OF COMPUTE – BYOE Absolute mobility from laptop, to HPC, cloud all and the way out to the edge. • Changing the packaging and mobility paradigm for IoT Edge application and data • Disrupts the barriers of portability and bridges the gaps between all available resources • From private resources, to public clouds and all the NVIDIA DGX way out to edge and IoT Local Compute COMPATIBLE AND INTEGRATION AWARE Designed for the complicated integration needs of compute • Container Runtime: • Works on all supported Linux Distributions (runtimes and kernels) • Designed for massive efficiency and performance • Additional support for alignment between user and kernel space • Container Image: • Designed for absolute mobility, user freedom, and reproducibility • Highly performant on shared and parallel file system deployments • Can be easily shared, archived, and controls compliant; containers are just data • Environment: • Optimized for application workflows like MPI and schedulers • Allows direct access to GPUs, InfiniBand, FPGAs, file systems, data, etc. NATIVE HOST INTEGRATION $ singularity exec ubuntu.sif pwd $ singularity exec ubuntu.sif python ./python_script_in_pwd.py $ cat python_script_in_pwd.py | singularity exec docker://python:latest python Data is shared between container and host as fluently as if contained applications were running natively on the host. BATCH SUPPORT #!/bin/sh #SBATCH --N 32 mpirun singularity exec ~/ubuntu.sif mpi_program.exe Singularity integrates with all batch resource managers, with zero modifications, by calling the Singularity command directly within the batch script MPI AND SLURM $ srun -n 32 singularity exec ubuntu.sif mpi_program.exe With a PMIx supporting launcher, you can run a fully contained MPI process directly from a compatible resource manager GPU / CUDA SUPPORT $ singularity exec --nv ubuntu.sif gpu_program.exe $ singularity run --nv docker://tensorflow/tensorflow:gpu_latest When a container includes a GPU enabled application and libraries, Singularity (with the “--nv” option) can properly inject the required Nvidia GPU driver libraries into the container, to match the host’s kernel MVAPICH2 APPLICATION PERFORMANCE Benchmarks published by MVAPICH team at Ohio State University http://mvapich.cse.ohio-state.edu/performance/singularity-application/ IMB NETWORK PERFORMANCE Benchmarks published by SDSC at UCSD https://dl.acm.org/citation.cfm?doid=3093338.3106737 IMB SendRecv Run using Singularity and Non-Singularity IMB PingPong Run using Singularity and Non-Singularity Content published here with explicit permission from the authors OSU NETWORK LATENCY Benchmarks published by SDSC at UCSD https://dl.acm.org/citation.cfm?doid=3093338.3106737 Content published here with explicit permission from the authors LS-DYNA PERFORMANCE Benchmarks published by the Dell EMC HPC Innovation Lab http://en.community.dell.com/techcenter/high-performance-computing/b/general_hpc/archive/2018/02/19/performance-of-ls-dyna-on-singularity-containers “The performance difference while running LS-DYNA within Singularity containers remains within 2%, which is within the run-to-run variability of the application itself..” SECURITY FOCUSED Designed for the security needs of compute • Container Engine: • Singularity has no root owned daemon processes • Implements privilege separation over an API to a secure thread • DoD: Singularity is the only allowed container system • Audited and certified by EU lab for use on the European Compute Grid • NSF grant for 3rd party security assessment (in progress, going well!) • Container: • Singularity containers are immutable • Cryptographically signed and verifiable • Public keys can be managed over standard HKP protocol (or Sylabs key services) • Environment Requirements: • Containers are run as the calling user • Blocks all privilege escalation from within the container SECURITY BLOCKS $ singularity exec centos.sif whoami $ singularity exec centos.sif sudo su - $ singularity exec centos.sif /proc/$$/root/bin/su You are always yourself within a Singularity context, and Singularity will block escalation attempts within the container Even if you know the root password, even if you have sudo installed, even if you implement a SUID hack, Singularity will prevent privilege escalation ADDITIONAL SECURITY FEATURES • System administrators, always in 100% control • Supports User Namespace (when kernel supports it) • Linux Capabilities (per user or group ACLs) • Directly integrates with host’s: • SELinux • AppArmor • Seccomp • Container execution can be limited by: • Container owner or group • Location on file system (trusted paths) • Whitelist/blacklist by signed container finger prints WHAT ELSE IS NEW AND COMING SOON • Backend code updated to GO • Fully OCI compatible (3.1: `singularity oci …`) • Integration with enterprise standards: • OCI: Image support with all container registries • CNI: Support for all container networking options (port forwarding, NAT, etc..) • CGroups: Resource limitations • SIF updates • Encapsulation of OCI and Docker formats • Immutable and 100% guaranteed reproducible • Cryptographically signed and verifiable • No tarballs or archives: SIF is the runtime container format • Multi-stage builds, and “disposal” development overlay • Nvidia HPC-CM container builder • Build tool integration: Spack, EasyBuild,… Docker, Img, Buildah, etc… • Native support for MacOS and Windows (coming soon) • Kubernetes Support (native CRI) NATIVE SINGULARITY SUPPORT ON MACOS Singularity Desktop coming soon (Q1 2019) BRIDGING THE GAP BETWEEN COMPUTE AND SERVICES Native integration between Singularity with OCI, Kubernetes and Nomad to be completed in Q1 2019. ARTIFICIAL INTELLIGENCE MULTISTAGE WORKFLOWS AI workflows typically have a “train” and “execute” workflow, where the training is the most computationally intensive Build Train Singularity enables this workflow and enables large scale distribution and provides the needed assurance, security and accountability for scale and Inference Distribute production EXPANDING THE WORKFLOW SUPPORT OF THE ECOSYSTEM Data • Parallel training