NCC Group Whitepaper Understanding and Hardening Linux Containers June 29, 2016 – Version 1.1 Prepared by Aaron Grattafiori – Technical Director Abstract Operating System virtualization is an attractive feature for efficiency, speed and mod- ern application deployment, amid questionable security. Recent advancements of the Linux kernel have coalesced for simple yet powerful OS virtualization via Linux Containers, as implemented by LXC, Docker, and CoreOS Rkt among others. Recent container focused start-ups such as Docker have helped push containers into the limelight. Linux containers offer native OS virtualization, segmented by kernel names- paces, limited through process cgroups and restricted through reduced root capa- bilities, Mandatory Access Control and user namespaces. This paper discusses these container features, as well as exploring various security mechanisms. Also included is an examination of attack surfaces, threats, and related hardening features in order to properly evaluate container security. Finally, this paper contrasts different container defaults and enumerates strong security recommendations to counter deployment weaknesses– helping support and explain methods for building high-security Linux containers. Are Linux containers the future or merely a fad or fantasy? This paper attempts to answer that question. Table of Contents 1 Introduction ................................................................................ 5 1.1 Motivation ........................................................................... 6 1.2 Virtualization Background ............................................................. 7 1.3 Benefits of An OS-Virtualization System ................................................. 10 1.4 Drawbacks of an OS-Virtualization system ............................................... 11 2 Linux Containers Overview .................................................................. 13 2.1 A Brief History of OS Containers ....................................................... 13 2.2 Linux Containers: where are they now? ................................................. 13 2.3 Prior Art: Linux Container Security, Auditing and Presentations ............................ 15 2.4 TL;DR Linux Containers ............................................................... 18 3 Namespaces ................................................................................ 20 3.1 Namespaces Background ............................................................. 20 3.2 Namespaces Implementation .......................................................... 20 3.3 Mount Namespace ................................................................... 21 3.4 IPC Namespace ...................................................................... 21 3.5 UTS Namespace ..................................................................... 22 3.6 PID Namespace ...................................................................... 22 3.7 Network Namespace ................................................................. 23 3.8 User Namespace ..................................................................... 26 4 Control Groups ............................................................................. 27 4.1 Cgroups Background ................................................................. 27 4.2 Working with Vanilla cgroups .......................................................... 28 4.3 Containers and cgroups .............................................................. 29 4.4 Future of cgroups .................................................................... 29 5 Capabilities ................................................................................ 30 5.1 Capabilities Background .............................................................. 30 5.2 Additional Introductory Resources ..................................................... 31 2 | Understanding and Hardening Linux Containers NCC Group 5.3 Understanding Capabilities ........................................................... 31 5.4 Exploring Capabilities ................................................................ 37 5.5 Capabilities and User Namespaces ..................................................... 39 5.6 Capability Defaults In Modern Containers ............................................... 40 5.7 A World Without Root ................................................................ 42 6 Configuration and Basic Use ................................................................ 43 6.1 LXC ................................................................................. 43 6.2 Docker .............................................................................. 45 6.3 CoreOS Rocket ...................................................................... 46 7 Understanding Container Threats ............................................................ 49 7.1 The Linux Kernel Itself ................................................................ 49 7.2 Exploring Container Threats ........................................................... 51 7.3 LXC Specific Threats .................................................................. 61 7.4 Docker Specific Threats ............................................................... 62 7.5 CoreOS Rkt Specific Threats ........................................................... 63 7.6 Indirect or Unexpected Threats ........................................................ 64 8 Recent Security Advancements .............................................................. 66 8.1 The User Namespace ................................................................. 66 8.2 Mandatory Access Control ............................................................ 69 8.3 Syscall Filtering with Seccomp ......................................................... 74 9 LXC, Docker and CoreOS Rocket ............................................................ 82 9.1 LXC ................................................................................. 82 9.2 LXC Background ..................................................................... 82 9.3 LXC Components .................................................................... 83 9.4 Brief LXC Security Analysis ............................................................ 83 9.5 Docker .............................................................................. 85 9.6 Docker Background .................................................................. 85 9.7 Docker Components ................................................................. 86 3 | Understanding and Hardening Linux Containers NCC Group 9.8 Brief Docker Security Analysis ......................................................... 87 9.9 CoreOS Rocket ...................................................................... 92 9.10 CoreOS and Rkt Background .......................................................... 92 9.11 Rkt Components ..................................................................... 93 9.12 Rkt Security Analysis .................................................................. 94 9.13 Container Defaults ................................................................... 97 10 Security Recommendations ................................................................. 98 10.1 Generation Container Recommendations ............................................... 98 10.2 LXC Specific Recommendations .......................................................105 10.3 Docker Specific Recommendations .....................................................106 10.4 CoreOS Rkt Specific Recommendations ................................................109 10.5 Relevant Kernel Hardening ............................................................110 11 The Future .................................................................................. 113 11.1 Containers on the Desktop ............................................................113 11.2 New Potential Namespaces ............................................................114 11.3 Additional Lightweight Isolation and Sandbox Platforms ..................................114 11.4 The Open Container Initiative..........................................................116 11.5 Containers In Other Platforms .........................................................117 11.6 Container Specific Operating Systems ..................................................117 11.7 Unikernels and Microhypervisors and Hybrid Models .....................................118 11.8 The Big Idea Of Microservices .........................................................120 12 The End .................................................................................... 122 12.1 Conclusion ..........................................................................122 12.2 Acknowledgements ..................................................................122 12.3 About The Author ....................................................................123 4 | Understanding and Hardening Linux Containers NCC Group 1 Introduction ``I am large, I contain multitudes'' – Walt Whitman Linux containers have recently developed into a production-ready technology for OS level virtualization,1 yet very little security research or best practices have been made public, and concerns for deployment2 and security3, 4 abound. This whitepaper seeks to expand on what little security information exists on what con- tainer technologies are capable of and can ultimately provide. This paper starts by examining how