• Abstract and Figures
• Public Full-text

Say Goodbye to Virtualization for a Safer Cloud Dan Williams Ricardo Koller Brandon Lum IBM T.J. Watson Research Center, Yorktown Heights, NY When it comes to isolation on the cloud, conven- The main culprit is the hugely powerful kernel tional wisdom holds that virtual machines (VMs) pro- syscall API that containers use to interact with vide greater isolation than containers because of their the host OS. [23] low-level interface to the host. A lower-level interface reduces the amount of code and complexityneeded in the Yet LightVM then goes on to try to introduce container- kernel that must be relied upon for isolation. However, like properties to VMs, without giving up the complex it is incorrectly assumed that virtualization mechanisms hardware-based virtualization mechanism implementing are required to achieve a low-level interface suitable for the low-level interface to guests. isolation. In this paper, we argue that the interface to In this paper, we make the observation that the level the host can be lowered for any application by moving of the interface between the guest and the host is not kernel components to userspace. We show that using a fundamentally tied to the actual mechanism used to sep- userspace network stack results in a 33% reduction in arate the guest from the host (like the virtualization or kernel code usage, which is 20% better than when re- process mechanism). In Section 3, we describe how the sorting to virtualization mechanisms and using a VM. high-level container interface can be arbitrarily lowered by introducing library OS (libOS) components to appli- cations or using userspace services in a manner we dub 1 Introduction microkernelification. While we do not focus on enforc- ing isolation in this paper, we assume guards can protect Cloud security is an ever-present and growing concern a given interface despite the mechanism.1 as more enterprises consider running workloads in the Furthermore, we claim that the mechanism used by cloud. Multi-tenancy brings challenges to security, as virtualization actually hurts isolation due to its complex- cloud users must trust the cloud provider to maintain iso- ity. To back up this claim, we introduce a rough metric lation between their workloads and any potentially mali- based on kernel function tracing, kernel code usage, with cious tenants that are co-located with them on the same the assumption that an interface that requires less kernel physical infrastructure. Currently, running each tenant in code results in greater isolation for tenants. We find that its own virtual machine (VM) is the most common prac- applying libOS techniques to the network stack alone is tice used by cloud providers for isolating tenants. enough to produce a 20% smaller kernel code usage than However, as containers gain popularity as a traditional virtualization, which is 33% smaller than reg- lightweight, developer-friendly alternative to VMs, ular Unix processes. it is reasonable to ask whether VMs are still needed for isolation. Arguments for isolation typically boil down to discussions about differences in the interface between 2 Isolation and Kernel Code Usage the guest and the host, which are often conflated with the mechanism in use. VMs, using hardware virtualization The ability to correctly isolate tenants from each other is mechanisms, utilize a low-level interface which is a fundamental part of designing and operating a cloud. thought to provide strong isolation. Containers, using However, there is no trivial quantitative metric for isola- process mechanisms, utilize a high-level interface, tion. Instead, practitioners often resort to heuristics such which is thought to contribute to poor isolation. For 1For example, using virtualization mechanisms, a monitor may example, the authors of LightVM describe the cause of implement guards, whereas using process mechanisms, system call container security concerns as follows: whitelisting (e.g., seccomp) may implement them. could be used by cloud practitioners to lock down cloud applications to some fixed interface, via guards. In a more extreme case, if the kernel code usage for a given interface was small enough, the cloud provider could im- plement a hypervisor, microkernel [18] or separation ker- nel [25, 27] with formal methods [18, 14] that exposes the desired interface, providing high assurance that the isolation property remains upheld. 3 No Need for Virtualization Mechanisms Figure 1: (a) A process/container typically accesses the host kernel through a high-level interface, such as using In this section, we claim that the interface mechanism its network stack via socket system calls. (b) A VM (i.e., the mechanism used to implement processes vs. uses a low-level interface, such as using a network tap VMs) has little to do with the interface level. Further- device to process layer 2 packets (the TCP stack is im- more, we describe how to arbitrarily lower the interface plemented in the guest kernel). level for any process, thereby reducing the kernel code usage and improving isolation. as how high-level (abstract) or low-level (hardware-like) the interface between cloud applications (guests) and the 3.1 Interface Mechanism 6= Interface Level host is. For example, a process expects a high-level, ab- Processors have evolved their interface mechanism (pro- stract POSIX interface, whereas a VM expects a low- tection features) by introducing protection rings and ex- level virtual hardware interface. ecution modes in order to adapt to new paradigms. Ini- A system that implements a high-level interface hides tially, three protection rings were developed to help im- complexityto the user at the cost of becomingmore com- plement processes [3]. We refer to these as the process plex itself, usually in the form of an increased code base. interface mechanism. Since then, processes have been More code can lead to more errors and vulnerabilities; used to implement containerized applications. In re- the industryaverage is between 15 and 50 errorsper 1000 sponse to the popularity of VMs, a hypervisor mode was lines of code [24, 10]. Conservatively, any vulnerability introduced (VT in Intel [19]), which is used in virtual in the underlying system that provides isolation can lead machine monitors like QEMU/KVM (see Figure 1(b)) to to an exploit that violates the isolation it provides. Thus implement VMs. We refer to this as the virtualization we say a system that can be implemented with less code interface mechanism. Cloud applications run in VMs on (e.g., one that provides a low-level interface) can provide top of a guest kernel which interacts with a monitor pro- greater isolation. cess, QEMU. QEMU interacts with the KVM module Suppose we consider the Linux kernel as the under- in the Linux kernel running in hypervisor mode. Both lying system that provides isolation between tenants in the monitor (via the process mechanism) and the guest a cloud environment. Most familiarly, as seen in Fig- (via the virtualization mechanism) interact with the ker- ure 1(a), Linux provides a high-level POSIX interface nel through low-level interfaces.2 for processes. Alternatively, using the KVM kernel mod- Most likely for historic reasons, people tend to equate ule, Linux can provide a low-level virtual hardware in- the interface mechanism with what it was designed for, terface. Figure 1(b) shows such a case, where the guest specifically, use of the virtualization mechanism tends to application has its own OS and uses low level (hardware- be associated with low-level-interface VMs. However, like) abstractions from the kernel. We define kernel equating the interface mechanism to the interface level or code usage as the number of unique kernel functions that drawing the conclusion that low-level interfaces require are used throughout the execution of cloud applications. the virtualization mechanism is an invalid generalization. This metric providesan estimate of how much of the ker- nel is needed to implement high or low-level interfaces. As described above, we would expect Linux configured 3.2 Lowering Process Interfaces to provide a low-level interface to result in lower kernel Figure 2 shows the expected relationship of interface code usage (and thus greater isolation) than Linux con- level to kernel code usage. Unsurprisingly, we depict tra- figured to provide a high-level interface including system calls that interact with large, complex subsystems like the 2Although out of the scope of this paper, we note that new interface mechanisms are emerging with enclave technologies such as Intel Soft- network stack and filesystems. ware Guard Extensions (SGX) [8]. Our point still applies: the interface In a practical setting, the kernel code usage metric level used on them is completely arbitrary. Figure 3: (a) A libOS can have a low-level interface to the host, without the kernel complexity of a VM. For ex- ample, the application can implement its own user-level TCP/IP library (like lwip [9]). (b) Kernel functionality can be implemented in user-level components, such as Figure 2: The level of interface between the guest and FUSE [2] filesystems through microkernelification. host can be lowered by libOS techniques or microker- nelification, without using virtualization mechanisms. move the interface level down to the machine level by ditional VMs in the lower left, at a point with a low inter- emulating machine instructions without hardware sup- face level and low kernel code usage. Similarly, we de- port (as is possible in QEMU [5]). Full emulation would pict traditional Unix processes/containers as a point with only need to interact with the kernel for I/O, and in that a high interface level and subsequently high kernel code case, it can minimize the amount of kernel code used usage. by only interacting with the lowest interface levels avail- However, process mechanisms can be used with dra- able: network tap and block devices.

Load more

  6

Copy Link