Intra-Unikernel Isolation with Intel Memory Protection Keys Mincheol Sung Pierre Olivier∗ Virginia Tech, USA The University of Manchester, United Kingdom [email protected] [email protected] Stefan Lankes Binoy Ravindran RWTH Aachen University, Germany Virginia Tech, USA [email protected] [email protected] Abstract ACM Reference Format: Mincheol Sung, Pierre Olivier, Stefan Lankes, and Binoy Ravin- Unikernels are minimal, single-purpose virtual machines. dran. 2020. Intra-Unikernel Isolation with Intel Memory Protec- This new operating system model promises numerous bene- tion Keys. In 16th ACM SIGPLAN/SIGOPS International Conference fits within many application domains in terms of lightweight- on Virtual Execution Environments (VEE ’20), March 17, 2020, Lau- ness, performance, and security. Although the isolation be- sanne, Switzerland. ACM, New York, NY, USA, 14 pages. https: tween unikernels is generally recognized as strong, there //doi.org/10.1145/3381052.3381326 is no isolation within a unikernel itself. This is due to the use of a single, unprotected address space, a basic principle 1 Introduction of unikernels that provide their lightweightness and perfor- Unikernels have gained attention in the academic research mance benefits. In this paper, we propose a new design that community, offering multiple benefits in terms of improved brings memory isolation inside a unikernel instance while performance, increased security, reduced costs, etc. As a keeping a single address space. We leverage Intel’s Memory result, the application domains for these minimal, single- Protection Key to do so without impacting the lightweight- application virtual machines are plentiful, encompassing ness and performance benefits of unikernels. We implement cloud- and edge-deployed micro-services/SaaS/FaaS-based our isolation scheme within an existing unikernel written software [6, 21, 22, 30, 44], server applications [22, 28, 29, 44, in Rust and use it to provide isolation between trusted and 53], NFV [12, 29–31], IoT [12, 15], HPC [25], efficient VM untrusted components: we isolate (1) safe kernel code from introspection/malware analysis [52], and regular desktop unsafe kernel code and (2) kernel code from user code. Eval- applications [36, 47]. uation shows that our system provides such isolation with One of the fundamental principles of unikernels is the very low performance overhead. Notably, the unikernel with elimination of the separation between kernel and user parts our isolation exhibits only 0.6% slowdown on a set of macro- of the address space. Thus, a unikernel instance running an benchmarks. application alongside the unikernel’s Library Operating Sys- tem (LibOS) possess a single and unprotected address space. CCS Concepts: • Software and its engineering ! Virtual Because of this particularity, it is possible for unikernels machines; Operating systems; Memory management; • Secu- to present interesting opportunities for performance ben- rity and privacy ! Virtualization and security; Oper- efits, such as replacing traditional system calls with regu- ating systems security. lar function calls [11, 34] so that significant mode switch overhead [45] can be avoided. However, from a security per- Keywords: Unikernels, Memory Protection Keys, Memory spective, a single and unprotected address space makes it so Safety that the entire unikernel must be viewed as a single unit of trust. Subversion of a kernel or application component will ∗Part of this work was done while Pierre Olivier was at Virginia Tech. result in the subversion of the entire unikernel with serious consequences, such as arbitrary code execution, critical data leaks or tampering, etc. VEE ’20, March 17, 2020, Lausanne, Switzerland We argue that the current level of isolation provided by © 2020 Association for Computing Machinery. unikernels is too coarse-grained for many scenarios. First, a This is the author’s version of the work. It is posted here for your personal single application may be made of mutually-untrusting com- use. Not for redistribution. The definitive Version of Record was published ponents [2, 49], such as if they came from different sources in 16th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE ’20), March 17, 2020, Lausanne, Switzerland, https://doi. with variable security coding standards. Second, regarding org/10.1145/3381052.3381326. the LibOS, although some are written in a memory-safe language [9, 24, 29, 51], they generally rely on untrusted VEE ’20, March 17, 2020, Lausanne, Switzerland Mincheol Sung, Pierre Olivier, Stefan Lankes, and Binoy Ravindran components for low level operations by using a traditional that should be enforced by the kernel. Our design allows unsafe language [9, 29, 51] or by using unsafe code blocks protecting the trusted components of the kernel from at- for languages such as Rust. Other unikernel LibOSes are tacks leveraging vulnerabilities in untrusted ones. We also written entirely in an unsafe language [18, 19, 25]. Third, in protect kernel space from unauthorized access by subverted scenarios where mutually untrusting components belong- user code. Our isolation techniques have a low overhead; ing to the same application need to be isolated [2, 16, 49], in particular, we still maintain the low system call latency a computing base that is trusted from the tenant point of feature of unikernels. view has to be established to enforce that isolation. In the current state of the unikernel model, this Trusted Comput- 2 Background and Motivation ing Base (TCB) cannot be the guest kernel as it is not itself In this section, we first describe the motivation for intra- isolated from the application. This implies falling back on unikernel isolation. Next, we give some background infor- the hypervisor to be this TCB, which is suboptimal from a mation about the unikernel we work with in this paper, performance standpoint. RustyHermit [24], as well as the programming language Several isolation mechanisms have been proposed in the it is written in (Rust) and the Intel MPK technology we use past to isolate an application’s untrusting components. They to provide isolation. operate at various levels: using hardware assisted virtualiza- 2.1 Unikernel and Isolation tion [3, 20], running components in different processes/using different page tables [2, 23], or using ISA extensions such A unikernel [29] consists of a single application compiled and as Intel SGX [1, 7]. None of these techniques can be easily statically linked with a minimal kernel LibOS. Unikernels single purpose applied to unikernels without breaking the single address are , i.e. one instance corresponds to one guest space principle and thus not only canceling the resulting VM running a single application on top of a hypervisor. A single and unprotected performance benefits but also introducing non-negligible unikernel instance also presents a address space additional performance overheads in the form of switching shared between the kernel and the application. costs [49]. All the code executes with the highest privilege level (for In this paper, we propose addressing the aforementioned example, ring 0 in x86-64) and thus there is no memory unikernel security issues by providing intra-unikernel iso- protection between kernel and user code/data in that address lation while maintaining the single address space feature space. of this OS model. To do so, we leverage the Intel Memory Such a model brings significant benefits in several do- Protection Keys (MPK) [10] technology. MPK is a new hard- mains [34], in particular in terms of performance [19, 25, 34]: ware primitive that provides per-thread permission control due to the elimination of kernel/user separation, system calls over groups of pages in a single address space with negli- can be replaced with regular function calls. This significantly gible switching overhead [35, 49], making it a compelling reduces system call latency, as there is no longer a costly candidate for use in unikernels. world switch between privilege levels [25]; expensive opera- We identify the various areas composing the address space tions such as page table switching [34] are eliminated. As a of a unikernel, i.e. the kernel’s safe/unsafe memory regions result, unikernels have been shown to outperform traditional (static data section, stack, and heap), and the user memory OSes in system intensive workloads [11]. regions (static data section, stack, and heap). Those areas are However, the lack of isolation within a unikernel (intra- isolated from each other by using MPK-based mechanisms to unikernel isolation) raises serious security concerns. Even enforce per-thread permissions on each memory area. Our if it executes a unique application, viewing a unikernel in- design principles are: it should (1) preserve a single address stance as a single and atomic unit of trust is too coarse- space; (2) isolate the various areas of the address space; and grained in current scenarios: a vulnerability in a relatively (3) have negligible cost. untrusted/vulnerable application component automatically We demonstrate our techniques on RustyHermit [24], a leads to the attacker taking over the entire system. In a unikernel written in Rust. This is done using an efficient unikernel, this concern also include kernel components, as isolation method for intra-unikernel components that relies there is no isolation between kernel and user space. We di- on easy-to-use code annotations made by the unikernel Li- vide intra-unikernel isolation issues into two categories: (1) bOS programmer. On top of that mechanism we implement the lack of isolation between kernel and user space and (2) two isolation policies. First, we isolate safe from unsafe Rust the lack of isolation between trusted and untrusted kernel kernel code so as to limit the possibilities for an attacker components in memory-safe unikernels. to exploit a vulnerability in the unsafe kernel code. Second, Lack of Isolation between Kernel and User Space. Mod- we re-introduce kernel and user space separation by isolat- ern applications are made of components (such as libraries) ing kernel from user code.