An Architectural Perspective for Cloud Virtualization Yiming Zhang Qiao Zhou Dongsheng Li Kai Yu Chengfei Zhang School of Computer School of Computer School of Computer School of Computer School of Computer NUDT NUDT NUDT NUDT NUDT Changsha, China Changsha, China Changsha, China Changsha, China Changsha, China [email protected] [email protected] [email protected] [email protected] [email protected] Yaozheng Wang Jinyan Wang Ping Zhong Yongqiang Xiong Huaimin Wang School of Computer School of Computer Central South University MSRA School of Computer NUDT NUDT Changsha, China Beijing, China NUDT Changsha, China Changsha, China [email protected] [email protected] Changsha, China [email protected] [email protected] whm [email protected] Abstract—Virtual machines (VMs) and processes are two ty [4]. VMs usually emulate existing hardware architectures important abstractions for cloud virtualization, where VMs that run a complete operating system (OS) providing the usually install a complete operating system (OS) running environment for executing various applications in the form user processes. Although existing in different layers in the of user processes. virtualization hierarchy, VMs and processes have overlapped Although existing in different layers in the virtualization functionalities. For example, they are both intended to provide hierarchy, VMs and processes have overlapped functionali- execution abstraction (e.g., physical/virtual memory address ties. They are both intended to provide execution abstraction space), and share similar objectives of isolation, cooperation (e.g., memory address space), and share similar objectives and scheduling. Currently, neither of them could provide the of isolation, cooperation and scheduling. On the other hand, benefits of the other: VMs provide higher isolation, security however, neither of them could provide the benefits of the and portability, while processes are more efficient, flexible and other: VMs provide higher isolation, security and portability, easier to schedule and cooperate. The heavyweight virtualiza- while processes are more lightweight and easier to schedule tion architecture degrades both efficiency and security of cloud and cooperate (with multi-process APIs like fork, IPC and services. signals). Therefore, both VMs and processes are needed by There are two trends for cloud virtualization: the first is to the mainstream software stacks, which not only decreases enhance processes to achieve VM-like security, and the second efficiency but also increases vulnerability [4, 5]. Recently, commodity clouds (like Amazon’s Elastic is to reduce VMs to achieve process-like flexibility. Based on Computing Cloud [6] and Alibaba’s Aliyun [7]) have com- these observations, our vision is that in the near future VMs pletely changed the economics of large-scale computing [8], and processes might be fused into one new abstraction for cloud providing a public platform where tenants run their specific virtualization that embraces the best of both, providing VM- applications (e.g., a web server or a database server) on level isolation and security while preserving process-level effi- dedicated VMs. The highly-specialized VMs require only a ciency and flexibility. We present a reference implementation, very small subset of the overall functionality provided by a dubbed cKernel (customized kernel), for the new abstraction. standard OS. Essentially, cKernel is a LibOS based virtualization architec- This has motivated two trends for cloud virtualization: ture, which (i) removes traditional OS and process layers and the first is to enhance processes to achieve VM-like security, retains only VMs, and (ii) takes the hypervisor as an OS and and the second is to reduce VMs to achieve process-like flex- imitates processes’s dynamic mapping mechanism for VMs’ ibility. For example, picoprocesses [9] augment processes pages and libraries. so as to address the isolation and efficiency problem of conventional OS processes, and Unikernels [4] statically 1. Introduction seal only the application binary and requisite libraries into a single bootable appliance image so that Unikernel VMs Currently, virtual machines (VMs) and processes are two could be lightweight and flexible. important abstractions for cloud virtualization. Hardware Based on these observations, our vision is that in the near virtual machines [1] have been widely used to guarantee future the two abstractions of VMs and processes might be isolation [2, 3] and improve system reliability and securi- fused into one new abstraction for cloud virtualization that embraces the best of both VMs and processes, providing VM-level isolation and security while preserving process- level efficiency and flexibility. We argue that a promising approach for this fusion is to start from the VM abstraction and add desirable processes’ features following the library operating system (LibOS) [10] paradigm, which has been widely adopted in the virtualization literature [4, 11, 9, 12, 13, 14, 15, 16]. We present a reference implementation, dubbed cKernel Figure 1: Conventional VMs/processes vs. cKernel VM (customized kernel), for the new abstraction. cKernel is appliances. essentially a LibOS based virtualization architecture, which (i) removes traditional OS and process layers and retains on- ly VMs, and (ii) takes the hypervisor as an OS and imitates effectively reduce the image size and memory footprint, processes’s dynamic mapping mechanism for VMs’ pages taking the first step towards making a virtual machine more and libraries. By drawing an analogy between conventional like a process. processes and VMs, cKernel has partially (i) realized the Meanwhile, another trend is to improve processes with fork API for VMs, a basis for conventional multi-process VM-like isolation and security. For instance, the picopro- programming abstractions, and (ii) supported dynamic lib- cess [9, 19] is a process-based isolation abstraction that rary loading and linking for the minimized VMs. can be viewed as a stripped-down virtual machine with- The rest of this paper is organized as follows. Sec- out emulated CPU kernel mode, MMU or virtual devices. tion 2 further introduces the background and motivation. Picoprocess-oriented library OSs [12, 15, 13, 14, 16] provide Section 3 introduces a promising approach (cKernel) to re- an interface restricted to a narrowed set of host kernel alizing the fusion of processes and VMs. Section 4 presents ABIs [12], implementing the OS personalities as library cKernel’s reference implementation. Section 5 introduces functions and mapping high-level APIs onto a few interfaces related work. Section 6 discusses future work. And finally to the host OS. Section 7 concludes the paper. For example, Drawbridge [12] refactors Windows 7 to create a self-contained LibOS running in a picoprocess yet still supporting rich desktop applications. Graphene [15] 2. Background broadens the LibOS paradigm by supporting multi-process APIs in picoprocesses. Bascule [13] allows OS-independent 2.1. VMs Vs. Processes extensions to be attached safely and efficiently at runtime. Tardigrade [14] utilizes picoprocesses to easily construct VMs rely on the virtual machine hypervisor, like Xen [1] fault-tolerant services. And Haven [16] leverages the hard- and Linux KVM [17], to provide an application with all its ware protection of Intel Software Guard Extensions (SGX) dependencies in a virtualized, self-contained unit. As shown to achieve shielded execution in picoprocesses against at- in Fig. 1 (left), the hardware dependency is provided through tacks from a malicious host. Similar designs of enhancing the abstraction of an emulated machine with CPU kernel processes include Linux Container [20], Docker [21], Open- mode, multiple address spaces and virtual hardware devices, VZ [22], and VServer [23]. which is able to provide the software dependency by running a conventional operating system together with various lib- 2.2. Vision raries, perhaps slightly modified for para-virtualization [1]. VMs have been widely used for cloud computing since Currently, neither VMs nor processes could provide the they provide desirable compatibility of major applications benefits of the other, and thus both of them are needed by reusing existing OS functions including the support of by the mainstream software stacks. This not only decreases various hardwares. efficiency but also increases vulnerability of cloud services. However, running a conventional OS in a VM brings The two treads (of enhancing processes to achieve VM- substantial overhead, since the guest OS may run many du- like security and reducing VMs to achieve process-like plicated management processes and each guest OS may de- flexibility) have motivated our vision that in the near future mand gigabytes of disk storage and hundreds of megabytes VMs and processes might be fused into one new abstraction of physical memory. This has recently motivated the revival for cloud virtualization, with the goal of making the best of of LibOS for VM appliances [4, 18, 11]. For example, both VMs and processes. Unikernel [4] implements the OS functions (e.g., networking Considering the great success of VMs in security and system and file system) as a set of libraries that can be backward compatibility, in this paper we will discuss a separately linked into the VM appliance at compile time. promising approach for the fusion. We propose to start from The Unikernel (system libraries), application libraries and the VM abstraction, strip off any unused functions