A Way Forward: Enabling Operating System Innovation in the Cloud Dan Schatzberg, James Cadden, Orran Krieger, Jonathan Appavoo Boston University 1 Introduction plication. In such an environment, three of the ma- jor objectives that general purpose operating sys- Cloud computing has not resulted in a fundamental tems were designed to meet are relaxed or elimi- change to the underlying operating systems. Rather, nated entirely. distributed applications are built over middleware that provides high-level abstractions to exploit the First, the burden to support multiple users is re- cloud’s scale and elasticity. This middleware con- moved from the operating system. In this environ- joins many general purpose OS instances. ment, the isolation enforced by the IaaS provider Others have demonstrated that a new operating eliminates the need for many system level security system built specifically for the cloud can achieve checks and accounting, and reduces the requirement increased efficiency, scale and functionality [11, for internal barriers between trusted and untrusted 14]. However, this work does not take into account code. the way applications are being deployed in cloud environments. In particular, entire physical or vir- Second, it becomes the responsibility of the tual machines are being dedicated to run a single ap- IaaS provider to arbitrate and balance competi- plication, rather than concurrently supporting many tive resource usage. In a deployment where entire users and multiple applications. nodes are assigned to a single application, much of In this paper we introduce a new model for dis- the complexity of existing operating systems (e.g., tributed applications that embraces a reduced role scheduling, memory management, etc.) is redun- of the OS in the cloud. It allows for the construction dant. of application-driven compositions of OS function- ality wherein each application can employ its own Third, a symmetric structure is unnecessary in customized operating system. a large-scale distributed application. Many cloud applications are already composed of multiple ser- 2 Role of the OS vices run across a set of compute nodes; As a result, OS functionality can be provided asymmetrically, For security and auditability, Infrastructure as a Ser- where only some nodes need full OS functionality, vice (IaaS) providers isolate their tenants at a very while other nodes can be much simpler. low level as physical or virtual compute nodes. In- dividual tenants own and manage their compute Given these observations, it is apparent that dis- nodes, software stack, networks and disks within an tributed cloud applications built on top of general IaaS cloud. purpose systems are comprised of unnecessary soft- Typically, scale-out cloud applications run across ware functionality with the risk of reduced perfor- a set of compute nodes solely dedicated to that ap- mance and added complexity. 1 3 A Way Forward For example, consider a typical web applica- tion comprised of four standard components: front- The reduced role of the operating system in the end Apache [4] servers, Java business logic, mem- cloud suggests a new way forward for providing cached [5] instances, and a database server. A typ- OS functionality and further optimising application ical deployment might distribute these components performance. Since we do not require the same across multiple nodes that each run the same under- OS functionality on all nodes, it becomes possible lying operating system. to combine general purpose operating systems with As illustrated in Figure 1, a MultiLibOS model specialized operating systems. Since we do not have can run Apache and the database on nodes run- to support multiple users or multiple applications ning Linux. Meanwhile, the Java business logic and on a single node, new OS functionality can be pro- memcached servers can run on their own highly spe- vided by application-specific library OSs [3] linked cialized library operating systems. The nodes ded- directly into the application’s address space. icated to these tasks can be rapidly added and re- We propose that operating system functionality moved in response to application demand. be structured in a model we call MultiLibOS.A One might structure the memcached nodes to run cloud application adopting this model is distributed on a specialized library OS designed for distributed across a mix of general purpose OSs and special- hash tables (dhash-libOS in the figure) that directly ized library OSs. The general purpose OS nodes identifies and processes cache requests at interrupt support complete OS functionality and legacy com- level. Such a system would have no need for virtual patibility, while the rest of the nodes execute simple, memory, scheduling, or rich interfaces. By avoid- customized, library operating systems. ing expensive kernel/user level context switches, it With the MultiLibOS model, we exploit the on- is feasible that a full memcached request can be demand nature of resource management in the cloud processed in a few hundred CPU cycles. In con- to allow applications to allocate dedicated nodes for trast, memcached running on top of Linux is likely a particular task. The hardware acquired for this to require tens of thousands of cycles to service a purpose, as well as the libraries used by the appli- request. cation on that hardware, can be focused on aspects unique to that application’s task. Issues of protec- As our past work, along with others, have demon- tion, fairness and general multiplexing are elimi- strated, specialized library OSs can result in sub- nated. Rather, application-centric aspects of system stantial advantages for Java applications [1, 12] by software can take a front seat: application specific removing redundant OS functionality (e.g., schedul- APIs, light-weight hardware abstraction, distributed ing), and allowing the JVM direct control over sys- primitives, etc. tem memory and page tables. A MultiLibOS, as il- lustrated, would allow such a JVM library OS (jvm- libOS in the figure) to be naturally integrated and Elastic set of application nodes used in a complex heterogeneous web application. NODE NODE NODE NODE jvm-libOS jvm-libOS dhash-libOS dhash-libOS biz-logic biz-logic memcached memcached 4 Implications A MultiLibOS Web Application Apache DB memcached memcached The MultiLibOS model introduces an intuitive way Linux Linux dhash-libOS dhash-libOS to asymmetrically distribute OS functionality across NODE NODE NODE NODE an application while preserving legacy compatabil- ity. In this section we describe the key implications of such a model as they apply to modern cloud ap- Figure 1: A web app structured as a MultLibOS Ap- plications. plication. 2 Simplicity needs to be customized to meet the specific needs of that application. Intuitively, it is simpler to provide an application with a special purpose feature, such as a high speed With the MultiLibOS model, we believe that op- messaging service built directly on top of a hard- erating systems will have as much room for innova- ware supported RDMA system, rather than imple- tion as application level libraries do today. In con- menting a fully functional general feature, such trast to today’s world where there is a small number as sockets and the associated software to support of operating systems, we believe that the MultiLi- TCP/IP and arbitrary networking hardware. Follow- bOS model will result in many families of library ing this intuition, we expect that the library OSs in OSs, each addressing different concerns for differ- the MultiLibOS model will be lightweight and sim- ent classes of applications. ple. In particular, it may be that a library OS would not need to provide complex protection logic. In addition, simplicity is critical to allow experi- mental system techniques to be introduced and ex- Hardware Specialization plored. With other OS models, new features and interfaces need to be integrated into a general pur- Just as the MultiLibOS model allows for customiza- pose system for widespread adoption. Plumbing a tion to the needs of an application, library OSs can new innovative feature through a complex general be optimized to the characteristics of specific hard- purpose OS is an enormous challenge. A key char- ware. acteristic of our model is that new features can have IaaS datacenters are intrinsically heterogeneous. an instant impact and be directly applicable to real Non-uniform latencies and bandwidth exist between world applications. different parts of the datacenter. Large datacen- ters may have many generations of hardware, each Application Specialization with different quantities and characteristics of pro- cessing, memory, and networking. Different sys- The MultiLibOS model allows libraries to be writ- tems may have different properties, e.g., network- ten that are specialized for a set of applications. As ing properties like scatter gather and RDMA, or dif- noted, applications that do not benefit from specific ferent compute accelerators like GP-GPUs. Illus- OS functionality need not include the library that trating this trend is HP’s Moonshot [8] which em- provides it. braces heterogeneity in the cloud infrastructure by Previous work has shown that applications ben- constructing a system out of server cartridges, each efit from low-level optimizations. For example, with a wide variety of configurations for different applications and managed code environments that applications. have control over page tables have achieved greater efficiency [2, 12]. Specialized support for message We hypothesize that the MultiLibOS will enable passing, locks and event driven systems have grad- even greater heterogeneity in the cloud. In the Mul- ually been incorporated into various operating sys- tiLibOS model, custom OS functionality can enable tems. In a network centric system, low level control hardware developers to provide radically different over the networking hardware can have a dramatic hardware [13] that could not easily support general effect on multi-core performance [10]. purpose software due to the lack of hardware fea- Providing specialized functionality is critical tures such as virtual memory or privileged domains.