Distributed Virtual Machines: A System Architecture for Network Computing Emin Gün Sirer, Robert Grimm, Arthur J. Gregory, Nathan Anderson, Brian N. Bershad {egs,rgrimm,artjg,nra,bershad}@cs.washington.edu http://kimera.cs.washington.edu Dept. of Computer Science & Engineering University of Washington Seattle, WA 98195-2350 Abstract Modern virtual machines, such as Java and Inferno, are emerging as network computing platforms. While these virtual machines provide higher-level abstractions and more sophisticated services than their predecessors from twenty years ago, their architecture has essentially remained unchanged. State of the art virtual machines are still monolithic, that is, they are comprised of closely-coupled service components, which are thus replicated over all computers in an organization. This crude replication of services forms one of the weakest points in today’s networked systems, as it creates widely acknowledged and well-publicized problems of security, manageability and performance. We have designed and implemented a new system architecture for network computing based on distributed virtual machines. In our system, virtual machine services that perform rule checking and code transformation are factored out of clients and are located in enterprise- wide network servers. The services operate by intercepting application code and modifying it on the fly to provide additional service functionality. This architecture reduces client resource demands and the size of the trusted computing base, establishes physical isolation between virtual machine services and creates a single point of administration. We demonstrate that such a distributed virtual machine architecture can provide substantially better integrity and manageability than a monolithic architecture, scales well with increasing numbers of clients, and does not entail high overhead. 1. Introduction Virtual machines (VMs) have evolved significantly in the last two decades and may soon serve as a widely available network computing platform [Lindholm&Yellin96, Inferno, Adl-Tabatabai et al. 96]. They are particularly well suited for network computing because they offer a uniform programming model, work on a wide range of systems, and provide a variety of high-level services not supported by native architectures and general purpose operating systems. Modern virtual machines offer services, such as dynamic extensibility, verification, just-in-time compilation, configurable security policies and garbage collection, which are much more sophisticated compared to their predecessors [IBMVM86] and not readily found in general purpose operating systems [Custer 93]. In addition to the growth in the complexity of virtual machine services, the scale of deployment for VM systems has changed as well. Unlike early virtual machine systems that were typically confined to a few dedicated mainframes per enterprise, modern virtual machines are deployed in organizations with hundreds or thousands of heterogeneous hosts. Active content is now pervasive on the Internet, where about 1% of the roughly 125 million pages indexed by AltaVista reference a Java applet. More than 90% of the approximately 120 million deployed web browsers contain the Java virtual machine, and transparently fetch and execute active content from the world wide web. Even though virtual machine services have become much more numerous and complex, and even though the scale of deployment for VM systems has changed drastically, the service architecture of virtual machines has remained unchanged over the last few decades. Today’s virtual machines still rely on a monolithic architecture in which all service components reside locally on the host intended to run the applications. Consequently, service implementations and service state are replicated across all virtual machines in an organization. As a result of this crude placement and replication of functionality, modern virtual machines suffer from security problems [Dean et al. 97], are difficult to manage, and impose high resource requirements [Madany 96]. Furthermore, colocation of VM services has resulted in non-modular systems that exhibit complex inter-component interactions, as observed [Accetta et al. 89, Bershad et al. 95, Engler et al. 95] for other monolithic systems. In particular, networks of monolithic virtual machines exhibit the following shortcomings: • The lack of separation between virtual machine components means that a flaw in a single component of the virtual machine places the entire machine at risk. Furthermore, since policy specification and security enforcement are performed on the same host that runs potentially untrusted applications, one-time security holes can lead to long-term security compromises [Thompson 84]. • Since each virtual machine is a completely independent entity, there is no central point of control in an enterprise. There are no transparent and comprehensive techniques for distributing security upgrades, capturing audit trails, and pruning a network of rogue applications. • Virtual machine services, such as just-in-time compilation and verification, have substantial processing and memory requirements. When performed on the client, they can reduce overall application performance. • Monolithic systems are not suitable for hosts, such as embedded devices, which lack the resources to support a complete virtual machine. In the rest of this paper, we describe a virtual machine architecture based on distributed service components that addresses these problems. Our architecture is founded on factoring virtual machine services into logical components and placing these services at appropriate locations in the network. We have designed and implemented a Java virtual machine based on this architecture. Our VM includes a Java runtime, a verifier, a security service, a generic binary rewriting service and a compiler. It differs from existing systems in that these services are factored into well-defined components and centralized where necessary. The rest of the paper is structured as follows. Section 2 explains our approach and the goals of our architecture. Section 3 provides an overview of our system, and Section 4 describes the individual services in detail. Section 5 presents an evaluation of the architecture. Section 6 discusses related work, and Section 7 concludes. 2. Approach and Goals Our approach to the problems posed by monolithic virtual machines is based on service decomposition and distribution. We identify the logical services in existing VMs, factor them into separate modules with well defined interfaces, and place related services at locations in the network that suit their function [Figure 1]. We identify three major categories for related services. First, the runtime provides fundamental virtual machine functionality such as interpreting bytecodes and implementing core libraries. Second, rule checking ensures that the code executed by the runtime respects a set of requisite constraints, such as typesafety and access limits. Third, code transformation changes the code to be executed, for example by translating it into a native representation, or by modifying its runtime behavior. Platform Security Code Rule Checker Services Transformation Checker Verifier Service Garbage JIT Compiler Collector Runtime Bytecode Runtime Interpreter Library Figure 1. Monolithic versus factored virtual machines. The diagram on the left illustrates the current state of the art for modern virtual machines. Services in a monolithic VM are often indiscernible and not isolated from each other. The diagram on the right illustrates our approach. We factor the virtual machine into three major service groups; namely, rule checking, code transformation and runtime services. Factoring a monolithic VM into individual services enables a modular service infrastructure, with clear component boundaries and explicit interaction. Modularization thus helps define the trusted computing base, enables the piecewise testing of components, simplifies auditing, and thereby can produce systems of high assurance. It also 2 enables us to migrate functionality out of clients into locations more suitable to their function. This flexibility in service placement reduces the size of the trusted computing base, decreases resource requirements in clients, and physically isolates services from each other. Service granularity and placement are determined by the security, manageability, performance and scalability requirements of the overall system. For general-purpose network computing, which forms the application domain of our architecture, these requirements can be summarized as follows: • Security: The trusted computing base should be well defined, small, and physically isolated from application code [Saltzer&Schroeder 75]. An organization should have the ability to make network-wide, mandatory access control decisions, and know that they are being enforced on all clients. • Manageability: Management of virtual machines should be uniform across platforms and there should be a central point of control for administration. • Performance: Services should place a minimal processing burden on client machines and not require a large investment in the service infrastructure. • Scalability: Virtual machine implementations should scale over the diverse architectures and platforms found in a typical network. The minimum memory and processing requirements of a virtual machine should be small, though the system should be able to utilize all available resources when necessary.