VAMOS: Virtualization Aware Middleware Abel Gordon? Muli Ben-Yehuday? Dennis Filimonovy Maor Dahany [email protected] [email protected] [email protected] [email protected] ?IBM Research—Haifa yTechnion—Israel Institute of Technology Abstract We argue that changes at the application layer are in- evitable: the shift to machine virtualization everywhere Machine virtualization is undoubtedly useful, but does requires that we adapt our applications to cooperate with not come cheap. The performance cost of virtualiza- the virtualization layer. tion, for I/O intensive workloads in particular, can be Adapting a nearly infinite number of applications is heavy. Common approaches to solving the I/O virtual- not feasible. However, in practice most server applica- ization overhead focus on the I/O stack, thereby missing tions depend on one or more middleware layers, such as optimization opportunities in the overall stack. We pro- a database server, a web server, or an application server. pose VAMOS, a novel software architecture for middle- By adapting the middleware to virtualized platforms, we ware, which runs middleware modules at the hypervisor can indirectly adapt many applications and regain lost level. VAMOS reduces I/O virtualization overhead by performance. We need to re-think the way we build mid- cutting down on the overall number of guest/hypervisor dleware. Most middleware have a modular design that switches for I/O intensive workloads. Middleware code already provides abstraction layers for operating system can be adapted to VAMOS at only a modest cost, by ex- services, such as memory management, disk or network ploiting existing modular design and abstraction layers. I/O, and CPU scheduling. We can exploit this to allow Applying VAMOS to a database workload improved its middleware software modules to cooperate with the hy- performance by up to 32%. pervisor or run at host level, without necessitating a re- write of the entire middleware. 1 Introduction This novel architecture model raises new opportunities for performance optimizations at the middleware layer, Machine virtualization provides many benefits: it en- which do not conflict with today’s optimization at lower ables server consolidation, makes it possible to run layers. For example, the middleware can interact with legacy environments on new platforms, and helps sim- the hypervisor I/O sub-systems using interfaces or pro- plify system management. However, it also degrades the tocols defined at the application level. Instead of using a performance of common workloads, particularly if they para-virtual block [4, 17], network [4, 17], file system [9] are I/O intensive workloads. or socket [19] interfaces, a database server could use a Extensive research has been carried out in order to para-virtual SQL interface while a web server could use reduce the virtualization (including I/O virtualization) a para-virtual HTTP interface. By using high level in- performance penalty, both at the hardware and soft- terfaces we could remove the cycles spent in the hyper- ware layers. However, these works were mainly fo- visor emulating the virtual hardware and we could also cused on improving the interaction between the hard- decrease the number of transitions between the hypervi- ware [12, 15, 22], the hypervisor [4, 16, 17, 18] and the sor and the guest, thus improving both I/O throughput operating system [9, 19, 21], avoiding changes at the ap- and latency [19]. plication layer. Virtual appliances and Platform as a Service (PaaS) While virtualization is quickly becoming ubiquitous, clouds are natural places to apply this new architecture applications remain oblivious to the changes in the un- model, primarily because the middleware layer there is derlying platform. Software architects and designers still under the control of the provider, and the adaptations will use the same principles and models that used to apply to be transparent to the customer’s applications. We can non-virtualized systems for building their applications. extend the model for Infrastructure as a Service (IaaS) 1 clouds, implementing the adaptations at the hypervisor level and exposing them as optional accelerators to the virtual machines. Assuming the customers decide to in- stall required extensions into the virtual machines im- age, as they might install para-virtual drivers such as VMware-tools or virtio drivers [17], they will benefit from the improvements. The main contribution in this work is VAMOS, a novel architecture for middleware running in virtual en- vironments. Using VAMOS middleware cooperates with the hypervisor to improve the overall application perfor- mance. A proof of concept of VAMOS shows up to 32% I/O performance improvements for database workloads when compared with standard approach to I/O virtual- ization. Figure 1: Comparing architecture models 2 VAMOS Architecture cooperate with the hypervisor and obtain direct access In this section we present VAMOS,an architecture model that allows cooperation between the hypervisor and the to the physical resources, bypassing the virtualization layer and mitigating part of the overhead. Figure 1 com- middleware, improving I/O performance. First, we ex- plain the causes for the virtualization overhead and then pares the traditional software architecture model with the VAMOS architecture. However, for VAMOS to be feasi- we describe the VAMOS architecture and its advantages. ble, we need: 2.1 Causes of Virtualization Overhead 1. An isolated runtime environment at the hypervi- sor level which is capable of executing middleware It is well known that virtualization has a significant over- code and providing the necessary services to access head [1, 5]. When we run a workload in a virtual ma- physical resources, such as network devices or disk chine, it does not get exclusive access to all the hard- drives. ware resources, even when we use hardware virtualiza- tion support such as Intel VMX or AMD SVM. Usu- 2. A communication channel between the middleware ally, the VM runs directly on top of the physical CPU, running in the guest, the middleware running in the however, when a sensitive instruction is executed, the host and the hypervisor. CPU transfers the control to the hypervisor for handling. During this handling period, the guest is temporary in- 3. A methodology for adapting existing middleware terrupted and the hypervisor consumes precious cycles code at a reasonable cost. handling the sensitive instruction, which might require emulating virtual hardware. In addition, the transitions Today’s hypervisors, such as KVM or Xen, or hosted have a fixed cost caused by the CPU switching between hypervisors, already provide a fully operational operat- the guest and the hypervisor contexts, and variable cost ing system to manage the physical hardware. We can caused by the hypervisor polluting the guest cache [5, 8]. take advantage of this fact and run middleware modules I/O intensive applications suffer the most [18] because as additional processes or runtime libraries in the host. they cause relatively many guest/host transitions. To get The cooperation between the middleware and hypervisor rid of the virtualization overhead, we need to reduce the can be implemented using regular mechanisms to com- number of transitions and/or reduce the handling cost. municate between user-space processes or kernel mod- ules. In addition, the middleware can access physical 2.2 Proposed Architecture resources via the host OS services, such as file systems, network interfaces, and virtual memory. For the guest The number of abstraction layers required to access the to host communication, we can exploit the channels im- hardware resources and the absence of cooperation be- plemented to run para-virtualized guests [4, 17]. Alter- tween the hypervisor and the application cause many natively, we can use more efficient channels, based on transitions to the hypervisor context. We can solve these polling and side core invocation techniques [2, 8, 11, 13]. problems running part of the middleware in the hyper- Machine virtualization typically requires isolating the visor context. Using this approach, the middleware can physical resources from untrusted virtual machines. If 2 we treat the host side middleware modules as trusted en- tities, we can allow these modules to access physical re- sources. However, such middleware modules might con- tain security vulnerabilities. Thus, to minimize the ef- fects of potential client attacks on the host, the hypervi- sor should restrict the privileges of those modules. This can be achieved using plugin-isolation techniques such as Vx32 [7] or Native Client [23]. While we have the appropriate environment to run part of middleware at the hypervisor level, we still need a methodology to adapt existing code. Modular de- sign [14, 20] and abstraction layers [10] are well known techniques implemented in most if not all middleware products to decouple different components and to ab- stract OS services. For example, as shown in Figure 2, MySQL abstracts and implements the I/O logic into pluggable storage engines. Another example is Apache Figure 2: MySQL Architecture Tomcat, which encapsulates the logic responsible for handling the communication with the clients into inde- where the hypervisor spends prestigious cycles emulat- pendent modules, called connectors. JVMs usually ab- ing the virtual hardware. stract OS services to have a single cross platform core By running the middleware modules responsible for implementation and easy to maintain platform dependent I/O operations at the hypervisor level, we can reduce modules [3]. We can take advantage of the modular de- the number of transitions between the guest and the hy- sign and run part of the middleware at the hypervisor pervisor as well as part of the virtual hardware emu- context. The following criteria characterize the modules lation. Instead of switching on every low-level event which will improve the application performance when such as packet sent or disk block written, we can switch moved to the hypervisor: when higher-level events (e.g., an SQL query) occur.