Performance Evaluation of Technology Juncheng Gu 5191-0572

Abstract—Although virtualization brings numerous benefits, it meanwhile incurs performance loss. Therefore, it’s indispensable to evaluate the performance of virtualization. In this work, we measure and analyze the performance of three major virtualization technologies including LXC, and KVM. We separately test the virtualization of processor, memory, disk, network, isolation with specific benchmarks, and then analyze the results by examining their design and implementation. The result shows LXC, the light weighted virtualization, achieves the best performance, and I/O virtualization (disk, network) is the performance bottleneck of all virtualization technologies. Our work can help user make informed decision about their choice of .

I. INTRODUCTION

1.1 Background Virtualization technologies have become very important and gained widespread usage in the area of cloud computing and big data application because of its tremendous benefits, such as flexibility, independence, isolation, security, high resource utilizing rate, power saving and so on.

In virtualization systems, resource virtualization of underlying hardware and concurrent execution of virtual machines are in the charge of software named monitor (VMM) or hypervisor (in Xen). VMM abstracts underlying hardware and provides the same view to virtual machines, which enables virtual machine to run on physical machines with different hardware configuration. Classifying from the perspective of VMM, there are three popular virtualization technologies: KVM, Xen and Resource Container, which adopt the full-virtualization, para-virtualization, and container-based virtualization respectively.

KVM (Kernel-based Virtual Machine) is a VMM using full-virtualization and it is one of the most recent virtualization techniques. [1] With hardware extension support such as Intel-VT or AMD-V, it can construct an inexpensive virtual machine upon x86 hardware structure with an acceptable performance. Intel VT-x is a function added in x86 hardware to switch to hypervisor when sensitive instructions are detected by the CPU.[2] Because x86 CPUs have instinctive drawbacks on virtualization, it makes CPU failing to detect sensitive instructions when guest execute them. Intel VT-x was developed to solve this problem, as it separate CPU modes to VMX root modes and VMX non-root mode. As shown in Figure1.1, VMX non-root mode is the execution mode for guest system. KVM is mainly to handle VM Exits and the execution of VM Entry instructions.

Figure 1.1 Intel VT-x Figure 1.2 KVM/QEMU flow

KVM has been a standard kernel module and added into kernel. Therefore it can take advantages of standard Linux kernel and technology. However the KVM kernel module cannot create a virtual machine by itself. It requires support of QEMU, a user-space process, which is an inherent hardware emulator.[3] Figure1.2 shows the corporation of KVM/QEMU. KVM kernel module switches the Linux kernel to a hypervisor. For each guest system, QEMU emulate a guest system and makes system calls. When the guest system starts to execute, QEMU calls ioctl() to instruct KVM kernel module to start guest system. The KVM performs a VM Entry and begins the executing the guest system. When a sensitive instruction occurs, VM Exit is executed and KVM identifies the instruction and exit the VM. This QEMU/KVM flow is repeated during a VM running.[4]

The virtual machine (guest O/S) of KVM has no privilege to access the I/O devices. As a feature of full- virtualization, the virtual machine has no knowledge about the host operating system because it is no aware that it’s not running on real machine. Besides, KVM has userspace which takes charge of the I/O virtualization by employing a lightly modified QEMU to emulate the behavior of I/O or sometimes necessarily triggering the real I/O device. Any I/O requests of guest O/S are trapped into userspace and emulated by QEMU.

Figure 1.3 KVM structure

Xen is a para-virtualization featured hypervisor, which was proposed in [5]. Xen needs to modify the kernel of both the host and the guest O/S, but it requires no change to application binary interface (ABI) and thus existing applications can run without extra modification.

Through para-virtualization, Xen achieves high performance because the guest O/S knows that it is running in virtual environment. In Xen, only the hypervisor itself runs in ring 0, and the guest O/S runs upon ring 1, which is different from full-virtualization. Xen imports hypercall which has the same function with syscall. A hypercall is to a hypervisor what a syscall is to a kernel. Guest domains call hypercall and raise software trap to the hypervisor, just as a system is software trap from an application to the kernel. Guest domains use hypercall to request privileged operations.

The structure of Xen is shown in Figure1.4, a special domain “Domain 0” is added in guest system as a control interface, which is created as Xen is booted. Domain 0 is responsible to create and manage other domains. Besides, it is also used to schedule physical memory allocations and physical disk and network devices accesses. Operating system with different kernel can run above Xen. In our experiments, we all use Linux kernel as both host and guest systems to obtain accurate data under same experiment condition for different virtualizations.

Figure 1.4 Xen structure

Resource Container is container-based virtualization approach, also known as operating system level virtualization.[6] It works at the O/S level and is a lightweight virtualization technology. It logically contains all the system resources being used by an application to achieve a particular independent activity. The difference between container-based and hypervisor-based virtualization is remarkable, which is illustrated in Figure… Hypervisor-based virtualization provides abstraction for full guest OS, while container-based virtualization provides abstractions directly for the guest processes. Unlike hypervisor-based virtualization with high performance overhead, resource container promises a near- native performance. Since container works at the operating system level, it requires that all virtual machines share a same OS kernel. In this way, the isolation in container is supposed to be weaker than traditional virtual machines. Here, we mainly consider Linux Container (LXC). The isolation in LXC employs kernel namespace, which is a feature of Linux kernel and it allows processes to have different views in the system. Besides, it mainly relies on external tool-- cgroup for resource management, such as configuring network namespace, and process control.

Figure 1.5 Container-based virtualization vs Hypervisor-based virtualizaiton

1.2 Motivation

However, benefits are not always for free. Although virtualization technology provides many merits, it inevitably incurs some performance loss. The existing VMM debases the performance of some specific operations. For example, I/O virtualization is the bottleneck of most VMM, because of the frequent trap or mode switching caused by I/O instructions. I/O intensive workload might be greatly influenced because of the bad performance of I/O virtualization. Besides, memory management in virtualization system is much more complicated than normal operating system, such as two-layer mapping. Although some optimization approaches, such as shadow page table, eliminate the complicity in normal case, there is still high overhead when page faults occur. Therefore, it is indispensable to measure and analyze the performance of virtualization technology, comparing with bare metal physical machine.

Secondly, as we mentioned before, there are a lot of virtualization technologies, each of which has specific advantages and shortcomings. To gain better performance or benefits, application and virtualization system should match each other’s feature. It is essential to know VMM’s feature and select the right VMM before deploying applications. That’s why measuring and analyzing the performance of VMM is expecting.

II． RELATED WORK.

A lot of study has been made on virtualization techniques, especially on comparison of Xen and KVM. In paper [8] [9] [10], researchers evaluate the performance of virtual machine monitor. P. Barham proposed the idea of Xen and compared XenoLinux with native Linux, VMware and User-mode Linux with SPEC CPU2000, OSDB, dbench and SPEC web99[5].

Chen evaluated synthetically the performance of OpenVZ, Xen and KVM, with SPEC CPU2006, RAMSPEED, Bonnie++, NetIO and made synthetical analysis about their performance characteristics. [11] [12] evaluated the performance for Xen, Mware and LXC, mainly verified that the isolation performance of hypervisor is better than the container-based virtualization. [13] made qualitative comparison among Xen, KVM and basic linux in their overall performance, implementation details and general features. This paper mainly measured the network performance using Netperf, and system performance using UnixBench.

Andrea Chierici presented their work of comparison of Xen and KVM in their overall performance, performance isolation and scalability. [14] This paper tested overall performance using their benchmark suite, which measured the overall performance with a CPU-intensive test, a kernel compile and IOzone write and read test. Performance isolation is measured by SPEC web2005, which indicated that Xen has good isolation properties for the memory, fork, CPU and disk stress test, but shows poor isolation properties for network sender and receiver test. KVM, on the contrary, shows good isolation properties for all the stress test including network receiver test, but performs poorly in disk test. To test scalability, they increase the number of guests that each was compiling Apache source code. The test shows that Xen has much better scalability performance than KVM.

Our work is to reproduce and extend the work in [10][14], quantitatively measured the overall performance of Xen, KVM and LXC, and compare to the native Linux system. We mainly measure the performance of these three virtualization techniques in aspects of processor, disk IO, memory and network.

III． APPROACH Our goal is to measure and quantitatively analyze the performance of three virtualization categories: full, para, and container-based virtualization, each of which we select a popular approach for experiments: KVM (full), Xen (para), and LXC (container-based).

Generally, the virtualization system can be divided into several subsystems: processor, memory, disk, and network. Consequently, we measure the VMM as a black box about their performance on the virtualization of processor, memory, disk, and network with several canonical benchmarks. Besides on the three VMM systems, these benchmarks are also executed on bare metal system. In this way, the performance of bare metal system provides a baseline for the comparisons among different virtualization approaches.

What’s more, we measure the performance isolation among VMs created by KVM. Besides benchmark characterization, we quantify the performance of some typical applications in the virtual environment.

The benchmarks we select for each subsystem are listed below:

TABLE 3.1 BENCHMARK LIST

Performance test Benchmark Processor SPEC CPU2006 Memory STREAM Disk Bonnie++ Network Netperf The basic configuration of the testbed is:

TABLE 3.2 TESTBED CONFIGURATION

CPU Intel i5-3470T Due Core 2.90GHz RAM 32GB Disk WD 1TB 7200r/min NIC Qualcomm Atheros AR8151 v2.0 Gigabit Host operating system Ubuntu 12.04 server x86_64 Kernel 3.2.0-23-generic

3.1 Benchmark Introduction 1.Processor Virtualization:

To observe the performance of processor virtualization, we applied SPEC CPU2006[15] for testing. This benchmark provides performance measurements that can be used to compare compute-intensive workloads on different compute systems. It has two components: CINT2006 for compute-intensive integer performance, CFP2006 for compute-intensive floating point performance. Here, we select three benchmarks for CINT2006 and CFP2006 respectively, which is shown in Table 3.3.

TABLE 3.3 SELECTED BENCHMARKS IN SPEC CPU2006

Benchmark Language Application Area Brief Description 401.bzip2 C Compression Do most work in memory rather than doing I/O 403.gcc C C Compiler Based on gcc Version 3.2, generates code for Opteron 473.astar C++ Path-finding Algorithms Path find library for 2D maps 416.gamess Fortan Quantum Chemistry Implement a wide range of quantum chemical computations. 435.gromacs C, Fortran Biochemistry/Molecular This test simulate protein Lysozyme in a Dynamics solution 444.namd C++ Biology/Molecular Dynamics This test case has 92,224 atoms of apolipoprotein A-I. 2.Memory Virtualization:

The performance of memory virtualization is evaluated with STREAM, a simple synthetic benchmark program that measures sustainable memory bandwidth.[16] It performs four types of vector operations(add, copy, scale, and triad), using datasets much larger than the cache memory available in the computing environment, which reduces the waiting time for cache misses and avoid memory reuse. The details of the four vector operations are as follows:

add : �! = �! + �!

copy: �! = �!

scale: �! = s�!

triad: �! = �! + s�! 3.Disk Virtualization:

In this experiment we use Bonnie++ v1.03 to measure the disk IO performance of the file system. Bonnie++ is a disk and I/O test benchmark, which can measure the file systems perform various tasks.[17] It can measure the file reading and writing speed for sequential files or random files, also can test creating and deleting files for both type. Bonnie++ makes 10 tests and they are mainly divided in 5 classes: sequential output, sequential input, random seek, sequential create and random create.

To investigate the virtualization of disk, we use Bonnie++ in four environments to compare the three types of virtualization and use host Linux as baseline to analyze the IO speed of various systems.

4.Network virtualization:

Generally, there are two network connection approaches in virtualization environment: NAT and bridge. Virtual machine can’t be accessed from the external network in NAT connection; while in bridge mode, virtual machine can obtain a physical IP as well as the physical machine in the same LAN. Here, we configured KVM, Xen, and LXC with bridge network connection. The performance of network virtualization is measured with Netperf.[18] Netperf works as client/server method. In the test process, netperf transfer bulk of data to netserver to measure the throughput of the connection. Netperf provides network bandwidth testing between two hosts on a network. It defines several tests (TCP_STREAM, UDP_STREAM, TCP_RR, UDP_RR, and TCP_CRR) to measure bulk(unidirectional) data transfer or request response performance.

• TCP_STREAM performance test is the default test type for the netperf program. Netperf transfers TCP bulk data to netserver to measure the throughput of the connection. • UDP_STREAM test is similar to TCP_STREAM, but measures UDP bulk data transfer throughput. One difference is the send size cannot be larger than the socket buffer size. • TCP_RR test is multiple times transaction of TCP request/response, all processes occurs in one TCP connection. • TCP_CRR test is multiple times transaction of TCP request/response, but each request build up a new TCP connection. • UDP_RR test is times transaction of UDP request/response.

IV. EXPERIMENTS AND EVALUATION 4.1 Processor Virtualization

To test the computation performance of the processor virtualizaiton, we run SPEC CPU2006 benchmark for integer and floating point test.

SPEC 2006 INT Base Rao

473.astar

403.gcc

401.bzip2

0 5 10 15 20 25 30

KVM Xen LXC Bare

Figure 4.1 Computing performance using CINT2006

SPEC 2006 FP Base Rao

444.namd

435.gromacs

416.gamess

0 5 10 15 20 25 30

KVM Xen LXC Bare

Figure 4.2 Computing performance using CFPT2006

From the experiment results in Figure 4.1 and Figure 4.2, in terms of processor virtualization, the performances of all three virtualization have few variations, especially in test with Floating point components of SPEC2006. Compared with bare mental host, LXC has almost the same performance, because it shares the same operating system with the host O/S. For integer SPEC CPU2006 test, Xen and KVM have slightly lower performance in SPEC CPU2006 test. It is because computation-intensive workloads consist of few sensitive instructions, which can be executed directly by the CPU and do not need to be emulated by VMM or hypervisor. Therefore, the overhead of processor virtualization could be negligible. This result illustrate that processor virtualization is not the performance bottleneck for full, para and container-based virtualization.

4.2 Disk Virtualization

Figure 4.3 Disk throughput using Bonnie++

To test disk virtualization, we run Bonnie+ in four environments (host, LXC, Xen, KVM). For sequential output, ‘per cha’ means consumption of CPU that repeatedly writes bytes and store in I-cache. ‘Block’ means writing blocks to disk, and the consumption is mainly in allocating the disk space. ‘Rewrite’ measures the consumption of the process to read and modify files, then rewrite the modified files to the system.

From Figure4.3, we can see the per-char write operation of KVM is the lowest in three virtualization environments. It is well-known that KVM implements its virtual disk I/O with the modified QEMU. The disk I/O requests of guest operating system in KVM are trapped into the kernel mode and the kernel schedules QEMU to emulate the disk I/O operation with overhead in performance.

Among all the three virtualization systems, only LXC’s performance is as well as the native host operating system (Figure 4.3). The I/O path of LXC is exactly the same with the native host operating system. Once an I/O request is raised, CPU will change into kernel mode, and then the driver is called to manage the corresponding hardware. There is no overhead. From another perspective, LXC is just a process of host O/S, only with better resource management and isolation character compared with normal processes, its I/O actions should be the same with normal process’s IO actions.

The performance of Xen is better than KVM but much worse than LXC and the host operating system. In Xen, the hypervisor performs CPU scheduling for the domains, and then each domain has its own OS kernel for thread scheduling. The extra scheduler brings performance overhead compared with the native operating system. Having multiple schedulers can also create complex issues with how they interact, adding CPU latency in the wrong situations. Moreover, each I/O device in Xen is exposed as asynchronous I/O rings to guest domains. The working process of this I/O ring is more complicated than native operating system. Consequently, Xen’s disk I/O performance should be worse than the host operating system. The experiment results of disk test in four environments are in accordance with the theory.

Besides the results we obtained above, we concentrated on the low I/O performance of KVM for per- char writes. Although KVM’s I/O performance is always the worst, the performance degradations of different I/O operations are different (Figure 4.4). Per-char operation writes only one char for each time, and the large overhead of I/O emulation results the huge performance degradation. However, there are not so much I/O request in block operation, KVM can still achieve 75% performance of the host O/S. In conclude, the experiment shows that even though KVM has the lowest I/O efficiency three VMMs, when operating by blocks, the performance of KVM I/O is acceptable.

Figure 4.4 Disk test of KVM I/O performance degradation (normalized by host O/S) 4.3 Memory Virtualization

Figure 4.3 Memory throughput using STREAM

To evaluate the performance of memory virtualization, we run STREAM benchmark and apply memory bandwidth as metric.

From Figure 4.5, we can see although there are four types of vector operations provided by STREAM benchmark, the performance differences between the four testing environments are almost the same.

Firstly, LXC container can achieve the exceed-native performance regardless of the vector operation. This is due to the fact that LXC (container-based virtualization system) has the ability to return unused memory to the host and other containers, enabling better use of memory. In another word, LXC performs better memory resource management than native operating system, which brings a little bit excess performance compared with the host O/S.

The worst results are observed in KVM. This overhead is caused by the virtual machine monitor layer that performs memory accesses translation, resulting in loss of performance. KVM as a full- virtualization technique, the guest O/S could only see the virtualized physical memory in self address space. To finish the address translation, VMM must do the extra translation from virtualized physical address to real physical address. Although many optimization method, like shadow page table, has been developed to avoid this two-layer address mapping, the translation overhead is still inevitable, especially when page faults or TLB miss occurs.

Different from full-virtualization system, in para-virtualization system, the guest O/S can directly access the real physical memory belonging to it. That’s why Xen performs better than KVM in memory virtualization. However, Xen’s performance is not as good as the host O/S. Although guest O/S has direct read access to hardware page tables, only VMM/hypervisor has the privilege to modify the page tables. This mechanism brings the overhead in memory virtualization compared with native operating system.

In summary, as full-virtualized VMM, KVM has the biggest overhead in address translation, which turned out to have the lowest memory bandwidth; Xen as para-virtualized hypervisor, VM is transparent to host O/S and no duplicate address mapping, the performance is better than KVM. LXC as an O/S level virtualization, it does not require extra overhead to allocate memory address, and it even employs a more flexible memory allocation method than host O/S. 4.4 Performance isolation Performance isolation is an important factor in the area of virtualization, which is a measure of how well guests are protected from extreme resource consumption in other guests. Ideally, virtual machines running on the same host should not be disturbed by each other. However, shared recourse between virtual machines may have a bad influence on the isolation. KVM, as a full-virtualized VMM, it has many advantages over the other two. It does not require modified kernel in the system, thus it is easy to create guest using KVM. In following experiments, we measure the performance isolation between guests created by KVM.

In our experiments, we create four KVM guests and choose one guest as the well-behaving virtual machine, whereas the other three guests are named as misbehaving virtual machine. Then we introduce a series of stress tests into misbehaving guests and quantify the performance degradation of the well- behaving guest when the misbehaving guests are running the stressed workloads. We use Isolation Benchmark Suite (IBS) [19] as the source of stress tests and still use the same benchmarks as above to measure the performance degradation of the well-behaving virtual machine.

First, we measure the performance of well-behaving guests as the baseline when all the misbehaving guests are idle. Secondly we run stress tests in one of the misbehaving guests. The stress tests include CPU intensive test, memory intensive test, disk intensive test and network intensive test. At the same time, we measure the performance degradation of the well-behaving guest, including CPU, RAM, Disk and Network. Then we increase the number of misbehaving guests to quantify the performance of well- behaving virtual machine.

Performance Isolaon

1.2

1

0.8

0.6

0.4

0.2

0 1 2 3 4

CPU RAM DISK NET

Figure 4.6 Performance isolation test using IBS

As is shown in Figure 4.6, the performance degradation of CPU and RAM is smaller than that of Disk and NET. For CPU, there is slight degradation when there is only one misbehaving guests. Because the platform has two physical core and each guests occupy itself physical core. The performance of processor will decrease when the number of misbehaving guests increase. Since the CPU support Hyper- Threading technology, the degradation is less than 40% when there are three misbehaving virtual machines. As the total memory size is 32 GB and every virtual machine only has 2GB memory, RAM has the similar degradation with CPU. For network and disk, since all the guests need to share the same disk and network, the performance will decrease greatly when the stress tests are running on the misbehaving guests. For example, the rate of disk sequential write will reduce to 20% and the throughput of TCP reduce to 40%. Based on the isolation experiment, CPU and RAM has the better isolation performance than Disk and Network. It is better to run compute-intensive workload concurrently on the guests built in the same physical machine.

4.5 Application Nowadays, with the development of virtualization technology and computer hardware, more and more people use virtualized nodes to do their jobs. For example, people may built their virtual computing environment by means of Amazon Elastic Compute Cloud.[20] People often run all kinds of applications on the virtual computing environment, instead of benchmarks. Therefore, apart from measuring the performance of benchmarks, we run some application on the guests created by KVM.

Figure 4.7 Enhanced DFSIO (IO intensive)

Figure 4.8 Wordcount (CPU intensive)

In this paper, we run some big data applications to measure their performance in the virtual environment. As is shown in Figure 4.7 & 4.8, we choose two kinds of workloads from HiBench[21]33. One is compute-intensive workload, such as wordcount. The other is IO-intensive workload, for example enhanced DFSIO. Wordcount counts the occurrence of each word in the input data and Enhanced DFSIO tests the HDFS throughput of the Hadoop cluster by generating a large number of tasks performing writes simultaneously. In our experiment, the input data size of wordcount is 2GB and Enhanced DFSIO writes 16 100MB files to HDFS. We run the two workloads using 1, 2, 3 and 4 VMs respectively.

Hadoop workload in VMs 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 1 2 3 4 wordcount enhanced DFSIO

Figure 4.9 Hadoop workload in VMs

Based on the Figure 4.9, the runtime of wordcount will decrease when we increase the number of VMs. When we use two VMs, the runtime will reduce to 60%. After we continue to increase the number of VMs, the runtime will nearly keep still. Because the performance of processor will reduce greatly after we run 3 or 4 VMs concurrently. Figure could validate the conclusion. From Figure 4.6, the performance degradation of network and disk is very big when multiple VMs are running in parallel. For example, the IO rate measured by DFSIO is 13.7 MB/sec when only one VMs is running and the IO rate will reduce to 2 MB/sec when the number of VM is 3. IO is a bottleneck for the virtualized platform. Therefore, the runtime of enhanced DFSIO will increase after we run multiple VMs.

V. CONCLUSION

Performance evaluation on virtualization technologies plays an important role in using or even improving their design. Virtualization brings many benefits, but also imports some overhead and leads to performance loss. Different virtualization technologies perform different features in this pair of tradeoffs. To gain a clear view and deep understanding of performance features of the major types of virtualization technologies, we designed this work.

We measured and analyzed three major virtualization technologies: LXC (container-based), KVM (full), and Xen (para). We separately tested the subsystems of each virtualized system with canonical benchmarks: processor (SPEC CPU2006), memory (STREAM), disk (Bonnie++), network (Netperf). Besides benchmarks, we run some typical applications and measure their performance. What is more, we quantify the performance isolation among guests using KVM.

From the perspective of virtualization technology, LXC has the best performance and Xen follows, while KVM appears the lower performance than LXC and Xen. Most of time, LXC performs the near- native performance because it’s just a special process of the native operating system and most of its actions have the same process with the actions of native operating system. Xen has a slight degradation in most experiments since it’s para-virtualized and the guest domain is aware of the virtualized system. However, KVM is observed the worst performance because of too much traps and emulations in this full-virtualized system. Consequently, container-based virtualization’s general performance is better than para-virtualization technology’s, and para-virtualization mostly performs better than full- virtualization. Nevertheless, container-based virtualized system must has the same kernel as the host operating system; para-virtualization requires changes to the kernel of guest operating system. On contrast, full-virtualization needs no modifications to guest operating system. Hence, there is also a tradeoff between performance loss and easy deployment.

When viewing the performance of different subsystem, we found that I/O (disk and network) might be the bottleneck of most virtualization technologies, especially full-virtualization technology. The performance of processor virtualization and memory virtualization brings no or acceptable degradation. Therefore, focuses should fixed on optimizing bottleneck (I/O virtualization) to improve the performance of virtualization technology.

REFERENCE

[1] Kivity, Avi, et al. "kvm: the Linux virtual machine monitor." Proceedings of the Linux Symposium. Vol. 1. 2007. [2] Uhlig, Rich, et al. "Intel virtualization technology." Computer 38.5 (2005): 48-56. [3] Bellard, Fabrice. "QEMU, a Fast and Portable Dynamic Translator." USENIX Annual Technical Conference, FREENIX Track. 2005. [4] Goto, Yasunori. "Kernel-based virtual machine technology." Fujitsu Scientific and Technical Journal 47 (2011): 362-368. [5] Barham, Paul, et al. "Xen and the art of virtualization." ACM SIGOPS Operating Systems Review 37.5 (2003): 164-177. [6] Banga, Gaurav, Peter Druschel, and Jeffrey C. Mogul. "Resource containers: A new facility for resource management in server systems." OSDI. Vol. 99. 1999. [7] “Linux Containers,” 2012. [Online]. Available: http://lxc.sourceforge.net [8] Menon, Aravind, et al. "Diagnosing performance overheads in the Xen virtual machine environment." Proceedings of the 1st ACM/USENIX international conference on Virtual execution environments. ACM, 2005. [9] Apparao, Padma, et al. "Characterization & analysis of a server consolidation benchmark." Proceedings of the fourth ACM SIGPLAN/SIGOPS international conference on Virtual execution environments. ACM, 2008. [10] Ongaro, Diego, Alan L. Cox, and Scott Rixner. "Scheduling I/O in virtual machine monitors." Proceedings of the fourth ACM SIGPLAN/SIGOPS international conference on Virtual execution environments. ACM, 2008. [11] Che, Jianhua, et al. "A synthetical performance evaluation of OpenVZ, Xen and KVM." Services Computing Conference (APSCC), 2010 IEEE Asia-Pacific. IEEE, 2010. [12] Xavier, Miguel G., et al. "Performance evaluation of container-based virtualization for high performance computing environments." Parallel, Distributed and Network-Based Processing (PDP), 2013 21st Euromicro International Conference on. IEEE, 2013. [13] Binu, A., and G. Santhosh Kumar. "Virtualization techniques: a methodical review of XEN and KVM." Advances in Computing and Communications. Springer Berlin Heidelberg, 2011. 399-410. [14] Deshane, Todd, et al. "Quantitative comparison of Xen and KVM." Xen Summit, Boston, MA, USA (2008): 1-2. [15] SPEC CPU2006 http://www.spec.org/cpu2006/ [16] STREAM http://www.cs.virginia.edu/stream/ [17] Bonnie++ http://www.coker.com.au/bonnie++/ [18] Netperf http://www.netperf.org/netperf/ [19] IBS http://web2.clarkson.edu/class/cs644/isolation/ [20] Amazon EC2 http://aws.amazon.com/ec2/ [21] Huang, Shengsheng, et al. "The HiBench benchmark suite: Characterization of the MapReduce-based data analysis." Data Engineering Workshops (ICDEW), 2010 IEEE 26th International Conference on. IEEE, 2010.

APPENDIX

STEP1 VMM installation: Xen (Song), KVM (Yan), LXC (Gu). Benchmark installation: SPEC CPU2006 (Song), STREAM & Netperf (Yan), Bonie++(Gu) Each of us is in charge of deploying the specific benchmarks for each metrics in the three virtualization systems and bare metal host. STEP2 VMs isolation test using KVM: Gu. Evaluating big data application in KVM virtualized environment: Song First draft of report: Yan SETP3 All of us will complete the report of our own experiment, and review the whole report.