INTL JOURNAL OF ELECTRONICS AND TELECOMMUNICATIONS, 2016, VOL. 62, NO. 3, PP. 231–236 Manuscript received August 12, 2016; revised September, 2016. DOI: 10.1515/eletel-2016-0031 Performance analysis of selected hypervisors (Virtual Machine Monitors - VMMs) Waldemar Graniszewski, Adam Arciszewski Abstract—Virtualization of operating systems and network results for CPU, NIC, kernel compilation time and storage infrastructure plays an important role in current IT projects. benchmarks’ tests are presented in Section IV. Finally, in With the number of services running on different hardware Section V, we draw some conclusions. resources it is easy to provide availability, security and efficiency using virtualizers. All virtualization vendors claim that their hypervisor (virtual machine monitor - VMM) is better than their II. BACKGROUND AND RELATED WORK competitors. In this paper we evaluate performance of different In this section we present some general background for solutions: proprietary software products (Hyper-V, ESXi, OVM, VirtualBox), and open source (Xen). We are using standard virtualisation technology (in Subsection II-A) and a short benchmark tools to compare efficiency of main hardware com- review of related work (in Subsection II-B). ponents, i.e. CPU (nbench), NIC (netperf), storage (Filebench), memory (ramspeed). Results of each tests are presented. A. Background Keywords—virtualisation, virtualmachines, benchmark, per- As mentioned earlier, in Section I, cloud computing and formance, hypervisor, virtual machine monitor, vmm services provided by data centers require robust software for their operation. With data center server consolidation, the I. INTRODUCTION portability of each solution plays an important role. In the N recent years the most popular IT projects have been last decade both proprietary software like VMware ESXi, Mi- I based on cloud computing. With hardware resources, es- crosoft Hyper-V, open source platforms, like Xen [2] and dual pecially RAM, CPU power, storage (HDD), network interface license programmes like VirtualBox [3] have been developed. cards (NICs) and other components, becoming cheaper, effi- Looking for the virtualisation technology market share we cient access to those resources is crucial, and is conducted by consulted recent reports of one of the leading information specialized software - hypervisors. There are many of different technology research and advisory companies - Gartner, Inc. hypervisors, e.g. VMWare, Xen, Hyper-V, Oracle VM, etc., According Gartner’s analysts about 80% of x86 server work- which can be installed on almost all platforms. Some of loads are virtualised. Therefore firms compete in offering more them are better suited and tuned for available hardware. All lightweight software, supporting more workloads and agile providers claim that their solutions are the best, but results development virtualisation solutions [4]. As of August 2016, often depend on the used applications, i.e.: web servers, file Gartner’s Magic Quadrant for x86 Server Virtualization Infras- servers, database applications, etc. tructure specifies 8 companies (in alphabetical order): Citrix, Server virtualization was a natural use of mainframes, where Huawei, Microsoft, Oracle, Red Hat, Sangfor, Virtuozzo, and access to computing resources was done via terminals. There VMware. According to this report VMWare is seen as the were also some formal requirements for computer architecture market leader, followed by Microsoft. Earlier, in the years proposed [1]. With development of PCs, virtualization, as seen 2010 and 2011, also Citrix (the company that currently owns in 1980s, was not continued for x86 computers because of Xen) was also placed in this quadrant [5]. According to Hess, weak hardware and OS system resources. In the past 10-15 Gartner’s report for the year 2015 is controversial [6]. The years, due to high performance and availability requirements Author suggests that solution of the Virtuozzo company should for cloud computing, the role of virtualization solutions in- be placed in the Visionaries instead of the Niche Players. creased. He also suggests positions of Challengers for Citrix Xen In this paper we would like to compare selected types of Server and Red Hat’s Enterprise Virtualization (RHEV) which server virtualization and test their performance. These tests also include containers. Another report prepared by Info-Tech include CPU, RAM, HDD, and NIC performance and are Research Group placed Citrix Xen in the group of Innovator conducted using standard benchmarking programs. and Red Hat and Oracle as Emerging Player. Both VMware The remainder of this paper is structured as follows. Section and Microsoft solutions remain in this report as Champion [7]. II provides a brief description of all platforms described in Taking into account market share of the virtualisation tech- the paper and a short review of related work. In Section nology, in this paper, we evaluate performance of different so- III we present a description of the test environment and the lutions: proprietary software products (ESXi, Hyper-V, OVM, methodology used to achieve performance comparison. Test VirtualBox), and open source (Xen). One of the most popular [8] classifications of hypervisors, Authors are with the Faculty of Electrical Engineering, Warsaw University also known as virtual machine monitors (VMM) is: of Technology, Warsaw, Poland e-mail: [email protected], [email protected]. • Type 1: native or bare metal hypervisors, 232 W. GRANISZEWSKI, A. ARCISZEWSKI • Type 2: hosted hypervisors. Morabito et al. [19] presented in their work a detailed In our selection of hypervisors for tests purposes only, Virtu- performance comparison of traditional hypervisor based virtu- alBox is a Type 2 VMM. The remaining hypervisors are Type alization and lightway solutions (containers). They compared 1, i.e. native or bare metal. However, Hwang et all claims that Kernel-based Virtual Machine (KVM), Linux Xen Containers Xen, possess characteristics of both types [9]. (LXC), cloud solution (OSv) and Linux container (Docker). Another often used [10] classification is: Due to our research interest, infrastructure needs, and ac- cording to the market share of the virtualisation technology • para-virtualization (PV), (see Subsection II-A), we evaluate performance of ESXi, • full virtualization (FV), Hyper-V, OVM, VirtualBox and Xen in this paper. • hardware-assisted virtualization (HVM). Owing to strong competition between all of the market III. METHODOLOGY players current hypervisors implement and can use at least As mentioned in Subsection II-B, there is a number of test 2 of mentioned techniques, i.e. para-virtualization (PV), full approaches and tools used for OS and hardware performance virtualization (FV) or hardware-assisted virtualization (HVM). analysis, e.g. [9], [11], [13]–[17], [19]. The methodology used Also in the case of VirtualBox, software detects if processor in this study is similar to that used by researchers from George supports hardware virtualisation or not and switch to appro- Washington University [9], but in contrary to their and other priate mode [3]. benchmarks, we tested newer versions of both native (bare Due to high computation cost for virtual machines, so metal hypervisors) and hosted hypervisors. called overhead for guest operating system, in the last decade, It has to be mentioned that the hypervisor performance tests there were also introduced container based operating systems. were conducted on a single physical machine with an Intel One of the proprietary solutions is Oracle Zone (former Core 2 Duo E8400 dual-core (clock speed of 3GHz). The test Solaris Zone) [11]. Another open source, container-based OS computer was also equipped with 4GB of DDR2 RAM, as virtualization platform example is Linux-VServer [12]. well as a 5400rpm 60 gigabyte hard drive. Each hypervisor was tested in an isolated manner, with a complete hard drive B. Related work format between installations. An Ubuntu 12.04 LTS virtual machine was created on each hypervisor. In recent years, many benchmark comparisons has been It should be noted that Oracle VirtualBox, as the only hosted published. VMware has published a performance comparison hypervisor, was run on Windows Server 2012. The remaining of their VMware ESX server 3.0.1 and Xen 3.0.3-0 [13]. part of the testing methodology remains the same, using the The company used SPEC, Passmark and Netperf tools for its same guest system. OVM is based on the Xen hypervisor [20] benchmark. and thus, as with Citrix XenServer, is run natively on the Danti compared KVM versus WirtualBox 4.0 [14]. He hardware [21] and, together with ESXi and Hyper-V, they are tested Windows 2008 R2 install time, Windows 2008 R2 bare metal solutions. installer load, Debian base system install time, host resource Performance tests were undertaken on the same operating utilization and guests system performances. system - Ubuntu 12.04 LTS. Each major component of the Li et al. [15] tested three hypervisors: a commercially virtual machine was tested separately. These components were available one (the exact name of which was not explicitly the CPU, memory, hard drive, and network interface. provided in their paper) and open source software, i.e. KVM, When a virtual machine is created, a certain number of Xen. They ran several MapReduce benchmarks, such as Word virtual CPUs (vCPU) is allocated to it. The number of vCPUs Count, TestDSFIO, and TeraSort and further validated their limits