See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/280092756 Performance Analysis of LXC for HPC Environments Conference Paper · July 2015 DOI: 10.1109/CISIS.2015.53 CITATIONS READS 6 470 6 authors, including: Edward David Moreno Ordonez Patricia Takako Endo Universidade Federal de Sergipe University of Pernambuco 115 PUBLICATIONS 108 CITATIONS 27 PUBLICATIONS 271 CITATIONS SEE PROFILE SEE PROFILE Djamel Sadok Stenio Fernandes Federal University of Pernambuco Federal University of Pernambuco 242 PUBLICATIONS 1,303 CITATIONS 100 PUBLICATIONS 673 CITATIONS SEE PROFILE SEE PROFILE Some of the authors of this publication are also working on these related projects: security embedded View project Context-Aware Energy Efficiency Management for Smart Buildings View project All content following this page was uploaded by Edward David Moreno Ordonez on 16 July 2015. The user has requested enhancement of the downloaded file. All in-text references underlined in blue are added to the original document and are linked to publications on ResearchGate, letting you access and read them immediately. Performance Analysis of LXC for HPC Environments David Beserra Patricia Takako Endo Jymmy Barreto, Djamel Sadok and Edward David Moreno University of Pernambuco and Stenioˆ Fernandes Federal University of Sergipe Caruaru, Pernambuco, Brazil Federal University of Pernambuco Aracaju, Sergipe, Brazil [email protected] Recife, Pernambuco, Brazil [email protected] fjpsb, jamel, sfl[email protected] Abstract—Despite of Cloud infrastructures can be used as by academia, and according to [9], this type of virtualization High Performance Computing (HPC) platforms, many issues offers a lightweight virtualization layer, which promises a near- from virtualization overhead have kept them unrelated. However, native performance. with advent of container-based virtualizers, this scenario acquires new perspectives because this technique promises to decrease Our investigation aims to answer which scenarios LXC can the virtualization overhead, achieving a near-native performance. offer a better performance than hypervisor-based virtualization In this work, we analyze the performance of a container-based or even equal to native environments for HPC applications. For virtualization solution - Linux Container (LXC) - against a this, we will conduct experiments with traditional benchmarks hypervisor-based virtualization solution - KVM - under HPC considering different VM allocation possibilities: many VMs activities. For our experiments, we consider CPU and communi- on the same host competing or coopering for resource usage, cation (network and inter-process communication) performance, and results show the hypervisor type can impact distinctly in and VMs allocated in distinct hosts without resource sharing. performance according to resource used by application. This work is structured as follows: Section II presents Keywords—Cloud Computing, HPC, Container-based virtual- virtualization basic concepts needed to understand this work; ization, Performance evaluation. Section III describes the related works; Section IV presents our main goals and the methodology used to perform our I. INTRODUCTION experiments; we present and discuss the results in Section V; and finally, we describe our considerations and delineate future High Performance Computing (HPC) is a generic term for works in Section VI. applications that make intensive use of one or more compu- tational resources and are technical-scientific nature [1] [2], like simulations of large-scale structures, such as the Universe II. VIRTUALIZATION BACKGROUND [3]. Currently, we can host many HPC applications on Cloud infrastructures; including Cloud Computing solutions for sci- Currently, we have four approaches to provide resource entific HPC applications, such as Nimbus [4] and Neblina virtualization: full virtualization, hardware-assisted virtualiza- [5]. HPC applications have used Cloud Computing infrastruc- tion, paravirtualization, and operating-system-level (OS-level) ture since the advent of scientific clouds [4] and virtualized virtualization. computer clusters [6] due to its facility to rent, to manage The full virtualization allows a guest OS running with- and to allocate resources according to demand. As benefits out kernel modifications. However, it imposes high overhead arising from Cloud Computing to HPC, we can highlight in VM performance. This overhead can be mitigated using high availability (HA), Operating System (OS) customization, hardware-assisted virtualization, a set of specific instructions elasticity, and cost reduction of resource maintenance [7], [6]. to virtualization that is present in almost processors and in In Cloud Computing environment, virtualization is a key some I/O elements [10]. technology for enabling its operation and, despite advantages, When using paravirtualization approach, hardware de- virtualization commonly suffers performance penalties due vices are accessed through special paravirtualized drivers, to resource sharing; presenting worst performance than non- obtaining a better performance when compared to full virtual- virtualized scenarios [8]. In this way, evaluate virtualization ization, even when it is assisted by hardware [7]. However, performance under HPC applications is relevant, specially the guest OS kernel must be modified in order to provide when we consider Message Passing Interface (MPI) applica- new system calls. This modification increases the performance tions running on Beowulf Clusters [7]. because reduces the CPU consumption, but at same time, it Trying to overcome this performance issue, container- reduces the security and increases the management difficulty. based virtualization solutions (such as Linux-VServer 1, The OS-level virtualization creates containers that allow OpenVZ 2 and Linux Containers (LXC) 3) have been proposed processes have their own isolated resources without hardware 1http://linux-vserver.org/ emulations, accessing hardware resources in a direct way. Each 2http://openvz.org/ container runs its own OS and file system, but shares the kernel 3https://linuxcontainers.org/ with another containers and the host OS [9]. In this work, our main objective is to analyse OS-level running some benchmarks in virtual clusters. They formulated virtualization performance against another virtualization tech- a hypervisor ranking for HPC, and concluded that KVM and nique. The virtualization solutions used in this work are pre- VitualBox present the best global performance and manage- sented in more detail in next Sub-section and the justifications ment facility, with KVM excelling in regard to computation for our choice are presented in Section III. capability and memory expansibility. Authors also observed that, different from Xen, KVM presents little performance A. KVM and LXC oscillations, that is considered a key characteristic in Cloud environments. Kernel based Virtual Machine (KVM) 4 is an open source virtualization tool integrated to Linux kernel. This virtualiza- However, these works did not considered OS-level virtual- tion approach takes advantage of the Linux kernel development ization technologies based on containers. Containers can offer as own evolution [10]. In KVM, the I/O and network manage- advantages for HPC, such as a short bootstrap time, and a less ment is done by a QEMU modified version. I/O requests done disk space usage. Authors in [13] noted containers present a by a VM are forwarded to the QEMU that redirects them to bootstrap duration 11 times smaller than KVM, and the disk the host OS (see Figure 1). usage is also smaller; while 50 VMs (KVM running Ubuntu) can be stored in a disk with 500Gb, more than 100 containers use less than a half of this space. In other hand, authors in [14] compared the performance of OpenVZ, Xen, and KVM using VMs as web servers. OpenVZ presented results better than native in some situations; however the KVM paravirtualization instructions were not activated. This configuration could collaborate to KVM inferior perfor- mance, since KVM with paravirtualization instructions can obtain performance better than Xen [10], [12]. Authors in [9] innovates when analysing the performance of main OS-level virtualization solutions running HPC appli- cations. They compare OpenVZ, LXC, and VServer against Xen performance considering CPU, disk, and network. All containers solutions and Xen presented an equivalent process- ing capacity, with all solution presenting performance close to a native environment. The network performance analysis Fig. 1. KVM and LXC considered bandwidth and delay according to packet size and their results indicated that LXC and VServer have better Linux Containers (LXC) is a lightweight virtualization performance, with smaller delay and bigger bandwidth for any mechanism that does not require emulation of physical hard- size of packet. Since the HPC applications were tested, thus ware. The main usage of LXC is to run a complete copy of far, LXC demonstrates to be the most suitable of the container- the Linux OS in a container without the overhead of running based systems for HPC [9]. a level-2 hypervisor (see Figure 1.b). The container shares For our work, we decided to use KVM because it has the the kernel with the host OS, so its processes and file system best performance and is the best hypervisor-based virtualizaton are completely visible from the host. On the other hand, the solution according to literature [12], [15], and LXC because