When the Virtual Machine Wins Over the Container: DBMS Performance and Isolation in Virtualized Environments
Total Page:16
File Type:pdf, Size:1020Kb
Journal of Information Processing Vol.28 369–377 (July 2020) [DOI: 10.2197/ipsjjip.28.369] Regular Paper When the Virtual Machine Wins over the Container: DBMS Performance and Isolation in Virtualized Environments Asraa Abdulrazak Ali Mardan1,a) Kenji Kono1,b) Received: September 10, 2019, Accepted: April 8, 2020 Abstract: Database management systems (DBMSs) are a common service in the cloud. Cloud providers employ virtualization to consolidate DBMSs for efficient resource utilization and to isolate collocated DBMS workloads. To guarantee the service-level agreement (SLA), the disk I/O performance and its isolation are critical in DBMSs because they are inherently disk I/O intensive. This paper investigates the disk I/O performance and its isolation of MySQL on two major virtualization platforms, KVM and LXC. KVM provides the hypervisor-based virtualization (virtual machines) while LXC provides the operating-system-level virtualization (containers). Containers are widely believed to outperform virtual machines because of negligible virtualization overheads, while virtual machines are to provide stronger performance isolation. Contrary to the general belief, our results show that KVM outperforms LXC by up to 95% in MySQL throughput without compromising the isolation. Furthermore, LXC violates the performance isolation; a container given 30% share of the disk bandwidth consumes the 70% share. Our analysis reveals that the file system journaling, which is mandatory to maintain file system consistency, has negative impact on both the performance and its isolation in the container. Because of the sharing of the journaling module in containers, the journaling activities are serialized and bundled with each other, resulting in poor performance and violation of the performance isolation. Keywords: virtualization, container, DBMS, journaling file system. performance degradation because less cache is allocated for them. 1. Introduction On the other hand, VMs have stronger performance isolation be- Virtualization technologies are the building block of cloud dat- cause no kernel components are shared among VMs. acenters. They are widely used in today’s datacenters to achieve The performance and isolation are important in virtualized en- server consolidation for efficient resource usage, less power vironments, where multiple users’ workloads are consolidated conusumption, and to isolate collocated applications on a single on the same machine. Users expect their workloads run in iso- machine. There are two virtualization approaches: operating sys- lated manner and to get the performance they pay for. Failing to tem (OS)- and hypervisor-based virtualization. Hypervisor-based achieve isolation results in performance degradation. This leads virtualization virtualizes hardware resources to provide the ab- to SLA violation which ultimately results in financial loss for the straction of virtual machines (VMs). This allows multiple OSes cloud providers [1], [2]. to co-exist on a single machine. OS-based virtualization also This work investigates disk I/O performance and isolation in known as containerization creates multiple virtual units in the LXC [3] and KVM [4], which are representatives of OS- and userspace, known as containers in which processes run directly hypervisor-based virtualization, respectively. Disk I/O perfor- on the host OS. Containers share the same host kernel in oppo- mance and isolation are critical in database management sys- site of VMs but are isolated from each other through abstraction tems (DBMSs) which are a common service in the cloud. Many mechanisms of Linux namespaces and Cgroup resource control cloud-based services such as Dropbox [5], Facebook [6] and at the OS level. Saleforce [7] make use of DBMS. Microsoft’s SQL Azure [8] Containers are lightweight and bring the performance advan- and Google Cloud SQL [9] provide DBMS as a cloud service. tage over VMs. However, these advantages have some shortcom- DBMS makes excessive use of disk resource by orchestrating a ings. Since containers share the same kernel components like large number of I/O operations and causing journaling updates “buffer caches” and other the data structures, the performance iso- to ensure consistency. Journaling is indispensable for recovering lation among containers becomes weak and hard to accomplish. from unexpected crashes in file systems, and can not be turned off For example, if one container accesses many files to increase the in DBMS. pressure on a shared buffer cache, other containers suffer from This paper presents an updated and extended DBMS perfor- mance and isolation comparison of LXC and KVM. Our previous 1 Keio University, Yokohama, Kanagawa 223–8522, Japan work [10] examines MySQL performance and isolation under the a) [email protected] Cgroup / b) [email protected] old function of disk I O control [11]. This work adopts c 2020 Information Processing Society of Japan 369 Journal of Information Processing Vol.28 369–377 (July 2020) the newer fuction of Cgroupv2 [12] for disk I/O control, and con- presents the work related to ours. Finally Section 7 concludes siders the up-to-date software version of KVM, LXC, and Linux the paper. kernel. 2. Background We have extended the previous paper by first, examining how the Cgroup in general achieves the disk I/O control in a sepa- Hypervisors and containers have different architectures and rate experiment. Second, invistigating DBMS performance and properties. Section 2.1 overviews the internal design of isolation under Cgroupv2, a newer version of Cgroup with new hypervisor-based and container-based virtualization using KVM I/O controllers, new unified groups, and new implementation that and LXC, with highlighting the I/O performance and perfor- overcome many of Cgroup implemetation problems [13]. One mance isolation. Section 2.2 presents the disk I/O control by of the most important difference between the new and the old cgroup in KVM and LXC, and analyzes the drawbacks with it. Cgroup, is the support of buffered write along side non-buffered disk write when applying I/O control [12]. This new feature sup- 2.1 Virtual machine and Container poses to make the I/O control stronger and may affects the previ- In hypervisor-based virtualization, a hypervisor runs on top of ous results. Regardless of a newer developed linux kernel, KVM, a physical machine and virtualizes the underlying hardware re- and LXC software version, it was important to compare DBMS sources onto which a guest OS is installed, resulting in multiple performance isolation under these new circumstances. Finally, virtual machines running on the same physical machine. On the considering the latency metric in our experiments for a better ex- other hand, containers shift the layer of virtualization from the planation and details of the underlying problem. hardware level to the OS level. Containers rely on mechanisms The key finding of our work is that VMs outperform containers provided by the host OS; namespaces to separate containers log- in both I/O performance and isolation in DBMS. This finding is ically and Cgroup to control resources among containers. contrary to the general belief. The container is widely believed Figure 1 (a) and (b) show the architecture of KVM and LXC, to beat VM in performance because container does not suffer respectively. In KVM, a guest OS runs on top of a host OS. Hard- from virtualization overheads. However, according to our results, ware devices such as disk drives are virtualized with QEMU de- KVM outperforms LXC with up to 94% in MySQL throughput. vice emulator [14]. A disk I/O request is handled in a guest OS, Furthermore, LXC fails to achieve isolation; a container given passed to a device emulator, and then processed by a host OS. 30% share of disk bandwidth consumes 70% share although the In LXC, user processes run directly on a host OS without any resource control mechanism, cgroupv2, enforces disk I/O limits. virtualization overheads. A disk I/O request from a container is Our contributions consist of analyzing DBMS performance handled in the same way as ordinary processes on the host. and performance isolation in the container. The sharing of a In exchange for the performance advantage, LXC provides journaling module among containers degrades I/O performance weaker isolation of performance. Since containers share buffer in DBMS. To guarantee the consistency, existing file systems caches and other data structures at the OS-level, the activity in- typically use journaling with transactions. A journaling module side one container is likely to affect the performance of other batches updates from multiple containers into a transaction and containers through the shared kernel components. Even though commits the transaction to disk periodically or when fsync is the resource controlling mechanism Cgroupv2 enforces resource invoked. DBMS application invokes a lot of fsync() to ensure limits on containers, it can be bypassed as it will be shown in consistency and to write all updates to disk. If a single transaction Section 4.1. contains updates from multiple containers, each container has to wait until the data belonging to other containers is flushed. Jour- 2.2 Disk I/O Control by Cgroup naling also interferes with performance isolation among contain- Both of LXC and KVM rely on Cgroup to enforce disk I/O ers. When an update-intensive container commits a transaction, it control. Cgroup supports two policies for controlling disk I/O: adversely affects the performance of other containers. An update 1) I/O throttling and 2) proportional-weight. I/O throttling caps from the container is likely to be batched together with updates the maximum usage of I/O bandwidth or I/O request rates. In I/O from other containers. Even if each transaction contains updates throttling, a container/VM cannot make use of an idle resource solely from one container, the transactions are serialized in the journaling module and cannot be committed in parallel. In ad- dition, the journaling interferes with disk I/O control of Cgroup.