Empirical Analysis of Database Server Scalability Using an N-Tier Benchmark with Read-Intensive Workload
Total Page:16
File Type:pdf, Size:1020Kb
Empirical Analysis of Database Server Scalability Using an N-tier Benchmark With Read-intensive Workload Simon Malkowski Deepal Jayasinghe Markus Hedwig CERCS CERCS Chair of Information Systems Georgia Institute of Technology Georgia Institute of Technology Albert-Ludwigs-University 266 Ferst Drive 266 Ferst Drive Platz der Alten Synagoge 30332-0765 Atlanta, USA 30332-0765 Atlanta, USA 79805 Freiburg, Germany [email protected] [email protected] [email protected] freiburg.de Junhee Park Yasuhiko Kanemasa Calton Pu CERCS Fujitsu Laboratories Ltd. CERCS Georgia Institute of Technology 1-1, Kamikodanaka 4-chome Georgia Institute of Technology 266 Ferst Drive Nakahara-ku 266 Ferst Drive 30332-0765 Atlanta, USA Kawasaki 211-8588, Japan 30332-0765 Atlanta, USA [email protected] [email protected] [email protected] ABSTRACT Categories and Subject Descriptors The performance evaluation of database servers in N-tier C.2.4 [Computer-Communication Networks]: Distri- applications is a serious challenge due to requirements such buted Systems—distributed databases;C.4[Performance as non-stationary complex workloads and global consistency of Systems]: Performance attributes; H.2.4 [Database management when replicating database servers. We con- Management]: Systems—distributed databases ducted an experimental evaluation of database server scal- ability and bottleneck identification in N-tier applications using the RUBBoS benchmark. Our experiments are com- General Terms prised of a full scale-out mesh with up to nine database Measurement, Performance, Experimentation, Verification. servers and three application servers. Additionally, the four- tier system was run in a variety of configurations, includ- Keywords ing two database management systems (MySQL and Post- greSQL), two hardware node types (normal and low-cost), Bottleneck, Database replication, Distributed systems, Mid- and two database replication techniques (C-JDBC and dleware, N-tier applications, RUBBoS. MySQL Cluster). In this paper we present the analysis of results generated with a read-intensive interaction pat- 1. INTRODUCTION tern (browse-only workload) in the client emulator. These Scalability of N-tier applications in general and database empirical data can be divided into two kinds. First, for a servers in particular present serious challenges to both aca- relatively small number of servers, we find simple hardware demia and industry. This is largely due to requirements resource bottlenecks. Consequently, system throughput in- such as non-stationary workloads and dependencies created creases with an increasing number of database (and appli- by requests that are passed between web servers, applica- cation) servers. Second, when sufficient hardware resources tion servers, and database servers. Some of these challenges are available, non-obvious database related bottlenecks have have been previously referred to as “gaps between theory and been found that limit system throughput. While the first practice” [10] in database replication. We attempt to start kind of bottlenecks shows that there are similarities between bridging these gaps with a large-scale experimental evalua- database and application/web server scalability, the second tion of database server scalability using a N-tier application kind of bottlenecks shows that database servers have sig- benchmark (RUBBoS [6]). For our study we have collected nificantly higher sophistication and complexity that require more than 500GB of data in over 6,000 experiments. These in-depth evaluation and analysis. experiments cover scale-out scenarios with up to nine data- base servers and three application servers. The configura- tions were varied using two relational database management systems (MySQL and PostgreSQL), two database replica- Permission to make digital or hard copies of all or part of this work for tion techniques (C-JDBC and MySQL Cluster), workloads personal or classroom use is granted without fee provided that copies are ranging from 1,000 to 13,000 concurrent users, and two dif- not made or distributed for profit or commercial advantage and that copies ferent hardware node types. bear this notice and the full citation on the first page. To copy otherwise, to The initial analysis of our extensive data produced several republish, to post on servers or to redistribute to lists, requires prior specific interesting findings. First, we documented detailed node- permission and/or a fee. SAC’10 March 22-26, 2010, Sierre, Switzerland. level bottleneck migration patterns among the database, ap- Copyright 2010 ACM 978-1-60558-638-0/10/03 ...$10.00. plication, and clustering middleware tiers when workload (c) Sample C-JDBC (a) Software setup. (b) Hardware node setup. topology (1/2/1/2L). Function Software Type Components !"#$% +,-&."'% Web server Apache 2.0.54 Normal Processor Xeon 3GHz 64-bit &"'("'% /011,"$ Apache Tomcat Memory 2GB 23'"% Application server 5.5.17 Network 6 x 1Gbps Cluster middleware C-JDBC 2.0.2 Disk 2 x 146GB 10,000rpm MySQL 5.0.51a Low-cost Processor PIII 600Mhz 32-bit Database server PostgreSQL 8.3.1 Memory 256MB MySQL Cluster 6.2.15 Network 5 x 100Mbps GNU/Linux Redhat FC4 Operating system Disk 13GB 7,200rpm )**$% 45$% Kernel 2.6.12 &"'("'&% &"'("'&% System monitor Systat 7.0.2 Table 1: Details of the experimental setup on the Emulab cluster. and number of servers increased gradually. These bottle- consists of 24 different interactions (involving all tiers) such necks were correlated with the overall system performance as register user, view story, and post comments. The bench- and used to explain the observed characteristics. Second, mark includes two kinds of workload modes: browse-only the explicit comparison of performance differences between and read/write interaction mixes. In this paper we solely varying system configurations yielded concrete configuration use the browse-only workload for our experiments. planning insights, which have escaped analytical approaches Each of our experiment trial consists of three periods: so far. Third, the identification of non-obvious bottlenecks ramp-up, run, and ramp-down. In our experiments, the tri- showed “migratory bottleneck phenomena” that can arise in als consist of an 8-minute ramp-up, a 12-minute run period, real systems and result in surprising performance effects. and a 30-second ramp-down. Performance measurements The paper makes two main contributions. First, using (e.g., CPU or network bandwidth utilization) are taken dur- an automated experiment creation and management sys- ing the run period using Linux accounting log utilities (i.e., tem, we collected a significant amount of data on database Sysstat) with a granularity of one second. server performance in N-tier applications. We present an experimental evaluation of database server scalability in a 2.2 Database Replication Techniques read-intensive scenario (i.e., setup, description, GBs of trace In this subsection we briefly introduce the two different data- data, and runtime statistics). Second, our analysis of this base replication techniques that we have used in our experi- data set shows interesting bottlenecks and their characteris- ments. Please refer to the cited sources for a comprehensive tics as mentioned above, which are potentially widely appli- introduction. cable. The detailed bottleneck migration patterns and ex- C-JDBC [11], is an open source database cluster middle- plicit performance comparisons, for example, can be useful ware, which provides a Java application access to a cluster of for datacenter administrators that manage N-tier systems by databases transparently through JDBC. The database can providing measured results on concrete N-tier system per- be distributed and replicated among several nodes. C-JDBC formance and resource utilization. balances the queries among these nodes. C-JDBC also han- The remainder of this paper is structured as follows. Sec- dles node failures and provides support for check-pointing tion 2 outlines experimental setup and methods. Section 3 and hot recovery. presents the analysis of our data on database scale-out through MySQL Cluster [4] is a real-time open source transac- database replication. In Section 4 we detail an interesting tional database designed for fast, always-on access to data scenario that shows how our methodology can be used to un- under high throughput conditions. MySQL Cluster utilizes cover unexpected bottlenecks with surprising performance a “shared nothing” architecture, which does not require any effects. Related work is summarized in Section 5, and Sec- additional infrastructure and is designed to provide 99.999% tion 6 concludes the paper. data availability. In our experiment we used “in-memory” version of MySQL Cluster, but that can be configured to use disk-based data as well. MySQL Cluster uses the ND- 2. EXPERIMENTAL SETTING BCLUSTER storage engine to enable running several nodes with MySQL servers in parallel. 2.1 Benchmark Applications Among N-tier application benchmarks, RUBBoS has been 2.3 Hardware Setup used in numerous research efforts due to its real production The experiments used in this paper were run in the Emulab system significance. Readers familiar with this benchmark testbed [1] with two types of servers. Table 1(b) contains can skip to Table 1(a), which outlines the concrete choices a summary of the hardware used in our experiments. Nor- of software components used in our experiments. mal and low-cost nodes were connected over 1,000 Mbps and RUBBoS [6] is an N-tier e-commerce system modeled on 100