View metadata, citation and similar papers at core.ac.uk brought to you by CORE
provided by Repositori Institucional de la Universitat Jaume I
Increasing the Performance of Data Centers by Combining Remote GPU Virtualization with Slurm
Sergio Iserte∗, Javier Prades†, Carlos Reano˜ †, and Federico Silla† ∗Universidad Jaume I, Spain Email: [email protected] †Universidad Politecnica´ de Valencia, Spain Email: [email protected], [email protected], [email protected]
Abstract—The use of Graphics Processing Units (GPUs) enough resources for all the applications. presents several side effects, such as increased acquisition In order to address these concerns, the remote GPU vir- costs as well as larger space requirements. Furthermore, tualization mechanism could be used. This software mech- GPUs require a non-negligible amount of energy even while idle. Additionally, GPU utilization is usually low for most anism allows an application being executed in a computer applications. Using the virtual GPUs provided by the remote which does not own a GPU to transparently make use of GPU virtualization mechanism may address the concerns accelerators installed in other nodes of the cluster. This associated with the use of these devices. However, in the feature would not only reduce the costs associated with the same way as workload managers map GPU resources to acquisition and later use of GPUs, but would also increase applications, virtual GPUs should also be scheduled before job execution. Nevertheless, current workload managers are not the overall utilization of such accelerators because workload able to deal with virtual GPUs. In this paper we analyze the managers would assign those GPUs concurrently to several performance attained by a cluster using the rCUDA remote applications as far as GPUs have enough resources for all GPU virtualization middleware and a modified version of the of them. Notice, however, that workload managers need Slurm workload manager, which is now able to map remote to be enhanced in order to deal with virtual GPUs. This virtual GPUs to jobs. Results show that cluster throughput is doubled at the same time that total energy consumption is enhancement would basically consist in replacing the current reduced up to 40%. GPU utilization is also increased. per-GPU granularity by a finer granularity that should allow GPUs to be concurrently shared among several applications. Keywords-GPGPU; CUDA; HPC; virtualization; InfiniBand; data centers; Slurm; rCUDA Once this enhancement is performed, it is expected that over- all cluster performance is increased because the concerns previously mentioned would be reduced. I.INTRODUCTION In this paper we present a study of the performance of GPUs (Graphics Processing Units) are used to reduce the a cluster that makes use of the remote GPU virtualization execution time of applications. However, the use of GPUs mechanism along with an enhanced workload manager able is not exempt from side effects. For instance, when an MPI to assign virtual GPUs to waiting jobs. To that end, we have (Message Passing Interface) application not requiring GPUs used the rCUDA [7] remote GPU virtualization middleware is executed, it will typically spread across several nodes of along with a modified version of Slurm [3], which is now the cluster using all the CPU cores available in them. At that able to dispatch GPU-accelerated applications to nodes not point, the GPUs in the nodes executing the MPI application owning a GPU and, therefore, the use of a remote (or virtual) will not be available for other applications given that all the GPU must be scheduled. CPU cores are busy. Another important concern related to the use of GPUs in II.PERFORMANCE ANALYSIS clusters is has to do with the way that workload managers In this section we study the impact that using the remote like Slurm [11] track the accounting of resources. These GPU virtualization mechanism has on the performance of a workload managers use a fine granularity for resources data center. To that end, we have executed several workloads such as CPUs but not for GPUs. In this regard, they can in a cluster by submitting a series of randomly selected job assign individual CPU cores to different applications, what requests to the Slurm queues. After job submission we have allows a shared usage of the CPU sockets present in a measured several parameters such as total execution time server among several applications. On the contrary, workload of the workloads, energy required to execute them, GPU managers use a per-GPU granularity. That is, the entire utilization, etc. We have considered two different scenarios GPU is assigned to an application in an exclusive way, for workload execution. In the first one, the cluster uses thus hindering the possibility of sharing these accelerators CUDA and therefore applications can only use those GPUs among several applications even if those accelerators present installed in the same node where the application is being Table I: Applications used in this study. Configuration details for each application Application Configuration Execution time (s) Memory per GPU GPU-Blast 1 process with 6 threads in 1 node 21 1599 MB LAMMPS 4 single-thread processes in 4 different nodes 15 876 MB mCUDA-MEME 4 single-thread processes in 4 different nodes 165 151 MB GROMACS 2 processes with 12 threads each one in 2 nodes 167 BarraCUDA 1 single-thread process in 1 node 763 3319 MB MUMmerGPU 1 single-thread process in 1 node 353 2104 MB GPU-LIBSVM 1 single-thread process in 1 node 343 145 MB NAMD 4 processes with 12 threads each one in 4 nodes 241
executed. In this scenario, an unmodified version of Slurm using some CPU-only applications, as it could be the case has been used. In the second scenario we have made use in many data centers. The previous applications have been of rCUDA and therefore an application being executed in a combined in order to create three different workloads as given node can use any of the GPUs available in the cluster. shown in Table II. Moreover, a modified version of Slurm [3] has been used so As can be seen, the eight applications used present dif- that it is possible to schedule the use of remote GPUs. These ferent characteristics, not only in the amount of processes two scenarios will allow to compare the performance of a and threads used by each of them and their execution cluster using CUDA with that of a cluster using rCUDA. time but they also present different GPU usage patterns, In order to present the performance analysis, we first what includes both memory copies to/from GPUs and also present the cluster configuration and the workloads used in kernel executions. Therefore, although the set of applications the experiments. considered is finite, it may provide a representative sample of A. Cluster Configuration a workload typically found in current data centers. Actually, the set of applications in Table I could be considered from The testbed used in this study is based on the use of a two different point of views. In the first one, the exact cluster composed of 16 1027GR-TRF Supermicro servers. computations performed by each application would receive Each of the 16 servers includes two Intel Xeon E5-2620 v2 the main focus. In this point of view, some applications processors (six cores with Ivy Bridge architecture) operating address similar problems, like LAMMPS, GROMACS, and at 2.1 GHz and 32 GB of DDR3 SDRAM memory at NAMD. However, in the second point of view, the exact 1600 MHz. They also have a Mellanox ConnectX-3 VPI problem addressed by each application is not the focus but single-port FDR InfiniBand adapter connected to a Mellanox applications are seen as processes that keep CPUs and GPUs Switch SX6025 (InfiniBand FDR compatible) to exchange busy during some amount of time and require some amount data at a maximum rate of 56 Gb/s. Furthermore, an of memory. Now the focus is the amount of resources re- NVIDIA Tesla K20 GPU is installed at each node. One quired by each application and the time that those resources additional node (without GPU) has been leveraged to execute are kept busy. From this second perspective, the set of the central Slurm daemon responsible for scheduling jobs. applications in Table I becomes even more representative. B. Workloads Table III displays the Slurm parameters used for launching Several workloads have been considered in order to each of the applications. The use of real and virtual GPUs provide a more representative range of results. The work- has been considered in the table. Notice that once Slurm loads are composed of the following applications: GPU- has been enhanced, Slurm users are able to submit jobs BLAST [10], LAMMPS [1], mCUDA-MEME [6], GRO- to the system queues in three different modes: (1) CUDA: MACS [9], BarraCUDA [5], MUMmerGPU [4], GPU- LIBSVM [2], and NAMD [8]. Table I provides additional Table II: Workload composition information about the applications used in this work, such as the exact execution configuration used for each of the Workload applications, their execution time, and the GPU memory re- Application Set 1 Set 2 Set 1+2 quired by each application. For those applications composed GPU-Blast 112 57 of several processes or threads, the amount of GPU memory LAMMPS 88 52 mCUDA-MEME 99 55 depicted in Table I refers to the individual needs of each GROMACS 101 47 particular thread or process. Notice that the amount of GPU BarraCUDA 112 51 memory is not specified for the GROMACS and NAMD MUMmerGPU 88 52 GPU-LIBSVM 99 37 applications because we are using non-accelerated versions NAMD 101 49 of these applications. The reason for this choice is simply Total 400 400 400 to increase the heterogeneity degree of the workloads by Table III: Slurm launching parameters Application Launch with CUDA Launch with rCUDA exclusive Launch with rCUDA shared GPU-Blast -N1 -n1 -c6 –gres=gpu:1 -n1 -c6 –rcuda-mode=excl –gres=rgpu:1 -n1 -c6 –rcuda-mode=shar –gres=rgpu:1:1599M LAMMPS -N4 -n4 -c1 –gres=gpu:1 -n4 -c1 –rcuda-mode=excl –gres=rgpu:4 -n4 -c1 –rcuda-mode=shar –gres=rgpu:4:876M mCUDA-MEME -N4 -n4 -c1 –gres=gpu:1 -n4 -c1 –rcuda-mode=excl –gres=rgpu:4 -n4 -c1 –rcuda-mode=shar –gres=rgpu:4:151M GROMACS -N2 -n2 -c12 -N2 -n2 -c12 -N2 -n2 -c12 BarraCUDA -N1 -n1 -c1 –gres=gpu:1 -n1 -c1 –rcuda-mode=excl –gres=rgpu:1 -n1 -c1 –rcuda-mode=shar –gres=rgpu:1:3319M MUMmerGPU -N1 -n1 -c1 –gres=gpu:1 -n1 -c1 –rcuda-mode=excl –gres=rgpu:1 -n1 -c1 –rcuda-mode=shar –gres=rgpu:1:2104M GPU-LIBSVM -N1 -n1 -c1 –gres=gpu:1 -n1 -c1 –rcuda-mode=excl –gres=rgpu:1 -n1 -c1 –rcuda-mode=shar –gres=rgpu:1:145M NAMD -N4 -n48 -c1 -N4 -n48 -c1 -N4 -n48 -c1
no change is required to the original way of launching all the samples after completing workload execution. The jobs. (2) rCUDA shared: the job will use the remote virtual nvidia-smi command was used for polling the GPUs. In GPUs, which will be shared with other jobs, and (3) rCUDA a similar way, Figure 1(c) shows total energy required for exclusive: the job will use the new remote virtual GPUs completing workload execution. Energy has been measured but it will not share them with other jobs. In the first case, by polling once every second the power distribution units CUDA will be used (column labeled “Launch with CUDA”). (PDUs) present the cluster. Used units are APC AP8653 In the second and third cases, rCUDA will be leveraged. PDUs, which provide individual energy measurements for In the second approach, the column labeled as “Launch each of the servers connected to them. After workload com- with rCUDA shared” shows that the amount of memory pletion, the energy required by all servers was aggregated required at each GPU must be specified in the submission to provide the measurements in Figure 1(c). command. In the third case, the column labeled as “Launch with rCUDA exclusive” shows that no GPU memory is As can be seen in Figure 1(a), workload “Set 1” presents declared because the GPU assigned to a given job will be the smallest execution time, given that it is composed of not shared with other jobs. the applications requiring the smallest execution times. Fur- thermore, using rCUDA in a shared way reduces execution C. Performance Analysis time for the three workloads. In this regard, execution time Figure 1 shows, for each of the workloads depicted is reduced by 48%, 37%, and 27% for workloads “Set 1”, in Table II, the performance when CUDA is used along “Set 2”, and “Set 1+2”, respectively. Notice also that the use with the original Slurm job scheduler (results labeled as of remote GPUs in an exclusive way also reduces execution “CUDA”) as well as the performance when rCUDA is used time. In the case for “Set 2” this reduction is more noticeable in combination with the modified version of Slurm. In this because when CUDA is used the NAMD application (with case, label “rCUDAex” refers to the results when remote 101 instances in the workload) spans over 4 complete GPUs are used in an exclusive way by applications whereas nodes thus blocking the GPUs in those nodes, which cannot label “rCUDAsh” refers to the case when remote GPUs can be used by any accelerated application during the entire be shared among several applications. Among both rCUDA execution time of NAMD (241 seconds). On the contrary, uses, the shared one is the most interesting one. The exclu- when “rCUDAex” is leveraged, the GPUs in those four sive case is considered in this paper only for comparison nodes are accessible from other nodes and therefore they can purposes. Figure 1(a) shows total execution time for each be used by other applications being executed at other nodes. of the workloads. Figure 1(b) depicts the averaged GPU Regarding GPU utilizacion, Figure 1(b) shows that the use utilization for all the 16 GPUs in the cluster. Data for GPU of remote GPUs helps to increase overall GPU utilization. utilization has been gathered by polling each of the GPUs Actually, when “rCUDAsh” is used with “Set 1” and “Set in the cluster once every second and afterwards averaging 1+2”, average GPU utilization is doubled with respect to the