This is a repository copy of Energy-based Cost Model of Virtual Machines in a Cloud Environment.

White Rose Research Online URL for this paper: http://eprints.whiterose.ac.uk/141570/

Version: Accepted Version

Proceedings Paper: Aldossary, M orcid.org/0000-0001-6772-700X and Djemame, K (2019) Energy-based Cost Model of Virtual Machines in a Cloud Environment. In: Proceedings of the Fifth International Symposium on Innovation in Information and Communication Technology (ISIICT 2018). ISIICT 2018: The Fifth International Symposium on Innovation in Information and Communication Technology, 31 Oct - 01 Nov 2018, Amman, Jordan. IEEE . ISBN 978-1-5386-6224-3 https://doi.org/10.1109/ISIICT.2018.8613288

© 2018 Crown. This is an author produced version of a paper published in Proceedings of the Fifth International Symposium on Innovation in Information and Communication Technology (ISIICT 2018). Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Uploaded in accordance with the publisher's self-archiving policy.

Reuse Items deposited in White Rose Research Online are protected by copyright, with all rights reserved unless indicated otherwise. They may be downloaded and/or printed for private study, or other acts as permitted by national copyright laws. The publisher or other rights holders may allow further reproduction and re-use of the full text version. This is indicated by the licence information on the White Rose Research Online record for the item.

Takedown If you consider content in White Rose Research Online to be in breach of UK law, please notify us by emailing [email protected] including the URL of the record and the reason for the withdrawal request.

[email protected] https://eprints.whiterose.ac.uk/ Energy-based Cost Model of Virtual Machines in a Cloud Environment

Mohammad Aldossary1, 2, and Karim Djemame2 1Department of Computer Science, College of Arts and Science, Prince Sattam Bin Abdulaziz University, Al Kharj, Saudi Arabia [email protected] 2School of Computing, University of Leeds, Leeds, U.K. {scmmald, k.djemame}@leeds.ac.uk

Abstract— The cost mechanisms employed by different service difficult to identify and not directly measured. Hence, the power providers significantly influence the role of cloud computing consumption of VMs can be gathered from their underlying within the IT industry. With the increasing cost of electricity, PMs, which is still difficult to achieve [8], [9]. Cloud providers consider power consumption as one of the major Many of the existing approaches model and identify the cost factors to be maintained within their infrastructures. energy consumption in PMs, as presented in [3], [10], [11] and Consequently, modelling a new cost mechanism for Cloud services that can be adjusted to the actual energy costs has attracted the the energy consumption in VMs, as proposed in [12], [13], by attention of many researchers. This paper introduces an Energy- considering only the CPU utilisation. Therefore, understanding based Cost Model that considers energy consumption as a key how the resource usage affects the power consumption is parameter with respect to the actual resource usage and the total required. An experimental study that investigates the effect of cost of the Virtual Machines (VMs). A series of experiments the resource usage (e.g. CPU, memory, disk and network) on conducted on a Cloud testbed show that this model is capable of the power consumption is presented in [14], [15]. The findings estimating the actual cost for heterogeneous VMs based on their show that the CPU utilisation correlates well with the power resource usage with consideration of their energy consumption. consumption, as supported in other work, for example [3], [10], Keywords— Cloud Computing, Cost Model, Resource Usage, [16]. Thus, the proposed model in this paper follows the same Power Consumption, Energy Efficiency. approach and considers the CPU utilisation only when modelling and identifying the energy consumption for the VMs. I.! INTRODUCTION Considering the challenges in Cloud cost models, the aim of The cost mechanisms that are offered by Cloud service this paper is to enable cost and energy awareness of resource providers have become even more sophisticated, as customers usage at the VM level, which contributes to overcome the are charged per month, hour or minute based on the resources challenge of identifying the actual energy usage and total cost they utilise. Nevertheless, there are still limited, as customers for the VMs. The outcome of this research can be used to help are charged based on pre-defined tariffs for the resource usage. make efficient decisions supported by cost and energy These pre-defined tariffs do not consider the variable cost of awareness. This paper’s main contributions are summarised as energy [1], which is considered as one of the biggest operational follows: cost factor by Cloud infrastructure providers. Consequently, •! A proposed Cost Modeller within Cloud system architecture modelling a new cost mechanism for Cloud services that can be to assess the actual consumption of Cloud infrastructure adjusted to the actual energy costs has attracted the attention of resources. many researchers [2]–[4]. In a Cloud environment each Physical Machine (PM) can run •! An Energy-based Cost Model that measures the actual cost a single VM or multiple VMs simultaneously. These VMs can for heterogeneous VMs by considering their resource usage be homogeneous or heterogeneous based on their and power consumption. characteristics, for example, the number of virtual CPUs •! An evaluation of the proposed model in an existing Cloud (vCPUs) and memory size. Thus, these parameters should be testbed in order to demonstrate its usability with clear cost taken into consideration along with their power consumption savings. when modelling and identifying the total cost for the VMs. The remainder of this paper is organised as follows: a Therefore, an energy-based cost model that considers energy discussion of the related work is summarised in Section II. consumption as a key parameter with respect to the actual Section III presents the system architecture that supports resource usage and the total cost is proposed. energy, performance and cost awareness of Cloud infrastructure The PMs power consumption can be directly measured services, followed by the descriptions of the required through monitoring tools either internal such as Running components and their interactions within the proposed Average Power Limit (RAPL) [5] and Intelligent Platform architecture. Section IV presents an energy-based cost model. Management Interface (IPMI) [6] or external such as Watt’s Up Section V presents the experimental setup followed by Power Meter [7]. However, VMs power consumption is experiments and evaluation in Section VI. Finally, Section VII concludes this paper and discusses the future work.

II.! RELATED WORK This section discusses the cost that is associated with the resource usage and power consumption of the VMs in Cloud environment. In this context, three cost models based on average workload usage are presented in [17]. Each of the models works with a specific metric in order to calculate the cost for a given workload. The first model calculates the cost using the average CPU utilisation. The second model quantified the cost based on the difference between the maximum and average CPU utilisation. Finally, the third model measures the cost based on the idle CPU utilisation. According to the usage scenario, all three models have individual advantages and can be applied to define the provisioning cost of Cloud providers. Fig. 1. System Architecture. However, all the models don't consider the cost of energy in their calculation. Another cost model based on CPU workload This proposed architecture summaries the high-level details is proposed in [18]. This approach is only applicable for non- of these three layers and mainly focuses on IaaS layer where the virtualised scenarios. Further, a cost optimisation algorithm to service operation takes place. In the IaaS layer, the admission, schedule the workload and minimise the execution time has allocation and management of VMs are performed through the been demonstrated in [19]. The authors have considered budget interaction between a number of components. These and deadline constraints without taking into consideration the components and their interactions within this architecture are overhead of energy consumption. summarised below. The highlighted component Cost Modeller The energy consumption-based cost models have been is the main component including the other contributions of our investigated in various research studies [20]–[25] in different work. The overall aim of the Cost Modeller is to advance aspects. For instance, an optimisation model to reduce the beyond the state of art via considering the awareness of energy operational cost is presented in [20]. The model considers two consumption, performance variation and total cost of Cloud factors in order to reduce the operational cost: 1) Dynamic infrastructure services. Voltage/Frequency Scaling (DVFS), and 2) turning the PMs A.! SLA Manger on/off over a time horizon. Furthermore, an energy-aware resource provisioning framework for cloud computing by The Service Level Agreement (SLA) Manager is responsible considering cost is proposed in [21]. The proposed framework for monitoring and measuring the application SLA’s agreed is evaluated using Google traces collected over a 29-day period terms at IaaS level. This component interacts with the VM from a Google cluster and conclusions with large energy Manager to check availability and capability of resources in savings. However, both of the studies presented above do not order to determine the SLA offer and interacts with the Cost consider the heterogeneity of PMs or VMs when designing their Modeller to assign the cost to the offered terms [26]. energy and cost models. B.! VM Manger Moreover, an example of a cost model for Infrastructure as The Virtual Machine (VM) Manager component is a Service (IaaS) provider to reduce the energy consumption is responsible for managing the VMs at service operation level. recently introduced in [22]. This approach motivated us to This component considers the best decision based on investigate the relationship between energy consumption and predefined policies (e.g. performance-aware, energy-aware and VMs workload in the cloud environment. cost-aware) in order to improve resource usage and reduce the Compared with the work presented in this paper, we propose energy cost and consequently the total cost of the VMs. In case an energy-based cost model that considers energy consumption as a key parameter with respect to the actual resource usage and of service performance degradation, this component will the total cost. Further, our approach demonstrates the cost and interact with the Cost Modeller to request measures or energy efficiency by considering the heterogeneity of PMs and predictions related to the resource usage, power consumption VMs. and cost that VMs would have for any particular host. C.! Infrastructure Manger III.! PROPOSED SYSTEM ARCHITECTURE The Infrastructure Manager manages the entire physical Cloud computing system architecture consists of three infrastructure that includes e.g. processors, memory, storage standard layers, which are Software as a Service (SaaS) where devices, networking and hardware energy meters. In this the service creation takes place, Platform as a Service (PaaS) component, the PMs are managed by the Virtualisation where the service deployment takes place, and Infrastructure as Manager (Hypervisor) that allows sharing of the physical a Service (IaaS) where the service operation takes place, as resources among the VMs. depicted in Figure 1. D.! Monitoring Infrastructure Equation 2. The active energy of the PM is attributed to The main role of this component is to monitor the PMs and heterogeneous and homogeneous VMs by considering the VM VMs resource usage (e.g. CPU, memory, network and disk), CPU utilisation and number of vCPUs assigned for each VM. PMs’ energy consumption (e.g. Watts) and performance related metrics (e.g. CPU utilisation and memory usage) during the 6789:;<=>?≅ (1) VM�∃%&∋()∗ = ���∃%&∋()∗ × 5 ΧΕΦΓΗΙ Μ execution of the applications at the service operation level. ∑ϑΚΛ 67Β9:;<=>?≅ where is the idle power consumption of the PM E.! Cost Modeller where the��� VMs∃%&∋()∗ are hosted; is the number of the VM�Ν∋Ο/Π(ΘΡ The overall aim of this component is to demonstrate that: 1) vCPUs assigned to the given VMx; is the number of enable the awareness of energy consumption, performance VMs running on the same PM; and ��ΠΤΥς− is the number variation and total cost of the VMs at the operational level, and of vCPUs assigned to a member ofVM the� VMsΝ∋Ο/Π(ΘΡ set hosted by the 2) predict the workload and power consumption as well as the same PM. total cost of the VMs at service operation. Therefore, this component supports: VM�+,−./∋()∗ = (���()∗ − ���∃%&∋()∗) × 1) Energy-based Cost Model that provides measuring the 678(?Ι[∴×9:;<=>?≅) (2) 5 ΧΕΦΓΗΙ Μ actual resource usage, power consumption and total cost ∑ϑΚΛ 67Β(?Ι[∴×9:;<=>?≅) relating to the VMs. The details of this model will be discussed in Section IV. where is the total power consumption of the PM, 2) Energy-based Cost Prediction Framework that predicts from which��� the()∗ PM’s idle power is deducted to the resource usage, power consumption and total cost for the identify the PM’s active power; ���∃%&∋()∗ is the CPU utilisation VMs. The details of this framework are presented in [14]. of the given VMx; and � ��is Θ−.& the CPU utilisation of a 3) Performance and Energy-based Cost Prediction member of the VMs set hosted���Θ−.& by the same PM. Framework that supports actuators (e.g. re-allocating, live Thus, the total power consumption, , for each VM at migrating and auto-scaling VMs) to tackle the performance any given time can be identified by summingVM�()∗ up its both idle variation and attempt to get the performance to the acceptable and active power consumption [8], as shown in Equations 3 and level with minimal impact on cost. The details of this 4, respectively. framework are presented in [15], [27].

6789:;<=>?≅ IV.! ENERGY-BASED COST MODEL ()∗ ∃%&∋()∗ 5 ΧΕΦΓΗΙ Μ VM� = ��� × ∑ϑΚΛ 67Β9:;<=>?≅ + The energy-based cost model introduced in this paper works 678(?Ι[∴×9:;<=>?≅) ( ()∗ ∃%&∋()∗) 5 ΧΕΦΓΗΙ Μ by firstly measuring the VMs workload as well as the PMs ��� − ��� × ∑ϑΚΛ 67Β(?Ι[∴×9:;<=>?≅) energy consumption through a monitoring system. After that, (3) this model would attribute the PM’s energy to the VMs in order to obtain the energy consumption for each VM. Then, the VMs which is equal to: total cost can be obtained based on the measured workload and (4) energy consumption for each VM. In order to achieve that VM�()∗ = VM�∃%&∋()∗ + VM�+,−./∋()∗ several steps are required: where is the total power consumption for one VM VM�()∗ Step 1: the VMs workload is measured through a monitoring (idle and active power) measured by Watt. is the VM�Ν∋Ο/Π(ΘΡ system [28] for each VM. Similarly, the PMs power requested number of vCPU and is the VM CPU VM�Θ−.& consumption can be directly measured through a monitoring utilisation. ⊥_,ΤΥς− is the total number of vCPU ∑Βα VM�Ν∋Ο/Π(ΘΡ system [28] for each PM, since each of the PM has a WattsUp for all VMs in the same PM. The is idle power [7] meter attached to it (see Section V). consumption and is the total��� power∃%&∋()∗ consumption for a single PM. ���()∗ Step 2: After the VMs workload and PMs power Hence, the presented energy-based cost model can fairly consumption are measured, the second step is to attribute the attribute the idle and active energy consumption of a PM to the PM power consumption to the new requested VM and to the same or different sizes of VMs in terms of the allocated vCPUs VMs already running on the PM. Hence, the power for each VM. For instance, when both a small VM with 1 vCPU consumption for the new VM can be done in two parts: 1) VMs and a large VM with 4 vCPUs are being fully utilised on the idle power consumption, based on the number of same PM, the large VM would have about four times the value VM�∃%&∋()∗ vCPUs assigned to each VM [8], as shown in Equation 1. The in terms of energy consumption as compared to the small VM idle energy of the PM (means the PM is running with no (see Section VI). This way the energy consumption can be fairly workload) is attributed to homogeneous and heterogeneous attributed based on the actual physical CPU utilisation used by VMs by considering the size of each VM in terms of the vCPUs each VM. assigned to them, and 2) VMs active power consumption, After identifying the power consumption for each VM, based on the VM CPU utilisation as well as the convert power to energy is required using Equation 5, since the VM�+,−./∋()∗ number of vCPUs assigned to each VM [8], as shown in energy providers charge by the Kilowatt per hour (kWh). 678>δε γ.η∋≅ (5) of the PM has a WattsUp meter [7] attached to directly measure VM�βς∋∗χΒ = αφφφ × ιϕφφ the power consumption on a per second basis for each PM. The where is the energy consumption of the VM, measured power values are then pushed to Zabbix [28], which VM�βς∋∗χΒ measured by Kilowatt-hour. is the total power is the monitoring infrastructure tool used in this testbed. consumption for one VM (idle andVM active�()∗ power) measured by Additionally, Zabbix also monitors the resources usage such as Watt times the period of time, measured by second. CPU, memory, network and disk, for each of the running PMs and VMs. The PMs power usage along with the VMs resource Step 3: The final step in this model is to obtain the total cost usage are sent to the Cost Modeller, which is responsible for of the VM based on the actual resource usage from Step 1 and measuring energy consumption along with the total cost for the power consumption from Step 2. The following Equation 6 is VMs. used:

���Θ−.& VM�γΤ−κ&ΠΤΡ− = λ5VM�Ν∋Ο/Π(ΘΡ × Μ 100

× (���� ��� ���� × ����Ρ){

+ |VM�Ν+_ΘΡκχ∋ × (���� ��� �� × ����Ρ)

+ |VM�.ΡΘΡκχ∋ × (���� ��� �� × ����Ρ)

+ |VM�∋−ΘΡκχ∋ × (���� ��� �� × ����Ρ)

+ VM�βς∋∗χΒ × ���� ��� ��ℎ (6) Fig. 2. Cloud Testbed Architecture.

where is the total cost of a single VM. VM�isγΤ−κ&ΠΤΡ− the resource usage of RAM times the cost for thatVM� resourceΝ+_ΘΡκχ∋ for a period of time and so on for each resource such as CPU, disk and network. is the energy consumption of the VM times the energyVM� βς∋∗χΒcost as announced by the energy providers.

V.! EXPERIMENTAL SETUP This section describes the environment and the details of the experiments conducted in order to evaluate the proposed energy-based cost model. A number of experiments have been Fig. 3. Monitoring Infrastructure. conducted on an existing Cloud Testbed. The details of this testbed and how it is monitoring the resources usage and energy consumption at the PM and VM levels will be discussed next. C.! Specifications of PMs and VMs A.! Cloud Testbed In terms of the PMs and VMs considered in the experiments presented in this paper, Tables I and II summarises the The Cloud Testbed consists of a cluster of 8 commodity Dell configurations of the PMs and VMs, respectively. servers, and each one of these servers has Centos version 6.6 installed as its operating system (OS). Two of these servers TABLE I. ! CONFIGURATIONS OF THE PMS. with four core X3430 and eight core E3-1230 V2 Intel Xeon Hostname CPU Memory Disk CPU were used. Also, each server has a total of 16GB RAM A four core Total of 16GB 250GB (Model X3430 Intel of RAM (four Number: WDC and 250GB up to 500GB of SATA HDD. Additionally, the Xeon CPU modules of WD2502ABYS) Host A testbed has a Network File System (NFS) share running on the (default clock 4GB DDR3 at head node of the cluster and providing a 2TB total storage for speed of 1600MHz) VM images. The architecture of this testbed is shown in Figure 2.40GHz) An eight core Total of 16GB 500GB (Model 2. The testbed utilises Virtual Infrastructure Manager (VIM), E3-1230 V2 of RAM (two Number: OpenNebula [29] version 4.10, and Virtual Machine Manager Intel Xeon modules of ST1000NM0033) Host B (VMM), KVM [30] hypervisor version 4.0.1 along with the CPU (default 8GB DDR3 at Linux Kernel version 2.6.32.24. clock speed 1600MHz) of 3.30GHz) B.! Monitoring Infrastructure The resources usage and energy monitoring on the Cloud Testbed is depicted in Figure 3. At the physical host level, each

(a) (b)

(c) (d)

Fig. 4. The Workload Results for Medium VM (for 30 minutes).

Host (A) Host (B)

Fig. 5. Power Consumption for Medium VM on (Host A and Host B).

TABLE II. ! CONFIGURATIONS OF THE VMS. total cost at VM level. Furthermore, the proposed model focuses on overall cost savings of the VMs that can be obtained Instance Type vCPU Memory Disk Network Small VM 1 vCPU 1GB 10GB 1GB when running the VMs on different hosts have different energy Medium VM 2 vCPUs 2GB 10GB 1GB characterisation. Large VM 4 vCPUs 4GB 10GB 1GB In order to design such experiments, a software tool called Stress-ng [35] is used. The aim is to generate synthetic periodic Rackspace [31] is used as a reference for the VMs workload patterns to represent real workload patterns of cloud configurations. Three types of VMs, small, medium and large applications by stressing all the resources, i.e. CPU, RAM, disk are provided with different capacities. The cost of the virtual and network on different types of VMs to their full utilisation. resources are set according to ElasticHosts [32] and VMware All the experiments are repeated five times 30 minutes each and [33]; and the cost of energy according to [34]. the statistical analysis is performed to consider the mean values of the results and eliminate any anomalies due to the dynamicity VI.! EXPERIMENTS AND EVALUATION of the cloud. A.! Design of Experiments The following experiments have been designed to show various aspects of energy consumption at the PM and VM !!!!!A number of direct experiments have been conducted on the levels. This way can help to assess how the power consumption Cloud Testbed. The overall aim of these experiments is to of the PMs is attributed to the VMs and explore the impact of evaluate the capability of the energy-based cost model for the actual resource usage and power consumption on the VMs measuring the actual resource usage, power consumption and total cost when being run on different hosts.

Fig. 6. PM Mean Power Consumption Attributed to each VM - Host A. Fig. 7. PM Mean Power Consumption Attributed to each VM - Host B.

Fig. 8. Mean Energy Consumption per VM (for 30 minutes) - Host A. Fig. 9. Mean Energy Consumption per VM (for 30 minutes) - Host B.

Fig. 10. The VMs Total Cost on (Host A and Host B). Fig. 11. The VMs Cost Saving on Host B.

active power consumption than Host A; therefore, when the B.! Results and Discussion VM is running on Host A, it has more energy consumption as The conducted experiments show the results for three types compared to when running on Host B, as shown in Figure 5. of VMs, small, medium and large when being run on different