Market Models for Federated Clouds Ioan Petri, Javier Diaz-Montes, Mengsong Zou, Tom Beach, Omer Rana and Manish Parashar
Total Page：16
File Type：pdf, Size：1020Kb
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2015.2415792, IEEE Transactions on Cloud Computing 1 Market models for federated clouds Ioan Petri, Javier Diaz-Montes, Mengsong Zou, Tom Beach, Omer Rana and Manish Parashar Abstract—Multi-cloud systems have enabled resource and service providers to co-exist in a market where the relationship between clients and services depends on the nature of an application and can be subject to a variety of different Quality of Service (QoS) constraints. Deciding whether a cloud provider should host (or finds it profitable to host) a service in the long-term would be influenced by parameters such as the service price, the QoS guarantees required by customers, the deployment cost (taking into account both cost of resource provisioning and operational expenditure, e.g. energy costs) and the constraints over which these guarantees should be met. In a federated cloud system users can combine specialist capabilities offered by a limited number of providers, at particular cost bands – such as availability of specialist co-processors and software libraries. In addition, federation also enables applications to be scaled on-demand and restricts lock in to the capabilities of a particular provider. We devise a market model to support federated clouds and investigate its efficiency in two real application scenarios:(i) energy optimisation in built environments and (ii) cancer image processing both requiring significant computational resources to execute simulations. We describe and evaluate the establishment of such an application based federation and identify a cost-decision based mechanism to determine when tasks should be outsourced to external sites in the federation. The following contributions are provided: (i) understanding the criteria for accessing sites within a federated cloud dynamically, taking into account factors such as performance, cost, user perceived value, and specific application requirements; (ii) developing and deploying a cost based federated cloud framework for supporting real applications over three federated sites at Cardiff (UK), Rutgers and Indiana (USA), (iii) a performance analysis of the application scenarios to determine how task submission could be supported across these three sites, subject to particular revenue targets. Index Terms—Federated Clouds, Cloud Computing, Cost Models, Market Mechanism, CometCloud F 1 INTRODUCTION are already being used for other applications and therefore likely to be unavailable over a particular ESEARCH in cloud computing has led to a variety simulation period; and/or (iii) are too expensive to of mechanisms for the acquisition and use of re- R acquire or use due to high operational costs. Cloud sources, enabling ‘elastic’ and on-demand acquisition computing provides a useful alternative to enable and use of such resources. The availability of cloud users to conduct more complex simulations, which systems also provides application developers with would be otherwise impossible due to limited avail- the potential to change the way these applications ability of local resources. Most significantly, the elastic interact with computational infrastructure (which, tra- nature of cloud computing enables resources to be ditionally, has been static and must be known a acquired dynamically (perhaps after carrying out an priori). Applications such as simulations are carried initial set of simulations), preventing the need to out using specialist software (such as EnergyPlus [3] guess the number of required resources beforehand. or Octave [4]) which require significant computational Using this approach, a user may be able to carry resources and data management capability, and can out a what-if investigation (on a smaller data set or generally be a time consuming process. The users with a restricted parameter range) on local resources, of these applications are also often interested in car- before making use of cloud based resources where rying out what-if scenarios by altering simulation the exact number of resources may grow with data parameters to determine various patterns within the volumes. This work focuses on understanding, from solution space. Being able to utilize computational the application perspective, what factors need to be resources at external sites provides one option for considered when integrating resources across multiple reducing execution times of such applications, espe- sites. In particular, our research question develops cially if local resources (i) do not support suitable around the decision process involved when consider- computational, data storage or hosting capability; (ii) ing the utilization of remote resources over local ones (especially from a cost perspective) and how remote I. Petri and O. F. Rana are with School of Computer resources that are part of a federated cloud could be Science & Informatics, Cardiff University, UK. E-mail: dynamically integrated and used alongside local ones. [email protected],[email protected] Tom Beach is with School of Engineering, Cardiff University, UK. E-mail: Federation of cloud systems (and the use of “cloud [email protected] bursting” techniques) enables connection of local in- J.Diaz-Montes, M. Zou and M. Parashar are with Cloud and frastructure to a global marketplace where partici- Autonomic Computing Center, Rutgers University, NJ USA. E-mail:[email protected],[email protected] pants can transact (buy and sell) capacity on demand. [email protected] This ability to scale out, on demand, provides one of 2168-7161 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCC.2015.2415792, IEEE Transactions on Cloud Computing 2 the unique benefits of cloud computing – although a composition of cloud providers as an integrated being able to undertake such scale out across multi- (or federated) cloud environment in a layered service ple providers/vendors remains a challenge. Accessing model. Assuncao et al. [20] described an approach for global services instead of increasing costs associated extending a local cluster to a cloud resource using with building new infrastructure (which may not be different scheduling strategies. Along the same lines, fully utilised and may only be needed to support Ostermann et al. [21] extended a grid workflow ap- peaks in workload over short time frames) can pro- plication development and computing infrastructure vide significant benefits. More importantly, organi- to include cloud resources and carried out experi- sations with spare capacity in their data centre can ments on the Austrian Grid and an academic cloud monetize that capacity by selling it to other providers installation of Eucalyptus using a scientific workflow through a marketplace, creating an additional source application. Similarly, Vazquez et al. [22] proposed an of revenue [17], [18], [32]. architecture for an elastic grid infrastructure using the In this paper, we present an application-centric GridWay meta-scheduler and extended grid resources federation between Cardiff (UK), Rutgers (US) and to support Nimbus. Bittencourt et al. [23] proposed Indiana(US) Universities. Using this environment, we an infrastructure to manage the execution of service make two main contributions: (i) how resources across workflows in a hybrid system composed of both multiple institutions can be federated to create a mar- grid and cloud computing resources. Analogously, ketplace; (ii) how specialist capability located across Riteau et al. [24] proposed a computing model where multiple, distributed sites – where matching can take resources from multiple cloud providers are used to place between task requirements and such capability create large-scale distributed virtual clusters. They – can be effectively utilised in a particular application used resources from two experimental testbeds, Fu- context. We use two specific applications as part of tureGrid in the United States and Grid’5000 in France our framework: (i) EnergyPlus – which is used to to achieve this. Goiri et al. [25] explore federation calculate energy flow in a built environment and from the perspective of a profit-driven policy for (ii) Octave used for cancer image processing, and outsourcing (a required capability) by attempting to show how a common framework integrating these minimise the use of the external resource (and thereby applications can emerge governed by specific cost the price of resources paid to the external provider). models. We also present a decision function that de- Toosi et al. [18] focuses on specifying reliable poli- termines where tasks should be computed based on a cies to enable providers to decide which incoming cost analysis. Our federation model uses CometCloud request to select and prioritise, thereby demonstrating [28], [16] – an autonomic computing engine based on that policies can provide a significant impact on the the Comet [29] decentralized coordination substrate, providers’ performance [26], [27]. Other significant that supports heterogeneous and dynamic federated work in Cloud federation is the use of