Comparison on Openstack and Opennebula

Comparison on OpenStack and OpenNebula performance to improve multi-Cloud architecture on cosmological simulation use case Eddy Caron, Lamiel Toch, Jonathan Rouzaud-Cornabas To cite this version: Eddy Caron, Lamiel Toch, Jonathan Rouzaud-Cornabas. Comparison on OpenStack and OpenNebula performance to improve multi-Cloud architecture on cosmological simulation use case. [Research Report] RR-8421, INRIA. 2013, pp.23. hal-00916908 HAL Id: hal-00916908 https://hal.inria.fr/hal-00916908 Submitted on 10 Dec 2013 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Comparaison de performance entre OpenStack et OpenNebula et les architectures multi-Cloud: Application à la cosmologie. E. Caron, L. Toch, J. Rouzaud-Cornabas RESEARCH REPORT N° 8421 December 2013 Project-Team Avalon ISSN 0249-6399 ISRN INRIA/RR--8421--FR+ENG Comparaison de performance entre OpenStack et OpenNebula et les architectures multi-Cloud: Application la cosmologie. E. Caron∗†, L. Toch‡, J. Rouzaud-Cornabas‡ Project-Team Avalon Research Report n° 8421 — December 2013 — 20 pages Abstract: With the increasing numbers of Cloud Service Providers and the migration of the Grids to the Cloud paradigm, it is necessary to be able to leverage these new resources. Moreover, a large class of High Performance Computing (hpc) applications can run these resources without (or with minor) modifications. But using these resources come with the cost of being able to interact with these new resource providers. In this paper we introduce the design of a hpc middleware that is able to use resources coming from an environment that compose of multiple Clouds as well as classical hpc resources. Using the Diet middleware, we are able to deploy a large-scale, distributed hpc platform that spans across a large pool of resources aggregated from different providers. Furthermore, we hide to the end users the difficulty and complexity of selecting and using these new resources even when new Cloud Service Providers are added to the pool. Finally, we validate the architecture concept through cosmological simulation ramses. Thus we give a comparison of 2 well-known Cloud Computing Software: OpenStack and OpenNebula. Key-words: Cloud, IaaS, OpenNebula, Multi-Clouds, DIET, OpenStack, RAMSES, cosmology ∗ ENS de Lyon, France, Email: [email protected] † ENSI de Bourges, France, Email: [email protected] ‡ INRIA, France, Email: [email protected] RESEARCH CENTRE GRENOBLE – RHÔNE-ALPES Inovallée 655 avenue de l’Europe Montbonnot 38334 Saint Ismier Cedex Comparison on OpenStack and OpenNebula performance to improve multi-Cloud architecture on cosmological simulation use case Résumé: Avec l’augmentation du nombre de fournisseurs de service Cloud et la migration des applications depuis les grilles de calcul vers le Cloud, il est ncessaire de pouvoir tirer parti de ces nouvelles ressources. De plus, une large classe des applications de calcul haute performance peuvent s’excuter sur ces ressources sans modifications (ou avec des modifications mineures). Mais utiliser ces ressources vient avec le cot d’tre capable d’intragir avec des nouveaux fournisseurs de ressources. Dans ce papier, nous introduisons la conception d’un nouveau intergiciel hpc qui permet d’utiliser les ressources qui proviennent d’un environement compos de plusieurs Clouds comme des ressources classiques. En utilisant l’intergiciel Diet, nous sommes capable de dployer une plateforme hpc distribue et large chelle qui s’tend sur un large ensemble de ressources aggrges entre plusieurs fournisseurs Cloud. De plus, nous cachons l’utilisateur final la difficult et la complexit de slectionner et d’utiliser ces nouvelles ressources quand un nouveau fournisseur de service Cloud est ajout dans l’ensemble. Finalement, nous validons notre concept d’architecture via une application de simulation cosmologique ramses. Et nous fournissons une comparaison entre 2 intergiciels de Cloud: OpenStack et OpenNebula. Mots-clés : Cloud, IaaS, OpenNebula, Multi-Clouds, DIET, OpenStack, RAMSES, cosmologie Comparaison de performance entre OpenStack et OpenNebula et les architectures multi-Cloud: Application la cosmologie.3 1 Introduction In recent years, distributed storage and computing have proved to be mandatory in IT. Internet Computing and Storage have considerably evolved, going from small isolated Physical Machines to large-scale cluster-like architectures driven by efficiency and scalability, one instance of these architectures is the “Clouds” [1, 2]. They aim being to be dynamically scalable and offer vir- tualized resources as a service over the Internet. Usually, most solutions deployed in Clouds focus on web applications and the load balanced is implicit or integrated. Nevertheless, Clouds can also be used in more computational-intensive domains as scalable computational resources. From a middleware point of view, Cloud infrastructures introduce new sets of resources with different features. Thus middleware environments should be extended to manage these platforms. Clouds is not yet an efficient solution for hpc but we can not neglected in future. In this paper, we propose a middleware to deal with a multiple Clouds (aka., Sky middleware), i.e., a Cloud middleware for several Cloud platforms working together. We show how we can upgrade its design is based on an existing Grid middleware for managing virtual resources. Fur- thermore, we show that our proposal can be used to execute a complex scientific application for cosmological simulation, ramses [3], on different Cloud middleware without modifying it. We show that Cloud middleware must be taken into account when deploying a new private Cloud on user owned hardware. Indeed, even with similar virtualization facility, they do not expose the same performance. Moreover, installed Cloud middleware must also be taken into account when provisioning resources for the same reason as resource provisioning can be slower or faster depending of the Cloud middleware. Moreover a study of the scalability for this application on Cloud resources is provided. Section 2 presents existing works on comparing Clouds and the application running on them. Then we present the existing Cloud APIs. We conclude the section by presenting multi-Clouds and brokering software. We use this state of the art to motivate our choice of software architecture. In Section 3, we present our extension of the Diet toolbox to support Cloud resources. First, we come back on the hierarchical architecture of Diet. Then we show how we have in- tegrate a mechanism to support multi-Clouds environments. Furthermore, we explain how an application can leverage these mechanisms to transparently use resources coming from different Clouds. Moreover, the classical Diet scheduler is still used to efficiently schedule client requests to the best fitting instantiation of the requested application. In Section 5, we describe our ex- perimental environment. We then highlight the performance variation between different Cloud middleware. We show the impact of this variability on our scientific application. Conclusion and future works are given in Section 6. 2 Related Work In the first part of this section, we present the existing works on comparing Cloud software and their underlying components and especially the virtualization one. We show that all Cloud middleware and commercial Cloud Service Providers have their advantages and drawbacks. We motivate our new study focused on the comparison of OpenStack and OpenNebula to run cosmological simulation through ramses. The large number of Cloud platforms has resulted in a large number of different APIs. It is thus very difficult for the user and for the middleware integrator to choose which one to support. Indeed, we do not want to add new codes every time we add the support of a new Cloud middleware or a new public Cloud Service Provider. Accordingly, in the Section 2.2, we show the different approaches to leverage the difficulty of managing different APIs to communicate with a set of Clouds. In the last part of this section, we present the existing works on creating multi-Clouds. RR n° 8421 4 E. Caron, L. Toch and J. Rouzaud-Cornabas Furthermore, we present a quick survey of Cloud brokering methods. Finally, we motivate our choice of integrating a Cloud brokering features in Diet. 2.1 Comparing Clouds The common base components between all Clouds (Infrastructure as a Service layer) solutions is a virtualization layer or hypervisor. Numerous studies [4, 5] have compared the different hypervisors. The comparison of different hypervisors and Cloud middleware solutions is out of the scope of this paper. To avoid the noise due to different hypervisors when comparing Cloud middleware, we always use the same hypervisor, KVM. At the Cloud middleware layer, several papers [6, 7, 8] have presented studies that compare the features of the different Cloud middleware. They only compares the functionalities and user communities. Other studies [9, 10] focus on the performance evaluation of scientific applications on a given Cloud. Another group of papers presents benchmarks and benchmark tool suites for Clouds. They are used to compare different Cloud middleware and Cloud Service Providers. CloudCmp [11] proposes to compare different Cloud Service Providers. To do so, it measures the elastic computing, persistent storage, and networking services. C-Meter [12] allows to measure the time to acquire and release virtual computing resources. It also allows to compare different config- urations of Virtual Machines. But, even on a single Cloud on a given template, it is requisite to closely study the performance given for the selected application. Indeed, variability is an important factor to study and can help to find the reason of it.

Load more