The Effects of Maturation of the Computing Market on Open Source Infrastructure Projects

Ivan Voras1, Marin Orlic1, and Branko Mihaljevic1 1University of Zagreb Faculty of Electrical Engineering and Computing, Unska 3, Zagreb E-mail(s): [email protected], [email protected], [email protected]

Abstract. Since its rise in around year 2008, One traditional definition of Cloud “Cloud computing” has become a staple buzzword computing is proposed by the USA's NIST, for large business users and scholarly institutions, stating that it is a “model for enabling ubiqutous, and has even succeeded in achieving mindshare convenient, on-demand network access to a and penetration into the consumer culture. The shared pool of configurable computer resources … technologies and ideas behind it have become that can be rapidly provisioned and released with applied, though with varying rates of success, and minimal management effort or service provider the initial enthusiasm has given way to practical interaction.” [2] A less technical and more considerations. This trend can be directly observed business-oriented definition was proposed by from comparing the current state of Open source the Financial times and Gartner, saying that the cloud computing projects with their past state. In “Cloud means someone else runs your computers this paper we present an overview of how Open and while you use what they deliver and source cloud computing landscape has changed in focus on delivering value” [3]. The source of this the last three years. quotation has an emphasis on the word value, which illustrates that there has passed enough Keywords. cloud computing, trends, open time for the idea to stop being a technological source, OpenStack, Nebula, , novelty and start being a target for business- OpenQRM, Nebula, mOSAIC, Abiquo, level expectations. This viewpoint brings a provisioning certain, well expected, change in the attitude of both the developers and the users of the 1. Introduction technology. The maturation of the market brings about a change in focus from pure Even though it sometimes might seem that innovation to stability and meeting business' “Cloud computing” is a buzzword which has needs. been with us for a long time, the reality is that The authors of this paper have been the idea first gained popularity around 2008, not involved with various Cloud Computing even five years from the time this article is projects almost throughout the entire time since written. In this timespan we have seen its rise its rise in popularity, observing and quantifying from a “high-tech” and uncertain technology, to the quality of Open source Cloud computing it becoming a “must-have” buzzword in all projects since 2009, with some ventures into situations, to its current plateau in which it is a comparing them to commercial offerings. We standard technology in the IT infrastructure observe that the trends of maturity and toolbox for most major companies. applicability can be described very well by the often-cited bell-curve of idea adoption [15], which can be demonstrated by e.g. comparing them to keyword search trends as shown in Figures 1 and 2. We would like to re-visit the projects we have described previously and Figure 1: trends for "Cloud computing" express our observations of how they have changed in the course of several years of development. 2. Previous work and paper organization

While there have been a number of papers published on the topics of definition of Cloud computing (such as [2], [4], [5] and [6]), describing what can it be used for (e.g. [7], [8]), Figure 2: The bell-curve of idea adoption [15] the state and applicability of Cloud computing in general (e.g. [9], [10]), there is a comparatively significantly smaller number of papers describing the maturity of the technologies and their ecosystems, or comparing the state of Cloud computing today with that of several years ago. Even though there are some claims that the maturation is still pending [11], there is also the beginning of recognition that Cloud computing is indeed becoming (or at least is treated as) a mature technology [12] [13] [14]. This is collaborated by research from Gartner in 2012 (Figure 3) which shows it on the “hype cycle” slope towards disillusionment about its expectations and in the direction of productivity. Based on the developments of the Open source IaaS projects we have tracked, we expect that the figure for 2013 will show Cloud computing significantly advancing in this direction. After the introduction and previous work sections, this paper will present a short overview of the projects we have tracked in Section 3, give a comparison to their current status in Section 4, provide summary of the Figure 3: The Hype Cycle for 2011, 2012, by state of Open source cloud computing in Section Gartner [13], [14] 5, and finally give conclusion and directions for future work in Section 6. level of compatibility with Amazon's services, which is also formalized in an agreement 3. Cloud computing IaaS projects in 2010. between Eucalyptus systems and Amazon [19]. It was probably the most popular Open source IaaS system at the time of our first review. In our earlier work [16] we have described 3 OpenNebula, Eucalyptus, Enterprise Ubuntu Enterprise cloud (UEC) was the first Cloud, OpenQRM, Abiquo, RedHat Cloud iteration of Canonical's Cloud computing Foundation Edition One, OpenStack, efforts. Ubuntu versions 9.10, 10.04 and 10.10 and mOSAIC projects. We will proceed to give were shipped with Eucalyptus and assorted very short descriptions of these projects in this management tools. While UEC was not a section. completely new Cloud , the OpenNebula1 is a toolkit for managing data amount of integration, especially in the area of centre infrastructure, and as one of the earliest easy installation, made it an interesting and distinct choice among the IaaS platforms. projects of its kind was not initially created with 4 Cloud computing in mind, calling itself a OpenQRM is a data-centre infrastructure “virtual infrastructure engine” [17] before management solution with strong support “Cloud computing” became a buzzword. It is a managing heterogeneous technologies, intended mature and popular product with a rapid release for small and medium-sized deployments. It has schedule, which has released 12 versions in the the ability to seamlessly manage both physical five years since its first release. The release rate and virtual resources, with P2V, V2P and V2V points to OpenNebula being a healthy, conversion capabilities for virtual machines. It is continually developed product. designed to be extensible and flexible, Eucalyptus2 began as an Open source clone supporting a wide range of products, of the (AWS) cloud but at the same time retains a comparatively simple architecture. infrastructure, implementing the same APIs and 5 functionalities as Amazon [18]. Its tools and Abiquo is an integrated Cloud computing infrastructure can be used to manage private solution for small and medium-sized clouds as well as those deployed in the deployments with an emphasis on business-level Amazon's services and a hybrid mixture of the concerns such as company-wide policies, two approaches. Eucalyptus maintains a high 3 Ubuntu Enterprise Cloud, http://www.ubuntu.com/cloud/private 1 OpenNebula, http://www.opennebula.org 4 OpenQRM, http:// www.openqrm.com 2 Eucalyptus, http://www.eucalyptus.com 5 Abiquo, http://www.abiquo.com, http://www.abicloud.org compliance, pricing, billing and reporting. It 4. Cloud computing IaaS projects in 2013. supports multiple , and has a mature web-based GUI. Abiquo’s scalable management In the time since our initial evaluation, we have and automation capabilities allow central observed not so much technological changes, as management of most data-centre resources. changes in the maturity and expectations from Red Hat Cloud Foundations Edition One Cloud computing. We will describe how the (RHCF)6 was Red Hat's initial offering of a projects from the previous section changed Cloud computing product, a portfolio of since our last evaluation. products, of which the most specific Cloud-like OpenNebula in 2010 was perhaps the components were based in a significant part on strongest candidate for Cloud computing their Grid computing tools. It offered a deployments, with a well-defined and elaborate collection of Red Hat's products which, when feature set. In the years 2010-2013, the project combined together, may be used to implement a has released 7 major versions with a large set of Cloud-like infrastructure. RHCF included not improvements. The amount of progress can be only typical Cloud computing components, but illustrated by the fact that in 2010 significant also the whole , middleware new features in version 2.0 were the Image products, messaging, clustering, and Repository which centralizes VM image management tools which are strictly speaking repository, MySQL support for backend data not Cloud computing-specific. collection and per-user resource limits (quotas), OpenStack7 was the second youngest project while in 2012 the version 3.8 brought in the in our evaluation (after mOSAIC), and at the support for Virtual Routers, direct management time it offered a somewhat complicated and of VMWare's VMFS volumes (through loosely tied bundle of functionalities: Compute VMWare's APIs), improvements in providing (the VM execution component), Imaging Service and consuming EC2 and OCCI (Open Cloud (for VM images) and the Object Storage (simple Computing Interface) APIs, as well as user key-value data storage). It was a promising and interface improvements (notably, the support ambitious project, but even though it had a large for Explorer [20]) – which indicate a number of strong backers (Rackspace and NASA shift from completing basic features to polishing being its founders) we were not sure if it was a product. Though the project remains strong going to meet the great expectations it was [21], the Google Trends indicator shows it surrounding itself with. steadily losing popularity when compared to Nimbus8 is a set of open source software OpenStack [22]. cloud computing components written in Java Eucalyptus started as an Amazon's EC2 clone and Python targeting the needs of the scientific and the interoperability with EC2 is still its community, but also trying to find its way into greatest strength. As one of the Cloud other business use-cases. The components may computing pioneers, it is a mature product be used with its own internal IaaS node, but whose latest version 3.2 was released in may also be used to manage other IaaS systems December of 2012. However, the pace of via the Amazon AWS/EC2 APIs. publishing new versions is slower than that of mOSAIC9 was a project mostly powered by OpenNebula, releasing 4 major versions in the European Union initiatives for technical period 2010-2013, of which the Open source universities. It was a joint effort of a consortium status of version 3.0 is unclear as it was labeled of European universities from eight countries, as “Enterprise edition” [23] and its code is not which are supposed to collaborate on its available in the project's GitHub repository [24]. execution. Some of the major features introduced in These are the projects we have found to be version 3.2 are a web-based graphical user viable, or to have a viable future, in the Cloud interface (which was missing in previous computing arena in the years 2009 and 2010 [16]. versions), improved logging capabilities and The current situation is a bit different, and we usage reporting, and support for EMC's VNX will describe it in the next section. SAN products [25]. In our previous investigation [26], Eucalyptus was ranked relatively close to OpenNebula, but the difference in features has grown since then. Ubuntu Enterprise Cloud received a complete change of direction, as far as technology is 6 Red Hat, http://www.redhat.com concerned, and has switched from Eucalyptus to 7 OpenStack, http:// www.openstack.org OpenStack as its IaaS backbone. This shift may 8 Nimbus, http://www.nimbusproject.org 9 mOSAIC, http:// www.mosaic-cloud.eu have been caused by Eucalyptus' partnering with Red Hat [27], and is possibly related to versions in 2010-2012), has collected a huge Eucalyptus dropping Ubuntu as one of its backing from IT infrastructure companies, and supported operating systems [25]. Since then, the Google Trends indicator shows it surpassing Canonical (the company behind Ubuntu) has every other Cloud computing product. Its become a founding member of the OpenStack feature set is growing and is reflecting the Foundation [28], with a focus on adding mass- competences of the project's big partners. deployment (so-called “MaaS” - Metal as a Nimbus remains an academic project Service, emphasizing that the hardware is developed by the University of Chicago, but interchangable) and orchestration features with a regular release schedule. In the period (starting the development of Juju) [29]. 2009-2013 there have been eight official releases. OpenQRM remained a comparatively much However, the project is lacking in “enterprise- smaller project then the ones described so far. In class” features, and as as an example, the latest the years 2010-2013 it has released 5 new version 2.0 implements (among many other versions, but there have been no releases in features) the support for the qcow2 image 2012. There has so far been one release in 2013, format (for the KVM hypervisor), improves version 5.0, a major refactoring of the system basic network configuration, and solves a bug which introduced a number of improvements, with files larger than 2 GB [36]. including a completely replaced user interface, a The mOSAIC project has released no usable new REST API, and the support for PostgreSQL downloads and the development pace seems to as the database backend. The project is a be progressing slowly [37]. We consider it reasonably good choice for smaller deployments unlikely that the project will produce any usable but lacks enterprise-class features [30]. result if continued at this pace, and even if it Abiquo was initially an Open source project did, that it will significantly lack in but has since refocused on commercial support functionalities compared to other, already and its source is no longer available. As a established projects. consequence, a smaller amount of information on it is available publicly, but available release 5. The state of Open source IaaS Cloud notes show an increase in available enterprise- computing projects class features such as virtual switches, SAN, and VMWare integration [31]. Of the nine Open source IaaS projects we have Red Hat's cloud strategy has become more tracked during the last few years, one can be consolidated and focused. Instead of “Cloud safely pronounced stalled or even practically Foundations”, now Red Hat offers concrete without a future (mOSAIC), one has stopped products, and a clear path for its customers being an Open source project (Abiquo), one which starts with , continues to a underwent an extensive reorganization which Hybrid IaaS solution, and ends with a PaaS took time and effort which were not spent in solution called OpenShift. Red Hat is leveraging upgrading the project's features (OpenQRM) one its experience in producing a popular enterprise continued to develop its own portfolio which is, operating system to integrate Cloud-like while technically Open source, is not really funcionalities and offer them piecemeal to their applicable outside its company's bundle of customers, in the form of a collection of products (Red Hat Cloud). Of the remaining, one products. switched direction into committing to OpenStack has grown significantly since our completely embracing another project (Ubuntu last evaluation. The project was started with the Cloud), and one is focused on academic Cloud controller and Compute node (called applications (Nimbus), leaving three top-tier “Nova”, contributed by NASA [33]) and the Open source IaaS projects with general Object storage node (called “Swift”, contributed usability: OpenStack, OpenNebula and by Rackspace [34]), but has since grown to Eucalyptus. include the Image Service (“Glance” for Of these, OpenStack has achieved the most managing VM images), Identity Service significant progress, and with it the most (“Keystone”, for managing users and access exposure and the largest mindshare, which we rights), Dashboard (“Horizon” offering a Web illustrate with Google Trends results presented GUI overview of the cloud), Networking in Figure 4. This search trend information was (“Quantum”), Block storage (“Cinder”), with the restricted to the time period January 2009- metering (“Ceilometer”) and Cloud January 2013, and to the “Software” category. orchestration (“Heat”) components expected in Large vendors such as IBM, HP, Cisco, Dell, 2013 [35]. The project has thus grown VMWare and Intel are all either “platinum”- significantly (releasing all four of its official level or “gold”-level members of the OpenStack 100 Foundation [38]. Their attraction to OpenStack 80 is in large part due to its modular architecture 60 and reliance on subsystems interacting over a 40 well-defined protocol, which means the 20 0 companies can easily replace certain 2009-06 2010-06 2011-06 2012-06 components with proprietary, “value-added” 2009-01 2010-01 2011-01 2012-01 2013-01 parts. The prevalence of OpenStack among Cloud Computing IaaS project is so eucalyptus overwhelming that we observe it could easily lead to a monoculture in the IaaS market. Figure 4: Google Trends search popularity OpenNebula and Eucalyptus have enough index for OpenStack, OpenNebula and installed user base and recognizability among Eucalyptus the technologically savvy that they will [5] Buyya, R., Yeo, . S., & Venugopal, S. (2008, continue to exist for some time, but the force of September). Market-oriented cloud the OpenStack branding will probably result in computing: Vision, hype, and reality for their slow decline into obscurity. As the set of delivering it services as computing utilities. features any new product in this category needs In High Performance Computing and to implement to be on-par with these products Communications, HPCC'08. 10th IEEE is large and complex, we consider that the International Conference on IEEE 2008. appearance of a new Open source IaaS product [6] Armbrust, M., Fox, A., Griffith, R., Joseph, in the near future is very unlikely. A. D., Katz, R., Konwinski, A. & Zaharia, M., A view of cloud computing. 6. Conclusion and directions for future Communications of the ACM, 2010, 50-58. work [7] Böhm, M., Leimeister, S., Riedl, C., & Krcmar, H., Cloud Computing–Outsourcing As the Cloud computing market matures, it is 2.0 or a new Business Model for IT also losing its width and becomes more Provisioning?. Application Management, concentrated into big projects. The introduction 31-56., 2011 of OpenStack was a welcome development as it [8] Scott, M., & Watson, R. The Value of Green gathered a large number of IT vendors to agree IT: a Theoretical Framework and on a direction, and it is foreseeable that the Exploratory Assessment of Cloud approaches and APIs created for OpenStack Computing, Proceedings of the 25th eBled could become a de-facto standard for future Conference, 2012. Cloud computing systems. [9] Thakur, G. S., Gupta, R., & Mukharjee, S. A While the projects described in this paper Survey on Cloud Computing and its address the IaaS aspect of Cloud computing, it Services. International Journal of Science, becomes clear that the next direction both for Engineering and Technology Research, 2012 research and practical implementation is in 1(1), pp-17. orchestration and provisioning, which we plan [10] Fox, A. Cloud Computing―What's in It for to make our next area of research. Me as a Scientist. Science, 331(6016), 2011, 406-407. References [11] Lawler, J., Howell-Barber, H., Yalamanchi, R., & Joseph, A. Determinants of an [1] Yadav N., Swami S., & Pal P. High Effective Cloud Computing Strategy, technology marketing: conceptualization Proceedings of the 28th Information and case study. Vikalpa, 2006, 31(2), 57. Systems Educators Conference, 2011 [2] Mell P., Grance T., The NIST Definition of [12] Buyya, R., Pandey, S., & Vecchiola, C. Cloud Computing, Recommendations of the Market-Oriented Cloud Computing and the National institude of Standards and Cloudbus Toolkit. arXiv preprint Technology, January 2011. arXiv:1203.5196., 2012 [3] Plummer, D., The business landscape of [13] 2011 Hype Cycle Special Report, Gartner, Cloud computing, Financial times, 2012. July 2011, Available online at [4] Plummer, D. C., Bittman, T. J., Austin, T., http://www.gartner.com/newsroom/id/1763 Cearley, D. W., & Smith, D. M.. Cloud 814, Accessed 2013-01-20 computing: Defining and describing an [14] 2012 Hype Cycle Special Report, Gartner, emerging phenomenon. Gartner, 2008. July 2012, Available online at http://www.gartner.com/newsroom/id/2124 315, Accessed 2013-01-20 [15] Rogers, E. M., & Shoemaker, F. F. 33rd International Conference on Communication of Innovations; A Cross- Information Technology Interfaces, 2011. Cultural Approach, 1971 [27] Eucalyptus Systems, Inc., Eucalyptus [16] Voras, I., Mihaljevic, B., Orlic, M., Pletikosa, Systems partners with Red Hat to deliver M., Zagar, M., Pavic, T., ... & Tomic, S., Open Cloud Solutions, Available online at Evaluating open-source cloud computing http://www.eucalyptus.com/news/12-15- solutions. In MIPRO, Proceedings of the 2010, Retrieved 2013-02-10 34th International Convention (pp. 209-214). [28] Canonical Inc., Why we're number one for IEEE, May 2011. OpenStack, Available online at [17] DSA Research, ONE 1.0 Overview, July http://blog.canonical.com/2012/09/19/why- 2008, Available online at were-number-one-for-openstack/, Retrieved http://opennebula.org/software: rnotes:rn- 2013-02-10 rel1.0, Retrieved 2013-01-25 [29] Canonical Inc., Ubuntu Cloud, Available [18] Eucalyptus Systems, AWS and Eucalyptus, online at http://www.ubuntu.com/cloud, Available online at Retrieved 2013-02-10 http://www.eucalyptus.com/aws- [30] OpenQRM Enterprise GmbH, Cncepts, compatibility, Retrieved 2013-01-25 Available online at http://www.openqrm- [19] Eucalyptus Systems, Amazon Web Services enterprise.com/about- (AWS) and Eucalyptus partner to bring openqrm/concepts.html, Retr. 2013-02-11 additional compatibility between AWS and [31] Abiquo Holdings SL, Release notes for on-premises IT environments, Available Aboquo 2.2, Available online at online at http://wiki.abiquo.com/display/ABI22/Relea http://www.eucalyptus.com/news/amazon- se+Notes+for+Abiquo+2.2, Retr. 2013-02-11 web-services-and-eucalyptus-partner, [32] Red Hat Inc., How to build and enterprise Retrieved 2013-01-25 open hybrid cloud, Available online at [20] C12G labs, OpenNebula 3.8 Release Notes, http://www.redhat.com/solutions/cloud- Available online at computing/start-building-clouds/, Retrieved http://opennebula.org/software:rnotes:rn- 2013-02-11 rel3.8, Retrieved 2013-02-05 [33] Llewellyn A., Nebula, NASA and [21] C12G labs, OpenNebula celebrates five OpenStack, published by NASA, Available years, Available online at online at http://blog.opennebula.org/?p=3743, http://open.nasa.gov/blog/2012/06/04/nebula Retrieved 2013-02-05 -nasa-and-openstack/, Retrieved 2013-02-13 [22] Google Trends: OpenNebula vs OpenStack, [34] Rackspace Inc., OpenStack: the open Available online at: alternative to cloud lock-in, Available http://www.google.com/trends/explore#q=o online at: pennebula%2Copenstack, 2013-02-05 http://www.rackspace.com/cloud/openstack [23] Eucalyptus systems, Inc., Eucalyptus 3.0 /, Retrieved 2013-02-13 documentation, Available online at [35] OpenStack project, OpenStack Roadmap, http://www.eucalyptus.com/eucalyptus- Available online at cloud/documentation/3.0, Retr. 2013-02-10 http://www.openstack.org/software/roadma [24] Eucalyptus systems, Inc., Eucalyptus p/, Retrieved 2013-02-13 GitHub repository, Available online at [36] Nimbus project, Nimbus Infrastructure https://github.com/eucalyptus/eucalyptus, Changelog, Available online at Retrieved 2013-02-10 http://www.nimbusproject.org/ [25] Eucalyptus systems, Inc., Eucalyptus 3.2 docs/2.10/changelog.html, Retr. 2013-02-13 documentation, Available online at [37] mOSAIC BitBucket code repository, http://www.eucalyptus.com/docs/release- Available online at notes/release-notes-3.2.0.pdf, Retrieved https://bitbucket.org/mosaic/mosaic-java- 2013-02-10 platform/commits, Retrieved 2013-02-14 [26] Voras I., Orlić M., Mihaljević B., Criteria for [38] OpenStack Foundation, Companies, Evaluation of Open Source Cloud Available online at Computing Solutions, Proceedings of the http://www.openstack.org/foundation/com panies, Retrieved 2013-02-14