INCORPORATING STORAGE WITH AN OPEN SOURCE CLOUD PLATFORM Ganpat Agarwal Software Engineer, EMC [email protected] Table of Contents Introduction ................................................................................................................................ 3 What is cloud computing? .......................................................................................................... 4 Cloud Computing Service Models .............................................................................................. 4 Storage as a Service in Cloud Computing .................................................................................. 5 Advantages of Storage as a Service .......................................................................................... 7 Disdvantages of Storage as a Service ....................................................................................... 7 Limitations with vendor based cloud service .............................................................................. 8 OpenStack – An Open Source Cloud Computing platform ......................................................... 9 Overview of OpenStack ............................................................................................................10 Cinder – OpenStack Block Storage Service ..............................................................................11 Enterprise Storage Solutions with OpenStack Cinder ................................................................13 EMC and OpenStack? ..............................................................................................................14 Writing an OpenStack Cinder Driver with EMC storage .............................................................15 Summary ..................................................................................................................................20 References ...............................................................................................................................21 Disclaimer: The views, processes, or methodologies published in this article are those of the author. They do not necessarily reflect EMC Corporation’s views, processes, or methodologies. 2014 EMC Proven Professional Knowledge Sharing 2 Introduction As cloud computing becomes well-established in the corporate world, organizations want to explore and leverage the functionalities that cloud computing provide. However, vendor-based cloud solutions often face compatibility issues. Thus, organizations are looking for open source- based platforms that enable cloud solutions where endless features could be implemented without compatibility or other issues. OpenStack, an open source cloud computing platform with no compatibility issues, enables organizations to integrate all the necessary and desired features. As there is a big amount of data need to be maintained with every implementation of cloud computing, storage providers have an important and exciting role to play. OpenStack provides a platform to integrate object and block storage features. As a leader in software defined storage, EMC has the potential to participate in the OpenStack open source program. In addition to providing its own storage solutions, EMC can leverage the many outstanding features that OpenStack provides. This Knowledge Sharing article will walk through the advantages that OpenStack provides over vendor-based cloud solutions, the storage as a service aspect in cloud computing platforms, and integration of an EMC block storage device with OpenStack. 2014 EMC Proven Professional Knowledge Sharing 3 What is cloud computing? Cloud computing is based on a fundamental principal of ‘reusability of IT capabilities’. In science, cloud computing is a synonym for distributed computing over a network, enabling programs or applications to run on many connected computers at the same time.[1] The term ‘cloud’ implies that it has an unlimited capability to expand. This expansion could be anything which is a part of a computing setup, i.e. software applications, computing resources, networking resources, storage resources, etc. The difference that cloud computing brings compared to traditional concepts of “grid computing”, “distributed computing”, “utility computing”, or “autonomic computing” is to broaden horizons across organizational boundaries. Cloud computing is a practical approach to experiencing direct cost benefits and has the potential to transform a data center from a capital-intensive setup to a variable-priced environment. Cloud Computing Service Models Popular service models of cloud computing include: Infrastructure as a Service (IaaS) Platform as a Service (PaaS) Software as a Service (SaaS) Network as a Service (NaaS) In addition, a new service model is approaching fast: “Storage as a Service” 2014 EMC Proven Professional Knowledge Sharing 4 Storage as a Service in Cloud Computing Computing produces a great amount of data which can be either input for some other computation job or be an end result of a computation process. As the complexity of work increases, so does data production, creating the need for a good data storage service for the cloud computing platform. Documents Music CLOUD Files Photos Videos Presentations Cloud storage is a model of networked enterprise storage where data is stored in virtualized pools of storage which are generally hosted by third parties. Hosting companies operate large data centers, and customers that require their data to be hosted buy or lease storage capacity from these hosting companies. The data center operators virtualize the resources according to customer requirements and expose them as storage pools, which the customers can use to store data. Physically, the resource may span multiple servers and multiple locations. The safety of the data depends upon the hosting companies and on the applications that leverage the cloud storage. [2] 2014 EMC Proven Professional Knowledge Sharing 5 Cloud storage is based on highly virtualized infrastructure and has the same characteristics as cloud computing in terms of agility, scalability, elasticity, and multi-tenancy. It is available both off-premises and on-premises. While it is difficult to declare a canonical definition of cloud storage architecture, object storage is reasonably analogous. Cloud storage software such as OpenStack Cinder[2], cloud storage products like EMC Atmos®[3] and Hitachi Content Platform, and distributed storage research projects like OceanStore or VISION Cloud are examples of object storage and infer the following guidelines. Cloud storage is: Made up of many distributed resources but still acts as one; often referred to as federated storage clouds. Highly fault tolerant through redundancy and distribution of data. Highly durable through the creation of versioned copies. Typically eventually consistent with regard to data replicas.[4] Introduction to EMC Atmos [5] 2014 EMC Proven Professional Knowledge Sharing 6 Many people use the public cloud applications provided by Google, Drop Box, and other service providers to leverage cloud storage options. Public cloud storage gives users the flexibility to store and retrieve their data from different devices at the same location. Public Cloud storage providers[6] Advantages of Storage as a Service Less capital expense Elasticity Depending on project and application cost, storage availability could be changed to meet all the needs. No need to think about storage maintenance. Can leverage the features provided by the storage service provider like data encryption, recovery, backup, etc. Disdvantages of Storage as a Service Risk of unauthorized physical access to the data that may be spread across multiple locations. Risk of supplier instability. Changing the storage vendor may delay production and increase project cost. 2014 EMC Proven Professional Knowledge Sharing 7 Limitations with vendor-based cloud service A cloud computing platform may need to accommodate different resources from different service providers or vendors, followed by assembling them. Possible limitations for using vendor-based services include: Inappropriate costs – Some vendors charge extra for their in-built cloud functionalities and services. Organizations are charged extra for data transfer or other services. Organizations should select the vendor and their features after proper analysis of project needs. Compatibility Issues – Vendors compatibility issues across their products can be a cause of concern. These should be taken into consideration prior to vendor selection. Inflexibility – Take care when choosing a cloud computing vendor so as to not lock-in the business by using proprietary applications or formats; i.e. importing a document created in another application into a Google Docs spreadsheet. Vendors should ensure ease of provisioning and de-provisioning cloud computing users. Vendor Lock-In – Vendor lock-in occurs when an organization becomes overly reliant on a single vendor for too many solutions and/or services. Organizations face this problem when they need to migrate from their current service provider to another vendor. Sometimes this cost becomes so high that they are not able to migrate and also creates a big problem for the environment[7]. Regulatory and Compliance Restrictions – In some countries, government regulations do not allow a customer's personal information and other sensitive information to be physically located outside the state or country. In that case,