DOCSLIB.ORG

Leveraging Cloud Computing for Optimized Storage Management

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Leveraging Cloud Computing for Optimized Storage Management Leveraging Cloud Computing For Optimized Storage Management IncludingEMC Proven EMC Professional Proven™ ProfessionalKnowledge Sharing Certification 2009 Information Storage & Management (EMCPA) Mohammed Hashim Rejaneesh Sasidharan Escalations & Training Manager Technical Lead-SME Global Technical Support Global Technical Support PSE Lab, Bangalore PSE Lab, Bangalore Wipro Technologies Wipro Technologies [email protected] [email protected] Leveraging Cloud Computing For Optimized Storage Management Mohammed Hashim Rejaneesh Sasidharan Escalations & Training Manager Technical Lead-SME Global Technical Support Global Technical Support PSE Lab, Bangalore PSE Lab, Bangalore Wipro Technologies Wipro Technologies [email protected] [email protected] Information Storage & Management (EMCPA) 2009 EMC Proven Professional Knowledge Sharing 1 Table of Contents Leveraging Cloud Computing For Optimized Storage Management.......................................... 1 Introduction.................................................................................................................................. 3 Cloud Computing and Data Storage ............................................................................................ 3 Industry Relevance and Article Overview................................................................................... 4 SOA.............................................................................................................................................. 4 SaaS.............................................................................................................................................. 6 Distributed System....................................................................................................................... 6 Grid Computing ........................................................................................................................... 7 Applying Cloud Computing to Storage ....................................................................................... 7 Cloud Computing Status in the Global Market............................................................................ 8 Cloud in Action............................................................................................................................ 8 Market Profile and Market Size of Cloud Computing................................................................. 9 Various Models of Cloud Computing........................................................................................ 11 Customer Adoption.................................................................................................................... 13 Drivers for Adoption and Industrial Outlook............................................................................. 14 Major Cloud Service Providers.................................................................................................. 16 Prominent Players ...................................................................................................................... 17 Evolution of Cloud based Storage ............................................................................................. 18 Need for Cloud based Storage ................................................................................................... 19 Comparison Chart of Major Cloud based Services................................................................ 20 Implementing a Cloud Computing Solution.............................................................................. 22 Optimizing a Cloud Storage Solution........................................................................................ 24 Managing the Cloud Solution .................................................................................................... 25 Managing Enterprise 2.0 and SLAs........................................................................................... 27 Enterprise SLAs and Cloud Computing .................................................................................... 28 Security in the Cloud ................................................................................................................. 28 Background Analysis................................................................................................................. 29 Securing the Cloud Solution ...................................................................................................... 30 Future of the Cloud .................................................................................................................... 34 Conclusion: Cloud Vision and Strategy..................................................................................... 35 Appendix A: Technical References ........................................................................................... 37 Bibliography .......................................................................................................................... 37 Websites................................................................................................................................. 37 Appendix B: Cloud Taxonomy.................................................................................................. 38 Cloud Technology Landscape.................................................................................................... 39 Appendix-C SaaS, Cloud and Web2.0..................................................................................... 41 Biography....................................................................................Error! Bookmark not defined. Disclaimer: The views, processes, or methodologies published in this compilation are those of the authors. They do not necessarily reflect EMC Corporation’s views, processes, or methodologies. 2009 EMC Proven Professional Knowledge Sharing 2 Introduction Cloud Computing spreads IT computing resources across internet cloud boundaries that are selectively accessed through service providers. Generally, users pay for computing capacity on-demand and are not concerned with the essential technologies or challenges used to achieve the increased and diverse storage scalability, server and other resource capacity and extensibility. Applications of the Cloud Computing model are expanding rapidly as connectivity costs fall and computing hardware becomes more efficiently operates at scale. The cloud’s services have expanded beyond web applications to include data storage, raw computing, and access to different specialized services. This is due to the increase in governments’ economic incentives for multiple users sharing common resources, and technological advancements that have improved collective hardware and software performance that earlier delayed distributed computing solutions. The cloud is becoming a popular solution to the problem of horizontal scalability. Cloud Computing and Data Storage Cloud-based storage has evolved from continuing attempts to decouple storage from applications so that each resource can be optimally scaled, utilized and managed. Cloud storage is a model of networked data storage where data resides on multiple virtual servers, generally hosted by third parties rather than dedicated servers. Hosting companies operate large data centers; users who require data hosting buy or lease storage capacity. The data center operators, in the background, virtualize the resources according to the customer’s requirements and expose them as virtual servers that the customers can manage. Physically, the resource may span multiple servers, data centers, or even continents. 2009 EMC Proven Professional Knowledge Sharing 3 Industry Relevance and Article Overview Cloud Optimized Storage is the new buzzword in storage networking; the world is rethinking managing their data storage. Gartner predicts that “By 2012, 80 percent of Fortune 1000 companies will pay for some cloud computing service, and 30 percent of them will pay for cloud computing infrastructure.” As Merrill Lynch analysts predict, the cloud market potential for business and productivity applications is about $96Bn by 2012 (Including SaaS of $30Bn). There are a series of major industry players waiting to adopt a Cloud Computing model to maximize the use of their services and boost revenues. This article focuses on Cloud Computing and Cloud based storage solutions and compares existing setups. It describes features of storage optimization, leveraging the current IT infrastructure, and the advantages and disadvantages of the model. The discussion analyzes SOA, SaaS, Grid and Cloud Computing; Cloud Architecture and Applying Cloud Computing to Storage; Outlining Cloud Storage Solution with optimal performance; Managing the Solution over existing Storage Infrastructure; Advantages and Risks with Cloud Computing and Comparing different Cloud Storage Solutions. This article provides insight on capabilities for service providers, data centers, and the core capabilities that end users should consider when evaluating Cloud Storage Solutions. These capabilities and benefits will shed light on how cloud based storage would benefit each of them. Engineers responsible for storage design and management will learn about the various elements of a Cloud Computing business model. The ability to increase capacity or add capability without investing in new infrastructure, training new personnel, or licensing new software are just a few of the many benefits. This
Load more

Recommended publications
  • Creating Rpms Guide
    CREATING RPMS (Student version) v1.0 Featuring 36 pages of lecture and a 48 page lab exercise This docu m e n t serves two purpose s: 1. Representative sample to allow evaluation of our courseware manuals 2. Make available high quality RPM documentation to Linux administrators A bout this m aterial : The blue background you see simulates the custom paper that all Guru Labs course w are is printed on. This student version does not contain the instructor notes and teaching tips present in the instructor version. For more information on all the features of our unique layout, see: http://ww w . g urulabs.co m /courseware/course w are_layout.php For more freely available Guru Labs content (and the latest version of this file), see: http://www.gurulabs.co m/goodies/ This sample validated on: Red Hat Enterprise Linux 4 & Fedora Core v3 SUSE Linux Enterprise Server 9 & SUSE Linux Professional 9.2 About Guru Labs: Guru Labs is a Linux training company started in 199 9 by Linux experts to produce the best Linux training and course w are available. For a complete list, visit our website at: http://www.gurulabs.co m/ This work is copyrighted Guru Labs, L.C. 2005 and is licensed under the Creative Common s Attribution- NonCom mer cial- NoDerivs License. To view a copy of this license, visit http://creativecom m o n s.org/licenses/by- nc- nd/2.0/ or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 943 0 5, USA. Guru Labs 801 N 500 W Ste 202 Bountiful, UT 84010 Ph: 801-298-5227 WWW.GURULABS.COM Objectives: • Understand
    [Show full text]
  • CERN R&D on Spack and the Turnkey Stack
    CERN R&D on Spack and the Turnkey Stack HSF Software Workshop - 2020/11/19 Valentin Volkl (CERN), Key4hep Software Group Thanks to Andre Sailer, Pere Mato, Ben Morgan for use of Slides Valentin Volkl: Key4HEP & Spack Table of Contents ● Key4HEP - Introduction and motivation ● The Spack package manager ● Technical considerations ● Deployment to CVMFS ● Workflow -Spack for Development Valentin Volkl: Key4HEP & Spack 2 Dependency Graph of Key4HEP Stack Experiment-specific Packages Valentin Volkl: Key4HEP & Spack 3 Dependency Graph of Key4HEP stack Experiment-specific Packages + HEP-specific packages Valentin Volkl: Key4HEP & Spack 4 Dependency Graph of Key4HEP stack Experiment-specific Packages + HEP-specific Packages + General Purpose Libraries Valentin Volkl: Key4HEP & Spack 5 Requirements for a Build System [ ] Need to be able to scale to a typical experiment software stack [ ] Combinatorics of multiple platforms, versions, Release/Debug ... [ ] Easy deployment to CVMFS [ ] Allow local builds independent of central CVMFS installations [ ] Support software development usecases The KEY4HEP stack contains some 300 packages ● 60 Experiment-specific ● 50 HEP-specific ● 200 System/General Purpose 14 GB install size, some 6h to build on single 4-core machine Valentin Volkl: Key4HEP & Spack 6 Key4HEP ● Future detector studies critically rely on well-maintained software stacks to model detector concepts and to understand a detector’s limitations and physics reach ● We have a scattered landscape of specific software tools on the one hand and integrated
    [Show full text]
  • Cloud Computing and Enterprise Data Reliability Luan Gashi University for Business and Technology, [email protected]
    University of Business and Technology in Kosovo UBT Knowledge Center UBT International Conference 2016 UBT International Conference Oct 28th, 9:00 AM - Oct 30th, 5:00 PM Cloud Computing and Enterprise Data Reliability Luan Gashi University for Business and Technology, [email protected] Follow this and additional works at: https://knowledgecenter.ubt-uni.net/conference Part of the Communication Commons, and the Computer Sciences Commons Recommended Citation Gashi, Luan, "Cloud Computing and Enterprise Data Reliability" (2016). UBT International Conference. 56. https://knowledgecenter.ubt-uni.net/conference/2016/all-events/56 This Event is brought to you for free and open access by the Publication and Journals at UBT Knowledge Center. It has been accepted for inclusion in UBT International Conference by an authorized administrator of UBT Knowledge Center. For more information, please contact [email protected]. Cloud Computing and Enterprise Data Reliability Cloud Computing and Enterprise Data Reliability Luan Gashi UBT – Higher Education Institution, Lagjja Kalabria, 10000 p.n., Prishtine, Kosovo [email protected] Abstract. Cloud services offer many benefits from information and communication technology that to be credible must first be secured. To use the potential of cloud computing, data is transferred, processed and stored in the infrastructures of these service providers. This indicates that the owners of data, particularly enterprises, have puzzled when storing their data is done outside the scope of their control. Research conducted on this topic show how this should be addressed unequivocally. The provided information on the organization of cloud computing models, services and standards, with a focus on security aspects in protecting enterprise data where emphasis shows how data access is treated with reliability from providers of these services.
    [Show full text]
  • Education for a Smarter Planet: the Future of Learning CIO Report on Enabling Technologies
    Front cover Education for a Smarter Planet: The Future of Learning CIO Report on Enabling Technologies Redguides for Business Leaders Dr. Jim Rudd Christopher Davia Patricia Sullivan Guidance to aid CIOs in strategic investment efforts The value of consumer IT, open platforms, and cloud computing in the future of education Practical examples of how enabling technologies are used today Introduction This IBM® Redguide™ publication is a supplement to the Future of Learning: Executive Insights Report. It provides an in-depth investigation into three enabling technologies and provides actionable guidance to aid CIO strategic and investment planning efforts. Specifically it discusses the value and role of consumer IT, open technologies, and cloud computing in the future of education. In addition, this guide provides real-world examples of the how these technologies work. This guide includes the following topics: Executive overview Exploring the technologies that enable the educational continuum Consumer IT Open platforms Cloud computing Creating Education for a Smarter Planet A roadmap for enabling a future vision Other resources for more information © Copyright IBM Corp. 2009. All rights reserved. 1 Executive overview Over the next decade, educational institutions will face significant change, transforming their relationships with students, teachers, and the workers of tomorrow. Signposts for the future are already visible, signaling significant changes to all segments of education as well as to their funders. These five signposts, which are technology immersion, personalized learning paths, knowledge skills, global integration, and economic alignment, are rapidly converging to produce a new and transformative paradigm that we call the educational continuum. This continuum dissolves the traditional boundaries between academic levels, education providers, and economic development initiatives to provide a single system for life-long learning, skills development, and workforce training.
    [Show full text]
  • Geospatial Data AS a Service: Use Cases
    GEOSPATIAL DATA AS A SERVICE USE CASES A Report of the National Geospatial Advisory Committee December 2018 Geospatial Data as a Service – Use Cases December 2018 Introduction This document was developed to complement the material in the NGAC report on Geospatial Data as a Service: A vital investment in American Enterprise and to provide real-world use cases of Data as a Service (DaaS) in action. DaaS opens new possibilities and innovation by improving access to and use of data. Users and producers across government, industry, open source, education, and the sciences are turning to DaaS to meet their huge appetite for information, exponential storage needs, data archiving and to minimize data duplication and reduce costs. To illustrate these points, we examined four case studies that provide more in-depth use cases from across Federal and local agencies and the private sector. These case studies showcase real world scenarios with large, distributed data sources to support a broad range of end user needs. The case studies presented are not intended to be an exhaustive list of examples of DaaS in use; there are many other relevant examples and programs of DaaS across the stakeholder community. To demonstrate use of DaaS for local governments and emergency response, we highlight the work of the Missouri Task Force One (MO-TF1) and Boone County Fire Protection District (BCFPD). MO-TF1 and BCFPD are using public cloud for basemap creation, data access, and delivery to enable a common operating picture of critical data during deployments. (Appendix 1 - Data as a Service & Cloud Computing: Disaster Response Just-in-Time Basemap Creation for Deployments).
    [Show full text]
  • Cloud Computing and Big Data Is There a Relation Between the Two: a Study
    International Journal of Applied Engineering Research ISSN 0973-4562 Volume 12, Number 17 (2017) pp. 6970-6982 © Research India Publications. http://www.ripublication.com Cloud Computing and Big Data is there a Relation between the Two: A Study Nabeel Zanoon1, Abdullah Al-Haj2, Sufian M Khwaldeh3 1 Department of Applied science, Al- Balqa Applied University/Aqaba, Jordan. 2 Faculty of Information Technology, University of Jordan/Aqaba, Jordan. 3Business Information Technology (BIT) Department, The University of Jordan/Aqaba, Jordan. 1Orcid ID: 0000-0003-0581-206X Abstract and combinations of each. Such data can be directly or indirectly related to geospatial information [3]. Communicating by using information technology in various ways produces big amounts of data. Such data requires Cloud computing refers to on-demand computer resources and processing and storage. The cloud is an online storage model systems available across the network that can provide a where data is stored on multiple virtual servers. Big data number of integrated computing services without local processing represents a new challenge in computing, resources to facilitate user access. These resources include especially in cloud computing. Data processing involves data data storage capacity, backup and self-synchronization [4]. acquisition, storage and analysis. In this respect, there are Most IT Infrastructure computing consist of services that are many questions including, what is the relationship between provided and delivered through public centers and servers big data and cloud computing? And how is big data processed based on them. Here, clouds appear as individual access in cloud computing? The answer to these questions will be points for the computing needs of the consumer.
    [Show full text]
  • Building Embedded Linux Systems ,Roadmap.18084 Page Ii Wednesday, August 6, 2008 9:05 AM
    Building Embedded Linux Systems ,roadmap.18084 Page ii Wednesday, August 6, 2008 9:05 AM Other Linux resources from O’Reilly Related titles Designing Embedded Programming Embedded Hardware Systems Linux Device Drivers Running Linux Linux in a Nutshell Understanding the Linux Linux Network Adminis- Kernel trator’s Guide Linux Books linux.oreilly.com is a complete catalog of O’Reilly’s books on Resource Center Linux and Unix and related technologies, including sample chapters and code examples. ONLamp.com is the premier site for the open source web plat- form: Linux, Apache, MySQL, and either Perl, Python, or PHP. Conferences O’Reilly brings diverse innovators together to nurture the ideas that spark revolutionary industries. We specialize in document- ing the latest tools and systems, translating the innovator’s knowledge into useful skills for those in the trenches. Visit con- ferences.oreilly.com for our upcoming events. Safari Bookshelf (safari.oreilly.com) is the premier online refer- ence library for programmers and IT professionals. Conduct searches across more than 1,000 books. Subscribers can zero in on answers to time-critical questions in a matter of seconds. Read the books on your Bookshelf from cover to cover or sim- ply flip to the page you need. Try it today for free. main.title Page iii Monday, May 19, 2008 11:21 AM SECOND EDITION Building Embedded Linux SystemsTomcat ™ The Definitive Guide Karim Yaghmour, JonJason Masters, Brittain Gilad and Ben-Yossef, Ian F. Darwin and Philippe Gerum Beijing • Cambridge • Farnham • Köln • Sebastopol • Taipei • Tokyo Building Embedded Linux Systems, Second Edition by Karim Yaghmour, Jon Masters, Gilad Ben-Yossef, and Philippe Gerum Copyright © 2008 Karim Yaghmour and Jon Masters.
    [Show full text]
  • Robust Linux Binaries.Pdf
    Robust Linux Binaries How to use Portage to provide a solid base stack for HEP G. Amadio https://xkcd.com/1987 Why do we fall into this kind of situation? ● We want software that’s not part of the system ● Use pip, conda, homebrew, etc to get it somewhere ● Not compiled with same compiler → incompatible ABI ● Often requires setting LD_LIBRARY_PATH and/or PYTHONPATH ● Problems if a package is both in the system and in add-ons ● Updates to the system do not take add-ons into account 3 Classic example: ROOT, Python, and Anaconda ● User has Anaconda installation with Python, wants ROOT ● User then tries to build ROOT with system compiler and link with Python from Anaconda installation ● libPyROOT.so has to link against system’s libstdc++.so and anaconda’s libpython2.7.so ● Problem: libpython2.7.so from Anaconda is not guaranteed to be ABI-compatible with system libraries ● Solution: install GCC from anaconda that was used to build Python and build ROOT with that compiler instead 4 Why LD_LIBRARY_PATH should be avoided ● LD_LIBRARY_PATH is commonly used to add directories to the linker’s search path ● Problem: LD_LIBRARY_PATH takes precedence, overrides important system libraries ○ See e.g. https://sft.its.cern.ch/jira/browse/SPI-1083 ● Solution: use a wrapper script or even better, don’t use LD_LIBRARY_PATH $ ssh lxplus7 $ lsb_release -d Description: CentOS Linux release 7.5.1804 (Core) $ source /cvmfs/sft.cern.ch/lcg/views/LCG_latest/x86_64-centos7-gcc7-opt/setup.sh $ ldd /usr/bin/git linux-vdso.so.1 => (0x00007ffdb55b3000) libpcre.so.1
    [Show full text]
  • Evolution of As-A-Service Era in Cloud
    Evolution of as-a-Service Era in Cloud Sugam Sharma Center for Survey Statistics and Methodology, Iowa State University, Ames, Iowa, USA Email: [email protected] Abstract. Today, a paradigm shift is being observed in science, where the focus is gradually shifting toward the cloud environments to obtain appropriate, robust and affordable services to deal with Big Data challenges (Sharma et al. 2014, 2015a, 2015b). Cloud computing avoids any need to locally maintain the overly scaled computing infrastructure that include not only dedicated space, but the expensive hardware and software also. In this paper, we study the evolution of as-a-Service modalities, stimulated by cloud computing, and explore the most complete inventory of new members beyond traditional cloud computing stack. Keywords. Cloud, as-a-Service 1. Introduction Today, the cloud computing also has emerged as one of the major shifts in recent information and communication age that promises an affordable and robust computational architecture for large-scale and even for overly complex enterprise applications (Fenn et al. 2008). It is a powerful and revolutionary paradigm that offers service-oriented computing and abstracts the software-equipped hardware infrastructure from the clients or users. Although, the concept of cloud computing is mainly popular in three praxises- 1) IaaS (Infrastructure-as-a-Service ), 2) PaaS (Platform-as-a-Service ), and 3) SaaS (Software-as-a-Service ), but in this data science age, should be equally expandable to DBaaS (Database- as-a-Service ) (Curino, et al., 2011; Seibold et al., 2012). Over the last few year, the cloud computing has evolved as scalable, secure and cost-effective solutions to an overwhelming number of applications from diversified areas.
    [Show full text]
  • NVIDIA CUDA Installation Guide for Linux
    NVIDIA CUDA Installation Guide for Linux Installation and Verification on Linux Systems DU-05347-001_v11.4 | September 2021 Table of Contents Chapter 1. Introduction........................................................................................................ 1 1.1. System Requirements...............................................................................................................1 1.2. About This Document............................................................................................................... 3 Chapter 2. Pre-installation Actions..................................................................................... 4 2.1. Verify You Have a CUDA-Capable GPU....................................................................................4 2.2. Verify You Have a Supported Version of Linux........................................................................ 5 2.3. Verify the System Has gcc Installed........................................................................................5 2.4. Verify the System has the Correct Kernel Headers and Development Packages Installed........................................................................................................................................5 2.5. Install MLNX_OFED.................................................................................................................. 7 2.6. Choose an Installation Method................................................................................................ 7 2.7. Download
    [Show full text]
  • Accelerate Delivery of Your Modern Data Analytics Platform with the Adatis Framework
    Accelerate delivery of your Modern Data Analytics Platform with the Adatis Framework Data analytics is more relevant in today’s business than ever but traditional data warehouse platforms are evolving. The demands placed on analytics platforms are to provide both the traditional, structured, standardised BI and to support new ways of analysing data based on agility, supporting rapid change and semi/unstructured data. To support this new form of advanced analytics, including machine learning, predictive and artificial intelligence, a new flexible, cloud-based platform is required. Adatis ’ Microsoft Azure based Data Platform Framework is used extensively on client “With Adatis’ projects to under-pin and fast track the development of enterprise data platforms that satisfy their clients data analytics requirements. The Adatis Framework has knowledge of Microsoft been developed over may years based on industry best practice and is constantly Azure Data Analytics, updated and enhanced to make the most of technology enhancements released by Microsoft. for the first time we are able to instantly The Adatis Framework uses the advanced data transformation capabilities of the Microsoft Azure stack, including Azure Data Factory, Azure ML Services, Azure monitor, predict, Databricks, and Azure Data Lake Store, to: troubleshoot Measurement and ▪ Ingest data from multiple sources and file types analyse inand ways review we ▪ Cleanse data and handle errors ▪ Track lineage and log activity would not believe ▪ Automate processes and generate code possible before.” ▪ Manage costs and resource use How it works The Adatis framework takes feeds from multiple source systems and, provides a technical framework and reference architecture for ingesting, preparing, shaping, integrating, and storing the data in a data lake Adatis Modern Data Platform | adatis.co.uk The Adatis framework brings many benefits to your team and your projects: Proven: We’ve built and refined the framework over years of working with companies on their transformation initiatives.
    [Show full text]
  • Lenovo Big Data Validated Design for Cazena Saas Using Cloudera on Thinkagile HX
    Lenovo Big Data Validated Design for Cazena SaaS using Cloudera on ThinkAgile HX Last update: 27 June 2019 Version 1.0 Configuration Reference Number: BGDCL03XX91 Reference architecture for Solution based on the Cloudera Enterprise with Cazena ThinkSystem HX platform running SaaS Data Lakes VMware vSphere virtualization Deployment considerations for Solution based on Cazena’s Fully scalable racks including detailed Managed SaaS Data Lake running validated bills of material on the ThinkAgile HX platform Dan Kangas (Lenovo) Venkat Chandra (Cazena) John Piekos (Cazena) Keith Adams (Lenovo) Ajay Dholokia (Lenovo) Gary Cudak (Lenovo) 1 Lenovo Big Data Validated Design for Cazena SaaS using Cloudera on ThinkAgile HX Table of Contents 1 Introduction .............................................................................................. 4 2 Business problem and business value .................................................. 5 Business Problem .................................................................................................... 5 Business Value ........................................................................................................ 5 2.2.1 Time to Production....................................................................................................................... 5 2.2.2 Deploy ML & Analytics Quickly with the AppCloud ..................................................................... 6 2.2.3 Plug & Play Enterprise Deployment: ..........................................................................................
    [Show full text]