DOCSLIB.ORG

Cloud Computing Storage

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Cloud Computing Storage CLOUD COMPUTING STORAGE From POSIX to key-value stores tmpmtp6I4.odt Confidentiel © Nexedi SA / ISO 16016 Table des signatures Nom Fonction Signature Date Organisme Rédigé par: PDG 2011-11-16 Jean-Paul Smets Nexedi SA Rédigé par: Ched de Projet 2011-11-16 Romain Courteaud Nexedi SA Approuvé par: PDG 2011-11-16 Jean-Paul Smets Nexedi SA Soumis par: PDG 2011-11-16 Jean-Paul Smets Nexedi SA tmpmtp6I4.odt Confidentiel © Nexedi SA / ISO 16016 Liste de diffusion Smets, J-P. Nexedi SA Fermigier, S. Nuxeo SAS Brettnacher, T. Nexedi SA Laisne,J-P. Bull SAS Courteaud, R. Nexedi SA tmpmtp6I4.odt Confidentiel © Nexedi SA / ISO 16016 Historique des modifications Version Date Page Description des modifications 001 2010-10-09 Toutes First draft 002 2010-10-10 Toutes 003 2011-04-19 Toutes First preliminary version, submitted for review 004 2011-11-16 Toutes Release Candidate tmpmtp6I4.odt Confidentiel © Nexedi SA / ISO 16016 Table of Content 1 Introduction..................................................................................................7 1.1 Goals and scope.......................................................................................................7 1.2 Plan.......................................................................................................................... 7 1.3 References................................................................................................................7 1.3.1.Applicable Documents..........................................................................................8 1.3.2.Reference Documents..........................................................................................8 1.4 Abbreviations.........................................................................................................10 2 Executive Summary....................................................................................11 3 Storage for the Cloud..................................................................................13 3.1 The Evolution of the Storage Paradigm..................................................................14 3.2 Scientific background.............................................................................................16 3.3 Cloud Storage Use Cases........................................................................................17 4 Key-Value Storage: Blocks or Blobs.............................................................19 4.1 Linux Block Device.................................................................................................20 4.2 NoSQL key-value stores..........................................................................................21 5 Distributed file systems: Shared Directories and Files................................23 5.1 From file system to key-value.................................................................................24 5.2 From key-value to file system.................................................................................25 6 NoSQL beyond Mere Storage......................................................................27 6.1 Key-valye Store Limitations....................................................................................27 6.1.1.Authentication.....................................................................................................27 6.1.2.Encryption..........................................................................................................28 6.1.3.Multiple namespaces..........................................................................................28 6.1.4.Indexing..............................................................................................................28 6.2 NoSQL Document Stores........................................................................................29 6.2.1.Querying through map-reduce code injection.....................................................29 6.2.2.Querying through indices....................................................................................29 6.3 NoSQL Columnar Stores.........................................................................................29 6.4 NoSQL Graph Databases........................................................................................29 6.4.1.Querying through graph crawling.......................................................................29 6.4.2.Querying through semantic indices....................................................................30 tmpmtp6I4.odt Confidentiel © Nexedi SA / ISO 16016 7 Storage Allocation API.................................................................................31 7.1 Existing Storage Allocation APIs.............................................................................31 7.2 SlapOS: A Universal Allocation API for Storage.......................................................32 7.2.1.Allocating Storage with slapos request in pytjon................................................32 7.2.2.Allocating Storage with slapos command line....................................................32 7.2.3.Multitenancy.......................................................................................................32 7.2.4.Automating Storage Scalability Testing..............................................................33 7.3 The SLAP REST API.................................................................................................33 7.3.1.Exchange format................................................................................................33 7.3.2.Response status code........................................................................................33 7.3.3.Instance Methods...............................................................................................34 7.3.4.Deleting an instance...........................................................................................35 7.3.5.Get instance information.....................................................................................36 7.3.6.Get instance authentication certificates..............................................................36 7.3.7.Bang instance.....................................................................................................37 7.3.8.Modifying instance..............................................................................................37 8 Recommendations......................................................................................39 8.1 The End of POSIX....................................................................................................40 tmpmtp6I4.odt Confidentiel © Nexedi SA / ISO 16016 1 Introduction 1.1 Goals and scope This document is a preliminary report for Task 2 of Work Package 1 (IaaS) of the COMPATIBLE ONE research project. It provides an overview of current prac- tices related to Cloud Computing storage recommendations for further research. 1.2 Plan This document is composed of 8 chapters: • Chapter 1 defines goals, scope, plan, referenced documents and abbrevi- ations; • Chapter 2 is an executive summary; • Chapter 3 introduces Cloud storage evolution and scientific background; • Chapter 4 provides an overview of key-value stores; • Chapter 5 provides an overview of distributed file systems; • Chapter 6 introduces desirable features for Cloud storage which go be- yond mere storage; • Chapter 7 proposes a universal API to allocate Cloud Storage; • Chapter 8 proposes recommendations for further Cloud R&D. 1.3 References This section describes applicable documents and reference documents. tmpmtp6I4.odt Confidentiel © Nexedi SA / ISO 16016 1.3.1.Applicable Documents ID Title Reference No Version/Rev. 12ème Appel à projets du FUI (Fonds Unique AD- Interministériel) N/A N/A 1 http://www.systematic-paris-region.org/fr/les- projets/appels-a-projets/appels-a-projets-du-fui 1.3.2.Reference Documents ID Title Reference No Version/Rev. QEMU-RBD RD-1 N/A N/A http://ceph.newdream.net/wiki/QEMU-RBD CEPH RD-2 http://ceph.newdream.net/ N/A N/A http://ceph.newdream.net/publications/ Sheepdog RD-3 N/A N/A http://www.osrg.net/sheepdog/ Sage A. Weil. Ceph: Reliable, Scalable, and High- RD-4 Performance Distributed Storage. Ph.D. thesis, N/A N/A University of California, Santa Cruz, December, 2007. P2P-like Tahoe file system offers secure storage in the cloud RD-5 http://arstechnica.com/open- N/A N/A source/news/2009/08/p2p-like-tahoe-file system- offers-secure-storage-in-the-cloud.ars Brian Warner, Zooko Wilcox-O'Hear, Rob Kinninmont. RD-6 Tahoe: A Secure Distributed file system N/A N/A http://tahoe-lafs.org/~warner/pycon-tahoe.html TAHOE LAFS RD-7 N/A N/A http://tahoe-lafs.org/trac/tahoe-lafs Zooko's Hack Log RD-8 http://insecure.tahoe-lafs.org/uri/URI:DIR2- N/A N/A RO:ixqhc4kdbjxc7o65xjnveoewym:5x6lwoxghrd5rxh wunzavft2qygfkt27oj3fbxlq4c6p45z5uneq/blog.html Memcachefs RD-9 N/A N/A http://memcachefs.sourceforge.net/ Moxi RD-10 N/A N/A http://code.google.com/p/moxi/ Moxi, a memcached proxy RD-11 http://www.slideshare.net/northscale/moxi- N/A N/A memcached-proxy Alexandre LISSY, Arnaud Laprévote. Espace de RD-12 N/A N/A stockage avec XtreemFS. tmpmtp6I4.odt Confidentiel © Nexedi SA / ISO 16016 ID Title Reference No Version/Rev. Memagent RD-13 N/A N/A http://code.google.com/p/memagent/ Linux Block Devices RD-14 http://www.kernel.org/doc/htmldocs/kernel- N/A N/A api/blkdev.html Membase RD-15 N/A N/A http://www.membase.org/ An introduction to block device drivers RD-16 N/A N/A http://www.linuxjournal.com/article/2890?page=0,0 Caching
Load more

Recommended publications
  • Backup4all Take Full Control of Your Backup
    Backup4all Take full control of your backup Backup4all User Manual Copyright © 2016 Softland Backup4all User Manual for Backup4all version 6 by Softland This user manual presents all the features in the Professional edition. If you own Lite, Standard or Portable, some of the features presented in this user manual might not be available in your edition. The Professional edition includes all the features. For a comparison between editions, please check the feature matrix topic. The information in this document is subject to change without notice. Although every precaution has been taken in the preparation of this book, Softland assumes no responsibility for errors and omissions. Nor is any liability assumed for damages resulting from the information contained herein. This documentation contains proprietary information of Softland. All rights reserved. Contents 3 Table of Contents Part I Introduction 9 1 Product Description................................................................................................................................... 9 2 System Requirements................................................................................................................................... 10 3 Installation ................................................................................................................................... 10 Command Line Parameters.......................................................................................................................................................... 11 4 Features
    [Show full text]
  • Brightstor Arcserve Backup for Linux Administrator Guide
    BrightStor® ARCserve® Backup for Linux Administrator Guide r11.5 D01211-2E This documentation and related computer software program (hereinafter referred to as the "Documentation") is for the end user's informational purposes only and is subject to change or withdrawal by Computer Associates International, Inc. ("CA") at any time. This documentation may not be copied, transferred, reproduced, disclosed or duplicated, in whole or in part, without the prior written consent of CA. This documentation is proprietary information of CA and protected by the copyright laws of the United States and international treaties. Notwithstanding the foregoing, licensed users may print a reasonable number of copies of this documentation for their own internal use, provided that all CA copyright notices and legends are affixed to each reproduced copy. Only authorized employees, consultants, or agents of the user who are bound by the confidentiality provisions of the license for the software are permitted to have access to such copies. This right to print copies is limited to the period during which the license for the product remains in full force and effect. Should the license terminate for any reason, it shall be the user's responsibility to return to CA the reproduced copies or to certify to CA that same have been destroyed. To the extent permitted by applicable law, CA provides this documentation "as is" without warranty of any kind, including without limitation, any implied warranties of merchantability, fitness for a particular purpose or noninfringement. In no event will CA be liable to the end user or any third party for any loss or damage, direct or indirect, from the use of this documentation, including without limitation, lost profits, business interruption, goodwill, or lost data, even if CA is expressly advised of such loss or damage.
    [Show full text]
  • Cumulus: Filesystem Backup to the Cloud
    Cumulus: Filesystem Backup to the Cloud Michael Vrable, Stefan Savage, and Geoffrey M. Voelker Department of Computer Science and Engineering University of California, San Diego Abstract uitous broadband now provides sufficient bandwidth re- In this paper we describe Cumulus, a system for effi- sources to offload the application. For small to mid-sized ciently implementing filesystem backups over the Inter- businesses, backup is rarely part of critical business pro- net. Cumulus is specifically designed under a thin cloud cesses and yet is sufficiently complex to “get right” that it assumption—that the remote datacenter storing the back- can consume significant IT resources. Finally, larger en- ups does not provide any special backup services, but terprises benefit from backing up to the cloud to provide only provides a least-common-denominator storage in- a business continuity hedge against site disasters. terface (i.e., get and put of complete files). Cumulus However, to price cloud-based backup services attrac- aggregates data from small files for remote storage, and tively requires minimizing the capital costs of data cen- uses LFS-inspired segment cleaning to maintain storage ter storage and the operational bandwidth costs of ship- efficiency. Cumulus also efficiently represents incremen- ping the data there and back. To this end, most exist- tal changes, including edits to large files. While Cumulus ing cloud-based backup services (e.g., Mozy, Carbonite, can use virtually any storage service, we show that its ef- Symantec’s Protection Network) implement integrated ficiency is comparable to integrated approaches. solutions that include backup-specific software hosted on both the client and at the data center (usually using 1 Introduction servers owned by the provider).
    [Show full text]
  • Tanium™ Appliance Deployment Guide
    Tanium™ Appliance Deployment Guide Version 1.6.2 October 16, 2020 The information in this document is subject to change without notice. Further, the information provided in this document is provided “as is” and is believed to be accurate, but is presented without any warranty of any kind, express or implied, except as provided in Tanium’s customer sales terms and conditions. Unless so otherwise provided, Tanium assumes no liability whatsoever, and in no event shall Tanium or its suppliers be liable for any indirect, special, consequential, or incidental damages, including without limitation, lost profits or loss or damage to data arising out of the use or inability to use this document, even if Tanium Inc. has been advised of the possibility of such damages. Any IP addresses used in this document are not intended to be actual addresses. Any examples, command display output, network topology diagrams, and other figures included in this document are shown for illustrative purposes only. Any use of actual IP addresses in illustrative content is unintentional and coincidental. Please visit https://docs.tanium.com for the most current Tanium product documentation. This documentation may provide access to or information about content, products (including hardware and software), and services provided by third parties (“Third Party Items”). With respect to such Third Party Items, Tanium Inc. and its affiliates (i) are not responsible for such items, and expressly disclaim all warranties and liability of any kind related to such Third Party Items and (ii) will not be responsible for any loss, costs, or damages incurred due to your access to or use of such Third Party Items unless expressly set forth otherwise in an applicable agreement between you and Tanium.
    [Show full text]
  • Backup4all User Manual
    Backup4all Take full control of your backup Backup4all User Manual Copyright © 2017 Softland Backup4all User Manual for Backup4all version 7 by Softland This user manual presents all the features in the Professional edition. If you own Lite, Standard or Portable, some of the features presented in this user manual might not be available in your edition. The Professional edition includes all the features. For a comparison between editions, please check the feature matrix topic. The information in this document is subject to change without notice. Although every precaution has been taken in the preparation of this book, Softland assumes no responsibility for errors and omissions. Nor is any liability assumed for damages resulting from the information contained herein. This documentation contains proprietary information of Softland. All rights reserved. Contents 3 Table of Contents Part I Introduction 9 1 Product Description................................................................................................................................... 9 2 System Requirements................................................................................................................................... 10 3 Installation ................................................................................................................................... 10 Command Line.......................................................................................................................................................... Parameters 12 4 Features
    [Show full text]
  • Hitachi Data Ingestor
    Hitachi Data Ingestor 6.4.7 Cluster Administrator's Guide This manual describes how to operate a Hitachi Data Ingestor (HDI) system in a cluster configuration. MK-90HDI038-30 July 2019 © 2017- 2019 Hitachi Vantara Corporation. All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including copying and recording, or stored in a database or retrieval system for commercial purposes without the express written permission of Hitachi, Ltd., or Hitachi Vantara Corporation (collectively “Hitachi”). Licensee may make copies of the Materials provided that any such copy is: (i) created as an essential step in utilization of the Software as licensed and is used in no other manner; or (ii) used for archival purposes. Licensee may not make any other copies of the Materials. “Materials” mean text, data, photographs, graphics, audio, video and documents. Hitachi reserves the right to make changes to this Material at any time without notice and assumes no responsibility for its use. The Materials contain the most current information available at the time of publication. Some of the features described in the Materials might not be currently available. Refer to the most recent product announcement for information about feature and product availability, or contact Hitachi Vantara Corporation at https://support.hitachivantara.com/en_us/contact-us.html. Notice: Hitachi products and services can be ordered only under the terms and conditions of the applicable Hitachi agreements. The use of Hitachi products is governed by the terms of your agreements with Hitachi Vantara Corporation. By using this software, you agree that you are responsible for: 1.
    [Show full text]
  • Open BAR a New Approach to Mobile Backup and Restore
    Universit`adegli Studi di Roma “Tor Vergata” Facolt`adi Ingegneria Dottorato di Ricerca in Informatica ed Ingegneria dell’Automazione Ciclo XXIII Open BAR A New Approach to Mobile Backup And Restore Vittorio Ottaviani A.A. 2010/2011 Docente Guida/Tutor: Prof. Giuseppe F. Italiano Coordinatore: Prof. Daniel P. Bovet to my parents because an example is worth a thousand words Abstract Smartphone owners use to save always more information, and more impor- tant data into the internal memory of their devices. Mobile devices are prone to be lost, stolen or broken; this causes the loss of all the information contained in it if these data are not backed up. While many solutions for making back- ups and restoring data are known for servers and desktops, mobile devices pose several challenges, mainly due to the plethora of devices, vendors, oper- ating systems and versions available in the mobile market. In this thesis, we propose a new backup and restores approach for mobile devices, which helps to reduce the effort in saving and restoring personal data and migrate from a device to another. Our approach is platform independent: in particular, we present some prototypes based on different mobile operating systems: Google Android, Windows Mobile 5 and 6 and Symbian S60. The approach grants the security of the information backed up and restored using novel cryptographic techniques optimized for mobile. Another feature of our approach lies in the capability of offering additional services to the final user or to administrator of the system. As an example, for users, we provide a service enabling the shar- ing of information in mobile devices among a group of selected persons.
    [Show full text]
  • Cloud Storage: the Risks and Rewards
    Cloud Storage: The Risks and Rewards Jérémy Legendre Norwich University School of Business and Management IS340 – Introduction to Information Assurance 2012-11-14 Copyright © 2012 Jeremy Legendre. All rights reserved. Page 1 Cloud Storage: The Risks and Rewards Contents Introduction ............................................................................................................................................................ 3 Why has Cloud Storage Become Popular? ................................................................................................... 3 Risks ........................................................................................................................................................................... 4 DropBox Compromises .................................................................................................................................. 4 iCloud Compromise ......................................................................................................................................... 4 Google Drive Compromises .......................................................................................................................... 5 Security of Data Transfers ................................................................................................................................. 6 FTP ......................................................................................................................................................................... 6 SSL
    [Show full text]
  • And Linux Tools for Backup O P S E R
    e s u y n a r o f e l b i s n o p s e r d l e h e b t o n n a c n o i s s i m m o C e h t d n a , r o h t u a e h t f o y l n o s w e i v e h t s t c e HOGSKOLAN ISKOVDE(HIS) Content createdby: see theIMSdeployedinourMoodle platform. the sources in ELP format (ExeLearning) or enjoy this cisoss.eu content. to Visit download Remember: This is not the best format to l f e r n o i t a c i l b u p . ) s / i 0 h . T 4 / . a n s o - i y s s b i / s m e s PILOT VERSION PILOT m n o e c C i l / n a g r e o p . o s r n u o E m e m h t o c m e o v r i t f a t e r r o c p / / p : s u s p t t h t h i ( w ” l d a e n d o . i n t n u i a f e n r r n e e e t h e t n b I d s e 0 a . n i h 4 a ) t e 8 n k i 5 o l 8 c A 5 n e 1 r o 0 i a - t h a 2 S 0 - m 2 r n o o A f i t n K i u - b 1 e i 0 r h t t S t f E A o - s 1 e - n d 5 o a 1 m 0 m 2 m e ( o b t c C y e a e j v o m i r t p h a c e s i i r h h C T w “ e s u y n a r o f e l b i s n o Backup systems p s e r d l Backup systems e h e b t Content created by: o n n a c HOGSKOLAN I SKOVDE (HIS) n o i s s i m 1.
    [Show full text]
  • Brightstor Arcserve Backup R11.5 for Windows Beta
    Data Sheet BrightStor® ARCserve® Backup r11.5 for Windows BrightStor® ARCserve® Backup for Windows offers world-class data protection for distributed servers, databases and applications, as well as clients for multiple environments, including Windows, UNIX, Linux, and NetWare. With its wide range of powerful, easy-to-use data protection tools, BrightStor ARCserve Backup ensures the integrity and availability of your most important asset—data Top Three Key Features As businesses grow, so do their Access to Information storage requirements, and data • Complete data protection solution Is Key to Success protection environments can become for Windows extremely complex. As a result, • Wide Platform and Application In an increasingly competitive businesses demand exceptional Support business environment, the availability flexibility from software tools. To meet • Centralized Management of the of information is often the differentiator the rising need to manage data growth Backup Environment between success and failure. The and provide around-the-clock proliferation of the Internet and the availability, many organizations are What’s New movement towards on-demand deploying the latest storage computing has only heightened the technologies, including Storage Area • Disk Staging Backup (D2D2T) need to effectively leverage Networks (SANs), Network-Attached • Microsoft SharePoint Agent information assets to increase revenue Storage (NAS) and Disk Arrays. • 64 bit platform support and profit opportunities. And at the heart of an organization’s information
    [Show full text]
  • Pii041409 Arc 125 A&O Prod Brief
    PRODUCT SUMMARY: CA ARCSERVE BACKUP R12.5 AGENTS AND OPTIONS CA ARCserve® Backup r12.5 Agents and Options CA ARCSERVE® BACKUP R12.5 OFFERS WORLD-CLASS HIGH PERFORMANCE DATA PROTECTION THAT COMBINES INNOVATIVE DATA DEDUPLICATION AND POWERFUL STORAGE RESOURCE MANAGEMENT (SRM) REPORTING WITH INDUSTRY LEADING SOFTWARE VTL BACKUP (D2D2T) AND COMPREHENSIVE SECURITY TOOLS; MAKING IT ONE OF THE MOST SECURE, RELIABLE AND EFFICIENT “OUT-OF-THE-BOX” BACKUP SOLUTIONS OFFERED FOR BOTH PHYSICAL AND VIRTUAL SERVERS. WITH ITS WIDE RANGE OF POWERFUL, EASY-TO-USE AND SINGLE MANAGEMENT DATA PROTECTION TOOLS, CA ARCSERVE BACKUP ENSURES THE INTEGRITY AND AVAILABILITY OF YOUR MOST IMPORTANT ASSET — DATA. CA ARCserve Backup Agents and Options You can easily tailor CA ARCserve Backup to fit your company's storage needs. Just choose from several powerful agents and options to build a solution that is as unique as your business. CA ARCserve Backup Agents let you extend protection to the specific applications you need to support. CA ARCserve Backup Options let you add extensive data protection features, including central management, advanced device and media management, disaster recovery and SAN management capabilities. CA’s unique pricing models let you reduce the complexity of procuring, installing, managing and tracking software product licensing keys so you don’t need to buy each technology you need individually. Choose from a la carte, Simplified Pricing (SIP) or Managed Capacity Pricing (MCP) models. For more information on the best ways to procure these technologies, see Buying the Right Agents & Options on page 5. Overview CA ARCserve Backup Agents and Options help you customize your storage environment, whether you have one server or many servers, or have an email server that is based on Microsoft Exchange or Lotus Notes, or if your environment runs Windows, NetWare or Linux.
    [Show full text]
  • Single Node Administrator's Guide
    Hitachi Data Ingestor Single Node Administrator's Guide Product Version Getting Help Contents MK-90HDI039-21 © 2013- 2017 Hitachi, Ltd. All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, or stored in a database or retrieval system for any purpose without the express written permission of Hitachi, Ltd. Hitachi, Ltd., reserves the right to make changes to this document at any time without notice and assumes no responsibility for its use. This document contains the most current information available at the time of publication. When new or revised information becomes available, this entire document will be updated and distributed to all registered users. Some of the features described in this document might not be currently available. Refer to the most recent product announcement for information about feature and product availability, or contact Hitachi Data Systems Corporation at https://support.hds.com/en_us/documents.html. Notice: Hitachi, Ltd., products and services can be ordered only under the terms and conditions of the applicable Hitachi Data Systems Corporation agreements. The use of Hitachi, Ltd., products is governed by the terms of your agreements with Hitachi Data Systems Corporation. Hitachi is a registered trademark of Hitachi, Ltd., in the United States and other countries. Hitachi Data Systems is a registered trademark and service mark of Hitachi, Ltd., in the United States and other countries. Archivas, Essential NAS Platform, HiCommand, Hi-Track, ShadowImage, Tagmaserve, Tagmasoft, Tagmasolve, Tagmastore, TrueCopy, Universal Star Network, and Universal Storage Platform are registered trademarks of Hitachi Data Systems Corporation.
    [Show full text]