DOCSLIB.ORG

Scale out Sync & Share with Seafile on Ceph

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Scale out Sync & Share with Seafile on Ceph Scale out Sync & Share with Seafile on Ceph Sönke Schippmann Stefan Wichmann Ceph Day @ CERN 17.9.2019 Scale out Sync & Share with Seafile on Ceph 1/13 Just another sync & share service ● Private cloud service, like owncloud/ nextcloud and many others ● better performance ● lightweight daemon ● not a groupware ● community / pro edition Scale out Sync & Share with Seafile on Ceph 2/13 Seafile at University of Bremen ● 9.000 Seafile users ● 1 virtual machine ● 3.000 active users ● 8 vCPUs (>50% idle) ● 21.000 libraries ● 32 GByte RAM ● 34 TByte data ● file storage: ZFS on RBD ● 150 million files ● migrating to S3 ● growing 25% per year storage Scale out Sync & Share with Seafile on Ceph 3/13 Storage backends ● Backends supported ● multiple backends (pro edition only): can be used in – file storage (default) parallel (since 6.3) – S3 (recommended for Ceph) – OpenStack Swift – Ceph / librados (faster, but no multiple backend support) Scale out Sync & Share with Seafile on Ceph 4/13 Configure S3 storage ● 3 buckets needed: – seafile-commits – seafile-fs – seafile-blocks ● configure S3 user ● consider placement target by buckets or user – separate data and index pool gives better flexibility Scale out Sync & Share with Seafile on Ceph 5/13 Enable multiple storage backends /seafile/conf/seafile.conf: [storage] enable_storage_classes = true storage_classes_file = /seafile/conf/storage_classes.json /seafile/conf/seahub_settings.py: ENABLE_STORAGE_CLASSES = True STORAGE_CLASS_MAPPING_POLICY = 'ROLE_BASED' # or 'USER_SELECT' Scale out Sync & Share with Seafile on Ceph 6/13 storage_classes.json [{"storage_id": "s3_storage", {"storage_id": "filesystem", "name": "S3", "name": "Filesystem", "is_default": false, "is_default": true, "commits": { "commits": { "backend": "s3", "backend": "fs", "use_https": true, "dir": "/seafile/seafile-data"}, "path_style_request": true, “blocks”: { ... }, “fs”: { ... } }] "bucket": "seafile-commits", "key": "yourkey", "key_id": “yourid", "host":"radosgw.example.com:7480" }, “blocks”: { ... }, “fs”: { ... }}, Scale out Sync & Share with Seafile on Ceph 7/13 Seafile caveats ● Before changing default storage to S3, repos on file storage need to be configured in database: insert into RepoStorageId (repo_id,storage_id) select r.repo_id, 'filesystem' from Repo r left join RepoStorageId s on r.repo_id=s.repo_id where s.repo_id is null Also with GarbageRepos, RepoTrash and (just to be sure) VirtualRepo ● In Seafile version 6.3: Bug in configuration code for S3 storage backend (fixed in 7.0.3, patch backport available) Scale out Sync & Share with Seafile on Ceph 8/13 Data model derived from Git Database commits/repo-id/commit-id RepoSize.head_id Branch.commit_id contains commit log text and root fs-id fs/repo-id/fs-id directory object fs/repo-id/fs-id fs/repo-id/fs-id file object file object blocks/repo-id/block-id blocks blocks blocks blocks 10 MByte chunks of data On file storage, object ID is split: 2 chars subfolder, 38 filename Scale out Sync & Share with Seafile on Ceph 9/13 Migrating data to S3 ● Seafile offers a script for offline migration ● downtime not acceptable (2-4 weeks) ● storage file names translate well to S3 object names: storage/commits/36125405-bba8-45d1-80f1-c4aeb351ecc1/ 19/1cdd9f5372bd5ee186d8bc5a1ea23236061ccd s3://seafile-commits/36125405-bba8-45d1-80f1-c4aeb351ecc1/ 191cdd9f5372bd5ee186d8bc5a1ea23236061ccd ● online migration scenario (scripts available): – copy all files, repo by repo – change repo's Storage-ID in Database Scale out Sync & Share with Seafile on Ceph 10/13 Backup and Restore ● make database dump ● restore per repository: ● copy Branch.commit_id – create new repo with WebAPI ● copy objects from S3 to a backup filesystem – copy all files with new repo ID – use filestorage layout – change head ● Seafile could directly ID/commit ID to value use backup file storage of Branch.commit_id for that backup Scale out Sync & Share with Seafile on Ceph 11/13 Running a Seafile cluster Scale out Sync & Share with Seafile on Ceph 12/13 Thank you ● More in Seafile's manual: https://manual.seafile.com ● Our slides, patch, migration and backup scripts: http://unihb.eu/4eNoRHGd Questions? [email protected] Scale out Sync & Share with Seafile on Ceph 13/13.
Load more

Recommended publications
  • Openstack Designate
    OpenStack Designate Stephan Lagerholm Graham Hayes What is OpenStack? OpenStack is a free open standard cloud computing platform, mostly deployed as infrastructure-as-a-service (IaaS) in both public and private clouds where virtual servers and other resources are made available to users. The software platform consists of interrelated components that control diverse, multi-vendor hardware pools of processing, storage, and networking resources throughout a data center. Users either manage it through a web-based dashboard, through command-line tools, or through RESTful web services. OpenStack began in 2010 as a joint project of Rackspace Hosting and NASA. As of 2012, it was managed by the OpenStack Foundation (Source: Wikipedia) 2 Designate • Designate started as a project to maintain DNS infrastructure for OpenStack Users. It was an ecosystem project that was in production at both HP’s and Rackspace’s clouds. During 2015, Designate was moved into OpenStack Foundation and in 2017 it became a registered trademark. • Designate is providing API, CLI and a Graphical User interface so that OpenStack Users can setup and make changes to DNS data. The zones are thereafter exposed to secondary DNS servers via Zone Transfers. • Officially Bind 9.X and PowerDNS 4.X is supported although other DNS servers are known to work too. Most resource Record Types such as A, AAAA, PTR, CNAME, NS, MX, etc are supported 3 Producer Producer Backend Producer Producer Producer Worker Customer Facing API Central DNS Servers Standard XFR Secured by TSIG Nova / DB Mini
    [Show full text]
  • Ovirt and Openstack Storage (Present and Future)
    oVirt and OpenStack Storage (present and future) Federico Simoncelli Principal Software Engineer, Red Hat January 2014 1 Federico Simoncelli – oVirt and OpenStack Storage (present and future) Agenda ● Introduction ● oVirt and OpenStack Overview ● Present ● oVirt and Glance Integration ● Importing and Exporting Glance Images ● Current Constraints and Limitations ● Future ● Glance Future Integration ● Keystone Authentication in oVirt ● oVirt and Cinder Integration 2 Federico Simoncelli – oVirt and OpenStack Storage (present and future) oVirt Overview ● oVirt is a virtualization management application ● manages hardware nodes, storage and network resources, in order to deploy and monitor virtual machines running in your data center ● Free open source software released under the terms of the Apache License 3 Federico Simoncelli – oVirt and OpenStack Storage (present and future) The oVirt Virtualization Architecture 4 Federico Simoncelli – oVirt and OpenStack Storage (present and future) OpenStack Overview ● Cloud computing project to provide an Infrastructure as a Service (IaaS) ● Controls large pools of compute, storage, and networking resources ● Free open source software released under the terms of the Apache License ● Project is managed by the OpenStack Foundation, a non-profit corporate entity established in September 2012 5 Federico Simoncelli – oVirt and OpenStack Storage (present and future) OpenStack Glance Service ● Provides services for discovering, registering, and retrieving virtual machine images ● RESTful API that allows querying
    [Show full text]
  • IBM Power Systems and Openpower Solutions for Hybrid Cloud Optimized Infrastructure and Solutions for Data and Computational Services in the Cloud
    IBM Systems Power Systems Solution Brief IBM Power Systems and OpenPOWER Solutions for Hybrid Cloud Optimized infrastructure and solutions for data and computational services in the Cloud Technology plays in increasingly critical role in the ability of organiza- Highlights tions in all industries and all regions of the world to compete and succeed. Ubiquitous access to the internet from anywhere is erasing traditional ●● ●●IBM® Power Systems™ and boundaries and empowering customers, partners, and even things—to OpenPOWER servers with POWER8® processor technology are purpose-built connect, respond, engage, compare and buy. There is a digital transfor- for more efficient processing of data and mation sweeping the planet and organizations that do not make bold analytic workloads in the cloud moves to adapt quickly and intelligently, will undoubtedly fall behind ●● ●●Delivers the broadest choice of cloud- and fail to grow. ready infrastructure and solutions to suit client’s needs and workloads: scale up, The challenge and responsibility of responding effectively to these scale out, converged, proven Reference Architectures, hybrid and public cloud changing dynamics rests solidly with today’s technology leaders. Without services strong leadership to chart a strategic path, organizations will flounder and will see their business decline. IT needs to adopt flexible infrastructure ●● ●●OpenStack based cloud management from IBM, ISV partners and open source strategies and agile development methods to deal with the: Linux distributions provide extensible, scalable and resilient solutions for public ●●●Explosion in amount of data that needs to be rapidly analyzed and and private clouds managed ●●●Security threats that are constant and constantly changing ●●●Need to reach global markets and supply chains ●●●Demand for fast and error-free client’ s engagement experiences IBM Systems Power Systems Solution Brief Organizations need a flexible infrastructure to enable growth and innovation while lowering overall IT costs.
    [Show full text]
  • 14. Comparison of Cloud Management Platforms
    14. Comparison of cloud management platforms Kimmo Ahokas Aalto University School of Science [email protected] Abstract tion cloud computing is divided into three different service models, namely Software as a Service (SaaS), Platform as a Cloud computing allows fast and efficient resource provi- Service (PaaS) and Infrastructure as a Service (IaaS). In this sioning within data centers. In large companies this can paper we are only interested in IaaS service model, which lead to significant savings, thus creating market for complete is defined as "The capability provided to the consumer is cloud platforms. In addition to commercial products, sev- to provision processing, storage, networks, and other fun- eral open source cloud platforms exist. This paper compares damental computing resources where the consumer is able four cloud management platforms and identifies the factors to deploy and run arbitrary software, which can include op- affecting future success of each of the platforms. We also es- erating systems and applications. The consumer does not timate the future development of the cloud platform market. manage or control the underlying cloud infrastructure but has control over operating systems, storage, and deployed KEYWORDS: cloud platform, IaaS, CloudStack, Open- applications; and possibly limited control of select network- Stack, OpenNebula, Eucalyptus, VMware ing components (e.g., host firewalls)." [10] Cloud management platform is a software system that 1 Introduction controls the allocation of physical resources on the data cen- ter. In the IaaS model users can launch virtual machines us- Cloud computing has rapidly changed the way in which re- ing the management console, which causes the platform to sources in data centers can be provisioned.
    [Show full text]
  • A Comparative Study of Current Open-Source Infrastructure As a Service Frameworks
    A Comparative Study of Current Open-source Infrastructure as a Service Frameworks Theo Lynn, Graham Hunt, David Corcoran, John Morrison and Philip Healy Irish Centre for Cloud Computing, Dublin 9, Ireland Keywords: Cloud Computing, Open Source, IaaS, Openstack, Cloudstack, Opennebula, Eucalyptus. Abstract: With the growth of cloud computing in recent years, several commercial and open source IaaS frameworks have emerged. The development of open source IaaS solutions offers a free and flexible alternative to commercial cloud services. The main contribution of this paper is to provide a qualitative comparative of current open-source IaaS frameworks. Existing research papers examining open source IaaS frameworks have focused on comparing OpenStack with a small number of alternatives. However, current research fails to adequately compare all major open source frameworks in a single study and notably lacks the inclusion of CloudStack. Our research paper provides the first overview of the five main open source cloud IaaS frameworks – OpenStack, CloudStack, OpenNebula, Eucalyptus and Nimbus. As such, this review provides researchers and potential users with an up to date and comprehensive overview of the features of each solution and allows for an easy comparison between the open source solutions. 1 INTRODUCTION the freedom to modify the source code and build a cloud that is pluggable and open to extensions while Cloud Computing technologies have seen significant reducing costs and avoiding vendor lock-in (Zhang adoption in recent years by enterprises, researchers et al., 2013; Wen et al. 2012; Bist et al., 2013). The and individuals. It is forecast that the Cloud development of open source solutions is of particular Computing industry will reach a market size of $241 importance for the further proliferation of private billion by 2020 (Reid and Kilster, 2011).
    [Show full text]
  • Scaling Openstack Clouds Using Peer-To-Peer Technologies
    Scaling OpenStack Clouds Using Peer-to-peer Technologies Master’s thesis in Computer Systems and Networks XIN HAN Department of Computer Science and Engineering CHALMERS UNIVERSITY OF TECHNOLOGY UNIVERSITY OF GOTHENBURG Gothenburg, Sweden 2016 Master’s thesis 2016:NN Scaling OpenStack CloudsUsing Peer-to-peer Technologies XIN HAN Department of Computer Science and Engineering Chalmers University of Technology And University of Gothenburg Gothenburg, Sweden 2016 Scaling OpenStack CloudsUsing Peer-to-peer Technologies XIN HAN © XIN HAN, 2016. Supervisors: Vincenzo Gulisano, Department of Computer Science and Engineering Joao Monteiro Soares, Ericsson Fetahi Wuhib, Ericsson Vinay Yadhav, Ericsson Examiner: Magnus Almgren, Department of Computer Science and Engineering Master’s Thesis 2016:NN Department of Computer Science and Engineering Chalmers University of Technology and University of Gothenburg SE-412 96 Gothenburg Telephone +46 31 772 1000 Cover: Illustration of P2P OpenStack cloud system. Each cloud is a standalone OpenStack cloud instance. Cloud instances are federated as one using P2P tech- nologies. Gothenburg, Sweden 2016 iv Scaling OpenStack Clouds Using Peer-to-peerTechnologies XIN HAN Department of Computer Science and Engineering Chalmers University of Technology and University of Gothenburg Abstract OpenStack is an open-source software platform for cloud computing, mostly de- ployed as an infrastructure-as-a-service (IaaS) and has a user base in industry and academia to date. Despite its popularity, OpenStack still has drawbacks in terms of scalability of number of compute nodes (metal machines) in a single cloud instance. More precisely, a single standard OpenStack cloud instance does not scale well and fails to handle user request once its number of compute nodes reaches a particular amount.
    [Show full text]
  • Introduction to Openstack Private Cloud
    Introduction to OpenStack Private Cloud In this guide In this e-guide: Get your head in the cloud: Intro to OpenStack components As the private cloud market is heating up, there are two clear Navigating the OpenStack players leading the way: VMware and OpenStack. However, services maze due to the VMware tax, along with rising performance issues, Overcome the most pressing many IT professionals are flocking to the attractive, lower-cost OpenStack security challenges alternative. Getting more PRO+ exclusive content In this exclusive e-guide, find out what sets OpenStack apart from its competitors, how to find out which OpenStack cloud service is right for your organization, and the steps to take to overcome its common security challenges. Page 1 of 17 In this guide Get your head in the cloud: Intro to Get your head in the cloud: OpenStack components Intro to OpenStack components http://searchservervirtualization.techtarget.com/tip/Get-your-head-in-the-cloud- Navigating the OpenStack Intro-to-OpenStack-components services maze Overcome the most pressing As more and more businesses consider the different options for a private cloud OpenStack security challenges platform, it's important to know what distinguishes open source OpenStack from Getting more PRO+ exclusive content other popular cloud computing software on the market. In this article, I will attempt to explain what OpenStack is and what it is not, and explore some essential OpenStack components without getting bogged down in technical details. The OpenStack ecosystem OpenStack is very similar to Amazon EC2 in that both platforms allow users to provision VMs from a dashboard or API.
    [Show full text]
  • HPE Digital Learner Google Cloud for Developers (Intermediate) Content Pack
    Content Pack data sheet HPE Digital Learner Google Cloud for Developers (Intermediate) Content Pack This self-paced eLearning Content Pack provides the required HPE Content Pack CP020 number training to help transition a traditional IT architect into a cloud architect utilizing the Google public cloud environment. This Content Pack 20 Hours length is a comprehensive intermediate training series that includes Content Pack Category 2 many core areas required to plan, design and implement category modern cloud infrastructures, applications and data in the Learn more View now Google public cloud. This training is intended for IT personnel who need to rapidly uplift their IT/cloud skills as part of an ongoing business and operational strategy to migrate IT to Google public cloud. Why HPE Education Services? • IDC MarketScape leader 5 years running for IT education and training* Audience Content Pack objectives • Recognized by IDC for leading with This Content Pack provides the information global coverage, unmatched technical • IT professionals including managers, expertise, and targeted education engineers and developers evaluating or necessary needed to transition a traditional consulting services* implementing application environments on IT architect into a cloud architect capable of • Key partnerships with industry leaders Google Cloud Platform working within a typical Google public cloud OpenStack®, VMware®, Linux®, Microsoft®, environment. This training series provides • Data professionals who are responsible ITIL, PMI, CSA, and SUSE
    [Show full text]
  • 60 Recipes for Apache Cloudstack
    60 Recipes for Apache CloudStack Sébastien Goasguen 60 Recipes for Apache CloudStack by Sébastien Goasguen Copyright © 2014 Sébastien Goasguen. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://safaribooksonline.com). For more information, contact our corporate/ institutional sales department: 800-998-9938 or [email protected]. Editor: Brian Anderson Indexer: Ellen Troutman Zaig Production Editor: Matthew Hacker Cover Designer: Karen Montgomery Copyeditor: Jasmine Kwityn Interior Designer: David Futato Proofreader: Linley Dolby Illustrator: Rebecca Demarest September 2014: First Edition Revision History for the First Edition: 2014-08-22: First release See http://oreilly.com/catalog/errata.csp?isbn=9781491910139 for release details. Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc. 60 Recipes for Apache CloudStack, the image of a Virginia Northern flying squirrel, and related trade dress are trademarks of O’Reilly Media, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O’Reilly Media, Inc. was aware of a trademark claim, the designations have been printed in caps or initial caps. While every precaution has been taken in the preparation of this book, the publisher and authors assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein.
    [Show full text]
  • Understand IBM Cloud Manager V4.2 for IBM Z Systems
    Open by design Understand IBM Cloud Manager V4.2 for IBM z Systems Kershaw Mehta [email protected] August 14, 2015 Open by design Many organizations begin the cloud journey with a local cloud Organizations are looking for benefits from local clouds ° Higher system and application availability ° Reduce labor costs and TCO ° Maximize existing hardware utilization ° Improve IT service quality ° Increase security by starting behind enterprise firewall And are preparing infrastructure to build the foundation ° Highly virtualized infrastructure ° Standardized virtual workloads ° Web-based provisioning ° Dynamic resource scaling 2 © 2015 IBM Corporation Open by design OpenStack leads the open cloud infrastructure movement OpenStack is a global collaboration of developers and cloud computing technologists working to produce a ubiquitous Infrastructure as a Service (IaaS ) open source cloud computing platform for public and private clouds. Platinum Sponsors Gold Sponsors Why OpenStack ° Open, modular design – flexible architecture and open APIs ° Vendor interoperability – protects clients’ current investment with simple path to new technology ° Rapid innovation – largest active open source community (over 16,000) effort enables faster developmental effort 3 http://OpenStack.org © 2015 IBM Corporation Open by design IBM uses open source software to deliver a vendor-neutral, open-by- design solution OAuth Cloud business apps Digital innovation platform World class cloud infrastructure IBM has 24 core contributors 24 and 10% of the OpenStack leadership team Active IBM developers in 120+ OpenStack projects IBMers working on OpenStack, from formation of the 450+ Foundation to code quality and new functionality 4 © 2015 IBM Corporation Open by design Introducing IBM Cloud Manager with OpenStack An easy to deploy and use cloud management software offering based on OpenStack with IBM enhancements and support • Enables on-premise rapid IT response to changing business demands, via self-service provisioning of infrastructure services.
    [Show full text]
  • Exploring Opportunities: Containers and Openstack
    Today, containers are being used for two major purposes. Containers oer deployment speed advantages over virtual machines because they’re smaller An entire system – operating system, applications, services, etc. – can exist inside a container. megabytes instead of gigabytes. Typical application containers can be started in seconds, These are called, appropriately enough, system or OS containers. System containers’ use cases whereas virtual machines often take minutes. Containers also allow direct access to device are eectively similar to those of virtual machines. drivers through the kernel, which makes I/O operations faster than with a hypervisor approach 2015 where those operations must be virtualized. Even in environments with hundreds or thousands However, application containers are dierent. Containers are a way of bundling and running of containers, this speed advantage can be signicant and contributes to overall applications in a more portable way. They can be used to break down and isolate parts of responsiveness new workloads can be brought online quickly and make boot storms become a applications, called microservices, which allow for more granular scaling, simplied management, thing of the past. Exploring superior security congurations, and solving a class of problems previously addressed with conguration management (CM) tools. They are not a replacement for virtualization or CM. Containers create a proliferation of compute units, and without robust monitoring, Opportunities: management, and orchestration, IT administrators will be coping with “container sprawl”, A developer can put an application or service inside a container, along with the runtime requisites where containers are left running, mislocated or forgotten. As a result, some third-party Containers and and services the application requires, without having to include a full operating system.
    [Show full text]
  • Open Source Solution for Cloud Computing Platform Using Openstack
    See discussions, stats, and author profiles for this publication at: http://www.researchgate.net/publication/263581733 Open Source Solution for Cloud Computing Platform Using OpenStack CONFERENCE PAPER · MAY 2014 DOI: 10.13140/2.1.1695.9043 CITATIONS DOWNLOADS VIEWS 8 826 204 5 AUTHORS, INCLUDING: Rakesh Kumar Neha Gupta JECRC Foundation University of Adelaide 20 PUBLICATIONS 69 CITATIONS 8 PUBLICATIONS 36 CITATIONS SEE PROFILE SEE PROFILE Shilpi Charu Kanishk Jain Rajasthan Technical University JECRC Foundation 10 PUBLICATIONS 33 CITATIONS 2 PUBLICATIONS 14 CITATIONS SEE PROFILE SEE PROFILE Available from: Rakesh Kumar Retrieved on: 10 August 2015 Rakesh Kumar et al, International Journal of Computer Science and Mobile Computing, Vol.3 Issue.5, May- 2014, pg. 89-98 Available Online at www.ijcsmc.com International Journal of Computer Science and Mobile Computing A Monthly Journal of Computer Science and Information Technology ISSN 2320–088X IJCSMC, Vol. 3, Issue. 5, May 2014, pg.89 – 98 RESEARCH ARTICLE Open Source Solution for Cloud Computing Platform Using OpenStack Rakesh Kumar1, Neha Gupta2, Shilpi Charu3, Kanishk Jain4, Sunil Kumar Jangir5 1,2,3,4,5Department of Information Technology, JECRC, Jaipur, India 1 [email protected], 2 [email protected], 3 [email protected] 4 [email protected], 5 [email protected] Abstract— OpenStack is a massively scalable open source cloud operating system that is a global collaboration of developers and cloud computing technologists producing the ubiquitous open source cloud computing platform for public and private clouds. OpenStack provides series of interrelated projects delivering various components for a cloud infrastructure solution as well as controls large pools of storage, compute and networking resources throughout a datacenter that all managed through a dashboard(Horizon) that gives administrators control while empowering their users to provision resources through a web interface.
    [Show full text]