Openio SDS on ARM a Practical and Cost-Effective Object Storage Infrastructure Based on Soyoustart Dedicated ARM Servers
Total Page:16
File Type:pdf, Size:1020Kb
OpenIO SDS on ARM A practical and cost-effective object storage infrastructure based on SoYouStart dedicated ARM servers. Copyright © 2017 OpenIO SAS All Rights Reserved. Restriction on Disclosure and Use of Data 30 September 2017 "1 OpenIO OpenIO SDS on ARM 30/09/2017 Table of Contents Introduction 3 Benchmark Description 4 1. Architecture 4 2. Methodology 5 3. Benchmark Tool 5 Results 7 1. 128KB objects 7 Disk and CPU metrics (on 48 nodes) 8 2. 1MB objects 9 Disk and CPU metrics (on 48 nodes) 10 5. 10MB objects 11 Disk and CPU metrics (on 48 nodes) 12 Cluster Scalability 13 Total disks IOps 13 Conclusion 15 2 OpenIO OpenIO SDS on ARM 30/09/2017 Introduction In this white paper, OpenIO will demonstrate how to use its SDS Object Storage platform with dedicated SoYouStart ARM servers to build a flexible private cloud. This S3-compatible storage infrastructure is ideal for a wide range of uses, offering full control over data, but without the complexity found in other solutions. OpenIO SDS is a next-generation object storage solution with a modern, lightweight design that associates flexibility, efficiency, and ease of use. It is open source software, and it can be installed on ARM and x86 servers, making it possible to build a hyper scalable storage and compute platform without the risk of lock-in. It offers excellent TCO for the highest and fastest ROI. Object storage is generally associated with large capacities, and its benefts are usually only visible in large installations. But thanks to characteristics of OpenIO SDS, this next-generation object storage solution can be cost effective even with the smallest of installations. And it can easily grow from a simple setup to a full-sized data center, depending on users’ storage needs: one node at a time, with a linear increase in performance and capacity. OpenIO SDS’s simple, lightweight design allows it to be installed on very small nodes. The minimal requirements for a node on ARM are 512MB RAM and 1CPU core, with packages available for Raspbian and Ubuntu Linux (supporting Raspberry Pi computers). The software’s true flexibility is powered by Conscience technology, a set of algorithms and mechanisms that continuously monitors the cluster, computing quality scores for all nodes, and choosing the best node for each operation. Thanks to Conscience, all operations are dynamically distributed and there is no need to rebalance the cluster when new resources are added. By partnering with SoYouStart, a dedicated server provider built on OVH’s infrastructure around the world, we have been able to build a cluster and run a complete set of benchmarks to demonstrate the benefts of an ARM-based storage solution that can quickly scale from three nodes to an unlimited number of nodes at a reasonable cost. Thanks to its very competitive offer, SoYouStart enables small and medium-sized organizations to build private infrastructures and making them available on high-speed public networks without the hassle of managing physical hardware purchasing and maintenance. For this benchmark, the OpenIO team worked on a 48-node confguration to take advantage of erasure coding and parallelism, but the minimal confguration supported in production starts at three nodes, allowing end users with the smallest storage needs to take advantage of object storage at reasonable prices. SoYouStart offers ARM-based servers in European and North American datacenters, and the confguration described in the following pages is replicable in those countries, allowing end users to be compliant with local laws and regulations. (https://www.soyoustart.com/en/server-storage/) 3 OpenIO OpenIO SDS on ARM 30/09/2017 Benchmark Description 1. Architecture For our test, we chose to work on a two-tier architecture. This allowed us to install the necessary software to run the benchmark testing suite on two X86 nodes, which also acted as Swift gateways, while the storage layer was built around 48 ARM servers (2 CPU cores, 2GB RAM, 2TB HDD, unlimited traffic, and a public IP address: https://www.soyoustart.com/fr/offres/162armada1.xml). Each node was confgured to reflect the minimum requirements for OpenIO SDS. FileSystem: • Rootvol (/): EXT4, 8 GB • Datavol for OpenIO SDS data (/var/lib/oio): XFS, 1.9 TB The object store was confgured using a dynamic data protection policy which enabled erasure coding for objects larger than 250 KB and three-way replication for smaller ones. OpenIO SDS is easy to deploy and scale thanks to the available Ansible role (available at https:// github.com/open-io/ansible-role-openio-sds) and the OVH/SoYouStart APIs. 4 OpenIO OpenIO SDS on ARM 30/09/2017 2. Methodology For the benchmark, the platform was populated with 500 containers and 50 objects in each container. Sizes of objects were 128KB, 1MB, or 10MB distributed in equal parts. A frst type of test (80% read / 20% write), which is close to a real use case scenario, was performed for each object size, and we launched seven different runs based on different levels of parallelism (5, 10, 20, 40, 80, 160, and 320 workers). A second test was designed to test the linear scalability of the solution. In this case, a 100% read run was launched against 10MB objects on three different cluster confgurations: 12, 24, and 48 nodes. Each runs lasted 5 minutes (long enough to see any performance issues). 3. Benchmark Tool We ran the tests using COSbench, a tool developed by Intel to test object storage solutions. It is open source, so results can be easily compared and verifed. In this case, we chose to use the Swift API, but OpenIO SDS is also compatible with the S3 API. COSbench (https://github.com/intel-cloud/cosbench) features include: - Easy to use via a web interface or on the command line - Exports signifcant metrics for comparative usage - All metrics are saved in CVS format 5 OpenIO OpenIO SDS on ARM 30/09/2017 The benchmark was organized in five phases: init: container creation prepare: container population with objects main: bench scenario (with all possible read/write/delete combinations) cleanup: object deletion dispose: container deletion Here is an example of a result page. 6 OpenIO OpenIO SDS on ARM 30/09/2017 Results 1. 128KB objects Response time (ms) Throughput (op/s) 128KB objects 80% read 800 160 144,59 138,9 139,67 141,23 600 120 98,46 400 80 70,46 Throughput(op/s) Response time(ms) 200 40 34,13 0 0 5 10 20 40 80 160 320 Number of workers Response time (ms) Throughput (op/s) 128KB objects 20% write 1800 40 34,23 34,46 34,82 35,23 1350 30 25,01 900 20 17,94 Throughput(op/s) Response time(ms) 450 10 8,72 0 0 5 10 20 40 80 160 320 Number of workers 7 OpenIO OpenIO SDS on ARM 30/09/2017 Disk and CPU metrics (on 48 nodes) 8 OpenIO OpenIO SDS on ARM 30/09/2017 2. 1MB objects Response time (ms) Throughput (op/s) 1MB objects 80% read 800 140 136,84 137,59 130,81 126,25 600 105 84,06 400 70 Throughput (op/s) Response time(ms) 48,26 200 35 29,49 0 0 5 10 20 40 80 160 320 Number of workers Response time (ms) Throughput (op/s) 1MB objects 20% write 2400 40 34,43 33 33,68 1800 31,23 30 1200 20,55 20 Throughput (op/s) Response time(ms) 600 11,83 10 7,15 0 0 5 10 20 40 80 160 320 Number of workers 9 OpenIO OpenIO SDS on ARM 30/09/2017 Disk and CPU metrics (on 48 nodes) 10 OpenIO OpenIO SDS on ARM 30/09/2017 5. 10MB objects Response time (ms) Throughput (op/s) 10MB objects 80% read 4000 70 69,68 63,41 3000 52,5 53,08 43,99 2000 35 32,09 Throughput(op/s) Response time(ms) 1000 21,17 17,5 11,38 0 0 5 10 20 40 80 160 320 Number of workers Response time (ms) Throughput (op/s) 10MB objects 20% write 6000 16 15,69 15,89 13,22 4500 12 10,75 3000 8 8,13 Throughput(op/s) Response time(ms) 5,14 1500 4 2,95 0 0 5 10 20 40 80 160 320 Number of workers 11 OpenIO OpenIO SDS on ARM 30/09/2017 Disk and CPU metrics (on 48 nodes) 12 OpenIO OpenIO SDS on ARM 30/09/2017 Cluster Scalability To demonstrate the linear scalability of OpenIO SDS, as well as its ability to scale quickly when needed, we simulated a cluster of 12 nodes as it was expanded to 24, then 48 nodes. We ran three benchmarks using 80 workers confgured to perform 100% read operations on 10MB objects with the three confgurations. Bandwidth (MB/s) Throughput (op/s) 10MB objects 100% read 1100 1.044,48 100 825 75 553,45 550 99,42 50 Bandwidth (MB/s) Throughput (op/s) 235,15 275 54,05 25 22,96 0 0 12 24 48 Number of nodes 80 workers 12 nodes 2,5 KIOps 24 nodes 5 KIOps 48 nodes 8,7 KIOps Total disks IOps Each disk delivers between 180 IOps and 210 IOps, which is very good for SATA disks (and is also the limit as we reach 100% disk utilization). 13 OpenIO OpenIO SDS on ARM 30/09/2017 On the 48-node cluster, we confgured the number of workers to increase progressively from 20 to 40, then 80, 160, and 320. After the COSbench preparation phase, we found that the highest bandwidth was achieved with 80 workers.