Reshaping Storage Accelerate for the Next OpenStack* Generation Data Together Center
* OpenStack is a registered trademark of the OpenStack Foundation Agenda
• Data Explosion & Storage Pain Points
• Software Defined Storage (SDS) vision
• OpenStack SDS Controller Prototype
• Ceph and Virtual Storage Manager
• Fujitsu Hyperscale Storage for OpenStack
• OpenStack Swift
• Summary
OpenStack Summit, Paris, Nov. 3-7, 2014 2 Agenda
• Data Explosion & Storage Pain Points
• Software Defined Storage (SDS) vision
• OpenStack SDS Controller Prototype
• Ceph and Virtual Storage Manager
• Fujitsu Hyperscale Storage for OpenStack
• OpenStack Swift
• Summary
OpenStack Summit, Paris, Nov. 3-7, 2014 3 Data Explosion:
From 2013 to 2020, the digital universe will grow by a factor of 10, from 4.4 ZB to 44 ZB It more than doubles every two years.
… cost challenges continue to grow……….
….data complexity is increasing
…. And DC’s spend 40% of their budget on storage
Data needs are growing at a rate unsustainable with today’s infrastructure and labor costs
Source: IDC – The Digital Universe of Opportunities: Rich Data and the Increasing Value of the Internet of Things - April 2014
OpenStack Summit, Paris, Nov. 3-7, 2014 4 Storage Pain Points
Storage Silos (Traditional) Challenges • Application mapped to specific • Cost: silo’d storage solutions appliance - Data Growth • Storage resources optimized to run - Maintenance, Operations & Support specific workload - Infrastructure • Isolated storage resources • Choice: lock-in - Vertically integrated solutions - Flexibility to innovate • Cloud deployment - Limited Scalability - Need for massively shared data
Traditional storage management is too complex and inefficient
OpenStack Summit, Paris, Nov. 3-7, 2014 5 Agenda
• Data Explosion & Storage Pain Points
• Software Defined Storage (SDS) vision
• OpenStack SDS Controller Prototype
• Ceph and Virtual Storage Manager
• Fujitsu Hyperscale Storage for OpenStack
• OpenStack Swift
• Summary
OpenStack Summit, Paris, Nov. 3-7, 2014 6 Software Defined Infrastructure (SDI)
Services Delivery SERVICE ASSURANCE Policy based dynamic provisioning App/SLA mapping to underlying Infrastructure Application Application Application Application A B C D
Orchestration Software Orchestration Software PROVISIONING MANAGEMENT Orchestration provisions, manages and optimally allocates resources based on the unique Infrastructure Attributes requirements of an application Power Performance Security Thermals Utilization Location
POOLED RESOURCES Resource Pool Network, Storage and Compute elements are abstracted into resource pools Storage Network Compute
Efficient SDI requires Application Infrastructure interaction Servers play a critical role for Compute, Network, Storage, Security, Energy and Manageability
OpenStack Summit, Paris, Nov. 3-7, 2014 7 SDS – A Key Component of SDI Dynamic, policy-driven storage resource management Services Delivery Abstracting Software from Hardware,
App #1 App #2 App #3 App #4 providing flexibility & scalability Aggregating diverse provider solutions, Orchestration Software increasing flexibility and drive down costs Infrastructure Attributes Provisioning resources dynamically (pay-as- Power Performance Thermals Utilization Location Latency Durability you-grow) increasing efficiency
Software Defined Storage Controller Orchestrating application access to diverse
Storage Systems storage systems through Service Level Agreements (SLAs), increasing flexibility and handle data complexity
SDS is a framework that delivers a scalable, cost-effective solution to serve the needs of tomorrow’s Data Center
OpenStack Summit, Paris, Nov. 3-7, 2014 8 SDS Architecture
Data Services SDS Controller • Visibility and Control of • Application that runs in ALL storage resources DP to optimize storage Applications • Communication between • Ex: Predictive Analytics Apps, Orchestrator and • Ex: De-Duplication Storage Systems • Ex: Tiering Orchestrator • Allocates storage resources to meet SLA’s
Data Services SDS Controller SDN Controller Compute Controller
Storage System Storage System Storage System ……….. Storage System [SAN] ……….. [Capacity] [Performance] [NAS]
SAN NAS Node JBOD Node Node Node Node
SDS : Consolidated Management of Scale-Out and Scale-Up Storage and plugs into SDI
OpenStack Summit, Paris, Nov. 3-7, 2014 9 Agenda
• Data Explosion & Storage Pain Points
• Software Defined Storage (SDS) vision
• OpenStack SDS Controller Prototype
• Ceph and Virtual Storage Manager
• Fujitsu Hyperscale Storage for OpenStack
• OpenStack Swift
• Summary
OpenStack Summit, Paris, Nov. 3-7, 2014 10 SDS Functional Partitioning
Free Pool Outside world (UI, CLI, Compute, Network Provision Controllers) Discover Compose Maintain Data Mgmt SDS Controller
Block File Object
Provision Storage Volume Operations Share Operations Object Operations System (Control Plane) (Control Plane) (Control Plane)
OpenStack Summit, Paris, Nov. 3-7, 2014 11 SDS Controller – One Control Plane
Heat Horizon CLI
Provision Storage System Manage Pools App App Manage Shares App Manage Volumes App (e.g. Nova) attach/ (e.g. VM) (e.g. Nova) (e.g. Webapp) detatch attach/ Provision Volumes volume detatch Template PUT/GET/DELETE share mgmt account, container, Get Storage URL object Cinder Manila Manage volumes, Glance Shares, images Provision Object volume Provisioning, Storage SDS Controller Maintenance Discovery, etc. Cinder+ Manila+ QoS etc. (Cinder+) Glance+ Drivers Drivers Drivers Provisioning, Object Maintenance etc. Metrics, metering Servers Storage Storage Storage E.g., Object Storage – System System System Swift, Ceph, Atmos, Amplidata Ceilometer
Work with dev community to identify path to enable SDS Controller natively in OpenStack
OpenStack Summit, Paris, Nov. 3-7, 2014 12 SDS Controller Prototype
SDS Controller
Provision Discover Maintenance Compose Data Type Life (Storage (Storage (Storage (Virtual Pools) Cycle Mgmt System) Capabilities) Systems, Pools) • Expand Cluster (Ceph • Discover storage • CUDL virtual pools • CUDL volumes & shares • Deassgin resource VSM, SwiftStack system, tiers, capabilities • Attach/Detach tiers to • Attach & Detach • Unprovision resource Controller) - driver framework virtual pools volumes & shares • Retire storage system (Swift, Ceph) • Admin UI for pool • Attach object store & virtual pools composition • Clone, Snapshot, • Support for Prototype Scope • Object support Backup, Extend, Shrink maintenance mode • Get volume stats • QoS monitor, alert & Legend enforce Not in Openstack • Policies, data life-cycle Exists in Openstack mgmt. automation Note: Operations may be within storage system and • CUDL = Create, SLO based resource among storage systems Update, Delete, List requests, scheduling • Virtual pools for all operations Publish prototype by end of Dec’14 in external git repo • Data service chaining, app assignment
OpenStack Summit, Paris, Nov. 3-7, 2014 13 Agenda
• Data Explosion & Storage Pain Points
• Software Defined Storage (SDS) vision
• OpenStack SDS Controller Prototype
• Ceph and Virtual Storage Manager
• Fujitsu Hyperscale Storage for OpenStack
• OpenStack Swift
• Summary
OpenStack Summit, Paris, Nov. 3-7, 2014 14 Ceph Introduction An “Open source” distributed storage software that provides object, block and file storage access
KEYSTONE SWIFT API GLANCE CINDER NOVA Intel Focus • Optimize Ceph block storage for Qemu/KVM Open stack deployments CEPH Object Gateway CEPH Block Device (RBD) • Deliver Open stack and Ceph S3 and Swift compatible object Virtual block device for virtual machines Reference Architectures gateway and apps CEPH RADOS Storage Cluster • Lead SDS Controller integration with Ceph (prototype and A Reliable, Autonomous, Distributed beyond) Object Store for unstructured data
• Ceph is the “only” Open source virtual block storage option. Also provides object, file (experimental) • Strong customer interest - several production implementations already
OpenStack Summit, Paris, Nov. 3-7, 2014 15 Intel Virtual Storage Manager - Design Goals
Consistent Configuration Segregated Storage Support • VSM controls cluster configuration through pre- • Separate storage assets by function or defined manifests performance – Storage Groups • Operator-selectable configuration options are • Drives with similar characteristics explicitly defined and validated by OEM or System identified – no ambiguous discovery Integrator • Capacity utilization and performance tracked • Reduces maintenance and support cost due to by Storage Group customer mis-configuration, “exotic” configurations
Efficient Asset Management Familiar Look and Feel • OpenStack UI schema • Manage server lifetime – add and remove • “Flat” navigation storage at unit of server • Tabular data grouping – minimize drill- • Manage device lifetime - repair storage at down unit of drive • Comprehensive one-page summary • Error and Status Reporting
Intel Virtual Storage Manager for Ceph OpenStack Summit, Paris, Nov. 3-7, 2014 16 Virtual Storage Manager
Web-based UI . Administrator-friendly interface for management and monitoring Configuration management . Storage Groups aggregate drives with similar performance . Zones aggregate drives within failure domains Capacity Management . Pools segregate allocation by application . Capacity utilization by storage group and pool Cluster Management . Manage capacity growth . Manage Server & disk maintenance Cluster Monitoring . Capacity and performance . OSD, PG, and monitor state VSM RESTful API . Software interface supports automation Management framework = Consistent configuration Operator-friendly interface for management & monitoring
1 7OpenStack Summit, Paris, Nov. 3-7, 2014 17 Managing the Cloud Storage System Lifecycle
OpenStack Summit, Paris, Nov. 3-7, 2014 18 Dashboard Overview
OpenStack Summit, Paris, Nov. 3-7, 2014 19 VSM Open Source Project
• Sign up: • Register for access at 01.org and github.com • Find it: • At 01.org/virtual-storage- manager • Get it: • Source and documentation • Apache 2 license • Contribute: • Join the VSM community and discussing features. • See 01.org/virtual-storage- manager for details
Intel Virtual Storage Manager for Ceph OpenStack Summit, Paris, Nov. 3-7, 2014 20 Agenda
• Data Explosion & Storage Pain Points
• Software Defined Storage (SDS) vision
• OpenStack SDS Controller Prototype
• Ceph and Virtual Storage Manager
• Fujitsu Hyperscale Storage for OpenStack
• OpenStack Swift
• Summary
OpenStack Summit, Paris, Nov. 3-7, 2014 21 ETERNUS CD10000 Hyperscale Storage for OpenStack
Dieter Kasper CTO Data Center Infrastructure, Emerging Technologies & Solutions, Fujitsu
INTERNAL USE ONLY 22 © Fujitsu 2014 3rd platform implications challenges for storage
3rd platform – Distributed software-defined scale-out storage Big Data Analytics/Social Business/Mobile Broadband/Cloud Services
Scalability Reliability Manageability Practically unlimited Full redundancy Central management scalability in terms of Self healing of huge storage performance & capacity Geographical dispersion amounts No bottlenecks Fast rebuild Unified multi-protocol No hot spots access (block, file, object) Seamless introduction of new storage
INTERNAL USE ONLY 23 © Fujitsu 2014 ETERNUS CD10000 Principles
transparent creation of data copies automatic recreation of lost redundancy
Storage Storage Storage Storage Storage Storage Node Node Node Node Node Node
pseudo-random distribution automated tech refresh
INTERNAL USE ONLY 24 © Fujitsu 2014 What is ETERNUS CD10000?
Integrated Applications Cinder Swift Manila OpenStack APIs
Backup Block Level Object Level File Level Access Access Access Application Interface Archive Central Management & SW enhancements Sync Fujitsu Software & Share Ceph Storage System S/W EC
Open Source 10GbE Frontend Network
Infiniband Backend Network Fujitsu Standard Capacity Nodes Capacity
ETERNUS CD10000 ETERNUS Hardware Performance Nodes Performance
INTERNAL USE ONLY 25 © Fujitsu 2014 Fujitsu Software Enhancements
Central software deployment Central network, logfile, and cluster management SNMP integration of all nodes and network components GUI for easier deployment, configuration, administration and maintenance Own Fujitsu Erasure Coding on Roadmap Still 100% compatible with underlying open source Integrated Middleware / Applications: Backup, Archive, Sync & Share, ...
INTERNAL USE ONLY 26 © Fujitsu 2014 Virtual Storage Manager
Web-based UI Configuration Management Capacity Management Cluster Management Cluster Monitoring VSM API
INTERNAL USE ONLY 27 © Fujitsu 2014 ETERNUS CD 10k HA/DR Design
VMs PY VMs PY VMs PY VMs PY Block Object Block Object
Redundant Client Interconnect (IP based) Redundant Client Interconnect (IP based)
Stor Stor Stor Stor Stor Stor Stor Stor Node Node Node Node Node Node Node Node
SSD SSD SSD SSD SSD SSD SSD SSD
SSD SSD ~ 100x SSD SSD SSD SSD ~ 100x SSD SSD Storage Storage SAS SAS Nodes SAS SAS SAS SAS Nodes SAS SAS
SAS SAS SAS SAS SAS SAS SAS SAS
SAS SAS SAS SAS DR Node SAS SAS SAS SAS (at 3rd site) SAS SAS SAS SAS SAS SAS SAS SAS
Redundant Cluster Interconnect (IP based) Redundant Cluster Interconnect (IP based) INTERNAL USE ONLY 28 © Fujitsu 2014 ETERNUS CD 10k HA/DR Design
App PY App PY App’ PY App’ PY … … File Block File Block
Redundant Client Interconnect (IP based) Redundant Client Interconnect (IP based)
Gold Pool VolA-1 VolA-1 2ndVolA-2 3rdVolA-2 2nd 3rd VolA-1 VolA-2 VolA-3 … VolA-3 VolA-3 primary primary primary 2nd 3rd Class repli=2 repli=3 repli=4 ECode Silver Pool VolB-1 VolB-1 SSD VolB-1 Gold 2ndVolB-2 3rdVolB-2 4thVolB-2 VolB-1 VolB-2 VolB-3 … 2nd 3rd 4th primary primary primary VolB-3 VolB-3SAS Steel VolB-3 Silver 2nd 3rd 4th SATA Bronze Bronze Pool VolC-1 VolCVolC-1 -1 rd 2ndVolC-2 3rdVolC3 VolC-2 -2 VolC-1 VolC-2 VolC-3 … 2nd 3rd 3rd primary primary primary VolC-3 VolCVolC-3 -3 2nd 3rd 3rd
INTERNAL USE ONLY 29 © Fujitsu 2014 ETERNUS Hyperscale SDS versus self-made open source storage
Build open source storage yourself Out of the box ETERNUS Hyperscale SDS
incl. support I incl. maintenance
+ Easy Deployment / Management + E2E Solution Contract
ETERNUS "out of the box" hassle-free & maintained hyperscale storage appliance.
INTERNAL USE ONLY 30 © Fujitsu 2014 Agenda
• Data Explosion & Storage Pain Points
• Software Defined Storage (SDS) vision
• OpenStack SDS Controller Prototype
• Ceph and Virtual Storage Manager
• Fujitsu Hyperscale Storage for OpenStack
• OpenStack Swift
• Summary
OpenStack Summit, Paris, Nov. 3-7, 2014 31 Intel’s Work With The Swift Community • Storage Policies • Framework for extending Swifts capabilities around durability, performance, geographic segmentation, etc. • Released summer ‘14 as the defining feature for Swift 2.0 • Part of Juno Integrated Release • Erasure Code Support • Builds upon Storage Policies • Community is hard at work targeting Kilo intercept • Design session later this week covering details • Work includes development of a common Python erasure coding library (PyECLib) with multiple pluggable C modules enabling algorithm choice
OpenStack Summit, Paris, Nov. 3-7, 2014 32 What is COSBench?
. Open Source Cloud Object Storage Storage backend Auth Benchmarking Tool tempauth . Announced at the Portland design summit 2013 OpenStack* Swift swauth keystone . Open Source (Apache License) Iometerdirect none Amplidata* Amplistor(block) . Cross Platform (Java + Apache OSGI) basic/digest . Distributed load testing framework librados Ceph rados GW (swift) . Pluggable adaptors for multiple object storage rados GW (s3) backends Amazon* S3COSBenchintegrated Scality sproxyd (object) . Flexible workload definition CDMI . Web-based real-time performance monitoring CDMI-base basic/digest CDMI-swift swauth/keystone . Rich performance metric reporting (Performance None none timeline, Response time histogram) Mock mock
OpenStack Summit, Paris, Nov. 3-7, 2014 33 Agenda
• Data Explosion & Storage Pain Points
• Software Defined Storage (SDS) vision
• OpenStack SDS Controller Prototype
• Ceph and Virtual Storage Manager
• Fujitsu Hyperscale Storage for OpenStack
• OpenStack Swift
• Summary
OpenStack Summit, Paris, Nov. 3-7, 2014 34 Summary OEM/ISV Partner Solution Reference Architectures Work with co-travelers to develop reference architectures to support SDS concepts Developing an SDS Controller Prototype
Initial open source prototype released Q4’14 Open Source Storage Software
Ceph, Swift
Virtual Storage Manager Simplified Management Tool for Ceph – open sourced Q4’14
OpenStack Summit, Paris, Nov. 3-7, 2014 35 Backup Slides
OpenStack Summit, Paris, Nov. 3-7, 2014 36 SDS Controller Use Case: Application Assignment
Backup Content Database Sharing
SDS Controller 2 Compose
1 Discover 3 Assign
Storage Storage Storage Storage Storage $$$ Gold System 1 System System System System $$ Silver [SAN] [Distributed] [NAS] [Distributed] [Distributed] $ Bronze IOPS – 10K IOPS – 5K Tpt – 100Mbps Tpt – 50 Mbps Tpt – 5 Mbps (DC1) (DC2) (DC2) (DC1) (DC1)
OpenStack Summit, Paris, Nov. 3-7, 2014 37 SDS Controller Use Case: Storage Overflow
App Free Server Pool
1 SDS Controller Policies
2 Monitor 3 Assign
81% Storage Storage Storage Storage Storage System 1 System System SystemApp 1 System Utilization [SAN] [Distributed] [NAS] [Distributed] [Distributed]
IOPS – 10K IOPS – 5K Tpt – 100Mbps Tpt – 50 Mbps Tpt – 5 Mbps (DC1) (DC2) (DC2) (DC1) (DC1)
OpenStack Summit, Paris, Nov. 3-7, 2014 38 SDS Controller Workflow
SDS Controller
Provision Discover Maintenance Compose Data Type Life (Storage (Storage (Storage (Virtual Pools) Cycle Mgmt System) Capabilities) Systems, Pools)
• Assign resource • Discover storage • CUDL virtual pools • CUDL volumes & shares • Deassgin resource • Provision resource system, tiers, capabilities • Attach/Detach tiers to • Attach & Detach • Unprovision resource • Discover QoS virtual pools volumes & shares • Retire storage system • Admin UI for pool • Attach object store & virtual pools composition • Clone, Snapshot, • Support for • Object support Backup, Extend, Shrink maintenance mode • Get volume stats • QoS monitor, alert & Legend enforce Not in Openstack • Policies, data life-cycle Exists in Openstack mgmt. automation Note: Operations may be within storage system and • CUDL = Create, SLO based resource among storage systems Update, Delete, List requests, scheduling • Virtual pools for all operations • Data service chaining, app assignment OpenStack Summit, Paris, Nov. 3-7, 2014 39 How can Intel help you get on the path to SDS? Intel® Data Plane Intel® Cache Development NETWORKING SSD’S AND NVM Acceleration Kit AND FABRIC SOFTWARE Software DEFINED Intel® Storage STORAGE Acceleration Library (ISA-L)
PROCESSOR PLATFORMS REFERENCE ARCHITECTURES Intel® QuickAssist SOFTWARE Technology ECOSYSTEM
OpenStack Summit, Paris, Nov. 3-7, 2014 40 Solution Reference Architectures (SRA’s) to drive Proof-Of- Concepts (POC’s) • WHY? • Reduce time to evaluate and get started - Low investment to implement a solution • Test a proven E2E SDS solution that solves a critical customer problem and works NOW • Quick Training for your team to get up to speed on new technology
• HOW? • Identify a problem that SDS can address • Work with Intel to select a Solutions Reference Architecture (SRA) to test • Intel will deliver the SRA which will include • White Paper (in all SRA’s) • Fully Configured Systems w/ SW (in some SRA’s) • Resources to get it up and running (in some SRA’s) • Use the SRA to Implement a Proof-Of-Concept (POC) • Drive results of the POC into Production
OpenStack Summit, Paris, Nov. 3-7, 2014 Solutions Reference Architectures (SRA’s) - WIP
Usage Models HPC File Cold Storage Block File VDI, E-Mail, VDI, E-Mail, File Consolidation, Consolidation, Consolidation, Virtualization Virtualization Consolidation Object Store Object Storage , Image store Capabilities / HPC storage Enterprise File VDI, File Share OpenStack cloud, File sharing, VDI, Tier 2/3 VDI, Tier 2/3 Scalable Use cases at Exascale Share using with in-line de- High- content cloud, test/dev with test/dev, automated Object Store with dupe and performance high- compute and remote office, management Erasure Code as compression storage with performance for storage on the disaster Storage the back-end caching large data sets same nodes recovery with automation compute and tools & cloud storage on the stacks same nodes Operating S3 I/F or NFS S3 I/F or OpenStack VMware Environ. / CIFS/NFS Orchestration VM/OS OpenZFS/ ---- agnostic agnostic RHEL/KVM ESX/vSphere ESX/vSphere ESX/vSphere Linux SDS Controller ------ProphetStor ProphetStor ------EMC VIPR
Storage Lustre Amplidata + Nexenta CEPH RH Storage, VMware Maxta MxSP Virtual TBD System Maginatics Cinder, Swift Virtual SAN SAN
Intel Xeon® Xeon® Xeon® Xeon® Xeon® Xeon® Xeon® Xeon® Technologies ISA-L Atom™ C2000 ISA-L Atom™ C2000 Atom™ C2000 10GbE Atom™ C2000 ISA-L 10GbE ISA-L 10GbE ISA-L ISA-L SSDs ISA-L 10GigE SSDs 10GbE SSDs 10GbE 10GbE 10GbE SSD Storm Lake SSDs PCSD SSDs SSDs SSDs OpenStack Summit, Paris, Nov. 3-7, 2014