BRKINI-1011.Pdf

Introduction to Storage Networking

J Metz, Ph.D. R&D Engineer, Advanced Storage BRKINI-1011 Cisco Spark

Questions? Use Cisco Spark to communicate with the speaker after the session

How 1. Find this session in the Cisco Live Mobile App 2. Click “Join the Discussion” 3. Install Spark or go directly to the space 4. Enter messages/questions in the space Agenda

• Introduction • Storage Perspectives • Block, File, and Object Storage • Block: FC, FCoE, iSCSI, NVMe • File: NFS, SMB • Object: Ceph, S3, Swift • Resources & References Abstract

• Storage networks are more than just “storage at the end of a network.” If you’re new to storage or storage networks, this session will survey delve into the technology forces behind the the major types of storage networks, including block (Fibre Channel, FCoE, iSCSI NVMe and NVMe over Fabrics), File (SMB and NFS) and Object (including CEPH, S3, Swift). Key to understanding the alphabet soup of these protocols is what kinds of applications they're well-suited for. We’re going to be looking at more than just how these technologies work, but also why we choose them for specific tasks, and where inside (and outside) the data centre they are implemented. At the end of this seminar you will be able to understand the “sweet spot” for deploying different storage networks, where the danger zones are, and have a foundation for knowing which is the right tool for the job.

• What this presentation is • A “Big Picture” approach • Understanding the right questions to ask • Examination of the trade-offs when choosing storage techniques • What this presentation is not • A product or configuration conversation • Competitive analysis

• Get your screenshots ready when you see this symbol:

• Useful for getting more information about a topic, URLs, etc.

• Problem:

• I need to have very fast access for my Database application

• Problem:

• I need to have multiple users share a single storage device for their home directories

• Problem:

• I need to store a lot of images all over the world

• There is a “sweet spot” for storage

• Depends on the workload and application type

• No “one-size fits all”

• Understanding “where” the solution fits is critical to understanding “how” to put it together

• Many presentations look like this

• Focus on connectivity, not solutions

Compute Network Storage

• Explains the relationships between devices for a specific technology

• Easy to break down the component parts and “zoom in” to different sections

• Starts off in the middle; assumes you already know why you want to use that kind of technology

• How do I know which one I should use?

• People will learn how to connect their favorite technology, without asking whether or not it’s the right one to use

BRKINI-1011 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 13 Fibre Channel (block) Mission Critical Servers Databases OLTP Apps Requirements Map* Random I/O Boot Servers • Different applications vmdk have different NFS/SMB(File) performance Shared home Directories Backup characteristics Clustered FS Microsoft AD • By designing for Sharepoint application requirements, Hyper-V Multimedia iSCSI (block) SAN does not have to be Unstructured Data “over-built” Web Content Archival Images • There are applications Object that can fit into different *not to scale areas

• Different problems have different solutions

• No “one size fits all approach”

• Many ways to solve a problem

• Lots of overlap

• Can easily get confused about which to choose

• If two different approaches can do the same thing, how do you know what to do?

• Block Storage • Host and Storage are very close together • Latency-sensitive • Rigid Architectures • High Performance, designed for highly transactional, rapidly changing data • Distance is for Disaster Recovery and Backup only

• File Storage • Less rigid architecture, less performant than Block • Inside and Outside Data Centre • Designed for sharing data among clients at scale • Distance can be for normal operations, Disaster Recovery, and Backup

• Object Storage

• Least performant

• For data that doesn’t change much, if at all

• Designed for scale and distance - access from anywhere

• Internal Storage

• Buffering & Caching; Load/Store Operations; Local I/O operations

• Local Network

• Inside a Data Centre

• Hosts communicate with storage targets or network-attached storage

• Wide-Area Network

• Backup and Disaster Recovery, Remote Offices, Distributed sharing systems (e.g., Drop-Box)

• Global Network

• Global object data/storage

• Cloud

• Catch-all storage for “storage not located here”

• A general guide to understanding “where” technologies play

• From the local (top of the chart) to the global (bottom) you can marry the needs of workloads and applications

• Most storage solutions focus in the green area

• Notice how the same technologies fit into multiple layers

• Boundaries are not rigid: new technologies move the goalposts

• There is no storage technology that is a “direct replacement” for another storage technology

• There are always trade-offs

• The risk is where these concepts overlap

• The trades are more “off” than on

• How much data are we talking about…

• Now?

• 5 years from now?

• 10 years from now?

• How often will the data change?

• How quickly do I need it?

• What are the regulatory (government and industry) compliance concerns?

• What are the availability and survivability objectives?

BRKINI-1011 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 27 Looking at Workloads • Each application has a unique signature • Write v. Read % • Random versus sequential access • Data v. Metadata % • Data Compressibility • Block/Chunk size • Metadata command Frequency • Use of asynchronous/compound commands • Intrinsic to characterising storage performance and guiding design decisions • Examples: • Virtual Server Infrastructure with Boot-From-SAN • Block/File/Object? • Virtual Desktop Infrastructure (VDI)? • Database (OLTP, SQL, NoSQL workloads)?

• Block • Transactional Data • Random Read/Write loads • Virtual Machine File System (VMFS) volumes • File • Locking and Sharing mechanisms for files • VMFS Volumes supported with latest versions of NFS v4.1 • Clustered File System Support • Dynamic Load Balancing across network add robustness to clustered file systems • Microsoft Active Directory, SQL, Sharepoint, Hyper-V environments • Replication and Backup workloads • De-Duplication • Object • Unstructured Data (mostly read-oriented), minimal writes or incremental updates • Web Content • Data Backups • Archival images • Multimedia • Geographically distributed back-end storage • Replication of back-end storage clusters across multiple data centres

• I/O Load Reference Slide • Block Size • Access Pattern • Locality of Access • Latency Sensitivity

• I/O Load Reference Slide • Constant/Bursty? • Spikes? Variance time frame? • Growth rate? • # of Concurrent users? • Examples • Video editing/post production, VoD/Video services • Healthcare and imaging • Database Transactions, e.g., Online Transaction Processing (OLTP) • Backup and recovery/Data Migration • Storage design impact • Load-balancing and multi-pathing • QoS • SSD applications (think $/IO, not $/MB)

Source: Greg Shultz. http://searchstorage.techtarget.com/tip/How-to-plan-for-I-O-intensive-environments

• Block Size Reference Slide • Small (512 bytes – 8k) • Medium (8-32k) • Large (32k+, 1MB possible)

• 4k is most often cited in benchmarks, but varies

• Tradeoffs • Small block sizes

• Reduce block contention, but has relatively large overhead due to metadata • Large block size

• Can permit large reads/writes, but can increase block contention with random access to small data flows

• Examples • Oracle recommends smaller block sizes for small rows with lots of random access • Larger block sizes for sequential access or very large rows

• Access Pattern Reference Slide • Random versus Sequential • Caching, and its effect on read/write mixtures • Leads to I/O blender effect • Random Access • Databases – data reads and modifications are made in a scattered manner across the data set • Sequential Access • Backup and restore, logging

Source: SNIA. http://www.snia.org/sites/default/files/PerformanceBenchmarking.Nov2010.pdf

• Locality of Access Reference Slide • Get storage as close to host as possible • Get backups as far away from storage as possible • Distributed Caching systems are becoming popular

• Good for predictable I/O patterns • Examples • Smaller systems: Hyperconverged • Non-time-sensitive webscale applications: Object • Time-sensitive: Block • Storage Design Impact • Consider Tiering Solutions • E.g. Mixed Block/File/Object modes for data centre storage design

• Latency Sensitivity Reference Slide • I/O dependencies? (ex. Transactions that require multiple I/Os to complete) • Complex application logic may reduce sensitivity to latency • Example: • VOIP/Cisco UC • Media Player applications • Telemetry data • Storage Design Impact • Are transactions serial? Consider sequential, in- order-block storage solutions • QoS and CoS for storage applications

• Common Usage • Internal to servers, bus-based architectures • Recent emphasis on PCIe-based storage solutions • All-Block solutions • Can build SDS and Hyperconvergence solutions with this foundation • Risk • Adding layers of software and abstraction changes the fundamental architecture • Trying to extend internal storage solutions (e.g., PCIe extensions) can have unintended consequences

• Common Usage • Connecting hosts with storage targets inside a Data Centre • Different connecting strategies emphasise or de-emphasise storage • Deterministic vs. Non-deterministic storage approaches • Block, File (and some Object) solutions • Some solutions trade off scale and performance for manageability • Risks • Assuming that “all storage is equal” and is just an I/O problem • Not understanding that “using Ethernet for storage” covers a lot of ground with massive margin for error

• Common Usage • Backup, disaster recovery, infrequent access to • Block, File and Object usage • Block storage is only used for backup - never connect a host to a block storage device across a wide-area network! • File and object storage is used for backup, but also for low-frequency, small size storage access • Risks • Expecting block storage to work the same over distances • Placing the wrong workload on long-distance links

• Common Usage

• Data replication services, Backup and Recovery, eventually consistent data

• Object Usage Only

• Risks

• Workload/Distance mismatch

• Workload transaction requirements

AWS Google Cloud Cloud Azure

• Common Usage • Varied: Workloads completed “in the cloud” use a SLA-model, and choose accordingly • Can be Block, File, or Object • But distributed to client machines via File or Object • Risks • Choosing the wrong storage type for your workload can be fundamentally hazardous to your employment future • Costs can skyrocket

• Need to focus on not just the “sweet PCIeNVMe/SCSI/SATA/SAS No Network spots,” but also the overlapping areas Danger L4-7: Object SDS Hyperconvergence

• Future presentations will discuss the L3-4: iSCSI NFS Local Network specifics of each layer Fibre Channel SMB NVMe-oF (FC-NVMe) NVMe-oF (iWARP) L2: FCoE NVMe-oF (RoCE)

Danger

Object FCIP iSCSI External, Wide-Area Network NFS SMB Danger

Object Global Network

Danger

AWS Google Cloud Cloud Azure

• Three types of storage access for servers

• Block

• File

• Object

• Each have distinct characteristics to relationships with hosts

• Distinct advantages/disadvantages

• These are not the same thing as File Systems!

• The unit in which data is stored and retrieved on disk and tape devices; the atomic unit of stored data

• What can you do with it? Application • Boot servers/VMs • Works very well with databases and transactions

• Pros File System • Host system has direct access to storage memory (drives, disk, NVM) • Highest performance capabilities Volume Mgmt • Cons • Heavy reliance on HA redundancy at every level of the architecture Physical Media

• Inside a Server

• CPU communicates with a File System in order to control a Volume

• File System = how to find where to read/write data

• Volume = where the data actually exists

• Key point about Block Storage: Host owns volume!

*Not To Scale

• Outside a Server

• CPU communicates with a File System in order to control a Volume

• Host has a network card to communicate with remote block storage

• Need High Availability (HA) in the network connections

• Key point about Block Storage: Host still owns volume!

*Not To Scale

• An abstract data object made up of (a.) an ordered sequence of data bytes stored on a disk or tape, (b.) a symbolic name by which the object can be uniquely identified, and (c.) a set of properties, such as ownership and access permissions that allow the object to be managed by a file system or backup manager

Source: SNIA

• Foundation for Network Attached Storage (NAS) • NFS, SMB • CIFS (deprecated) • Enterprise • Purpose-built, structured and unstructured data over one or more protocols • Scalable and higher performance • Supports large number of clients • Features: tiering, caching, de-duplication, multi-tenancy, replication, multi-protocol support, etc. • Suited for large data sets, large number of clients • Clustered NAS (Scale-up or scale-out) • Petabyte scale, 1000s of drives

Source: SNIA http://www.snia.org/sites/default/education/tutorials/2012/spring/file/AnjanDave_Understanding_Enterprise_NAS.pdf

• Outside a Server • Host has local (block storage), but connects to remote storage to access files • File system “sees” both storage devices as a single directory structure

*Not To Scale

• The encapsulation of data and associated metadata

BRKINI-1011 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 54 Object Storage – What is an Object? System Metadata File + Metadata = Object Name, owner, size Creation date/time, last modified Permissions Custom Metadata

Custom Metadata System Metadata Genre, Author/Composer Location, Security Context – HIPPA, PCI Sharing – user access list File Deletion date, etc.

• Files are tracked by a Metadata Server Cluster

• Mechanisms vary, but hosts will talk to the metadata server to find out where a file is, and then access that location

• Can be deterministic (Global Namespace mapping)

• Can be non-deterministic (address indices managed by proxy nodes or monitors)

*Not To Scale

• Where does Software-Defined Storage and Hyperconvergence fit into all this?

• These are architectures and implementation that use block, file, and/or object strategies

• The programming that controls the storage is decoupled from the physical hardware • Consists of Storage Tier Only • Generally speaking… • Shared pool of storage • Devices are “dumb” and “cheap” (e.g., “commodity hardware”) • Architecture is defined by software control/management plane • Data plane can be block, file, or object storage • Emphasises storage services such as deduplication or replication • Magic Sauce: is control plane management

• New converged infrastructure offering that uses SDS

• Tight Integration of x86 servers for compute and storage, networking and virtualisation, in an all-in-one appliance

• Integration of hypervisors and physical infrastructure

• Is it Block? File? Can vary depending upon vendor

• Storage is presented via a distributed filesystem or object store

• Block: Almost always iSCSI; File: Almost always NFS

• Has an abstraction layer for control plane management

• Magic Sauce: Each node can talk to each other node, centralised management, intuitive UI

BRKINI-1011 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 60 Block Storage: Fibre Channel, FCoE, iSCSI, NVMe over Fabrics Fibre Channel and FCoE What is Fibre • Deterministic network for block storage Channel? • Carries SCSI, NVMe, and FICON traffic • Considered “Gold Standard” of storage for performance, reliability, and availability • Lossless network with high degree of resiliency

BRKINI-1011 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 67 • Deterministic network for block storage • Uses same Fibre Channel protocol and traffic What is Fibre types • Lossless network with high degree of resiliency Channel over • All of the above, but run over lossless L2 Ethernet? Ethernet

• Many devices • FC switch opens up the access • Multiple servers can use the array • Dedicates storage space LUN • Intelligent Storage Array • Single point for storage provisioning • Large enough for multiple servers • Single spot for data integrity • Logical drives • Multiple disks make up a logical drive • Performance enhancement FC SAN • Redundancy, data protection

• Purpose-built for storage traffic

• Run NVMe and SCSI side-by-side

• Built-in discovery protocols and fabric tools

• Built-in security (e.g., Zoning)

• End-to-end qualification and support

• Unique skill sets

• (Often) Separate network infrastructures, budgeting, procurement

• Pre-planned environment is required

• Additional/separate management environment, tools

• Introduction to Fibre Channel • http://fibrechannel.org/fibre-channel-101/ • Introducing FC-NVMe • http://fibrechannel.org/fibre-channel-nvme-got-questions/ • Deep-Dive into Advanced FC-NVMe • http://fibrechannel.org/take-a-dive-deep-into-nvme-over-fibre-channel-fc-nvme/ • Fibre Channel Solutions Guide • http://fibrechannel.org/fibre-channel-solution-guide-2017/ • Cisco MDS Products • Available on Ciscolive.com

 Encapsulation of SCSI command descriptor blocks and data in TCP/IP byte streams

 Works on any Ethernet switch

 Zoning not required

 Works on TCP and subject to losses in network

 Well suited for applications with less I/O requirements while reducing the TCO

BRKINI-1011 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 60 iSCSI-Enabled Hosts (Initiators) iSCSI iSCSI Storage Network iSCSI iSCSI • Storage is accessed at a block-level via iSCSI iSCSI Appliance iSCSI • Good performance via standard (Target) Ethernet NIC IP • Enhanced performance with TCP Network Offload Engine (TOE)

• Lower TCO relative to direct Fibre iSCSI Gateway Channel HBA/Fabric... to a point FC HBA • Standards-based FC Attached Fabric Host (Initiator) Storage Pool (Target)

• Low cost alternative to Fibre Channel-based SAN • Cheap initiators (Software-based initiation available) • iSCSI storage arrays often cost less than FC arrays • Smaller deployments can be deployed very quickly • No special training/skills needed to implement and manage the technology

• Easier programmability • Example: Basis for most of the OpenStack® block storage (Cinder) work

• Use existing high speed LAN infrastructure

• Can be used with vSphere VMFS volumes

• Speed and performance greatly increased with 10 Gbps Ethernet

• Supports authentication (CHAP) and encryption for security, as well as multipathing for increased throughput and reliability

• Because iSCSI is most commonly deployed as a software protocol, it adds to CPU overhead vs. using hardware-based initiators • Reduces number of VMs-per-physical host • Performance is typically less than, or more unpredictable than, that of FC SANs • Typically doesn’t scale as high as FC storage systems • Network latency and non-iSCSI network traffic can diminish performance • Troubleshooting large systems often gets customers into trouble (e.g., QoS, Jumbo Frames)

• Everything You Wanted To Know About Storage But Were Too Proud To Ask: Part Rosé (iSCSI Pod)

• https://www.brighttalk.com/webcast/663/244049

• Evolution of iSCSI

• https://www.brighttalk.com/webcast/663/197361

• The Napkin Dialogues: Lossless iSCSI

• https://blogs.cisco.com/datacenter/the-napkin-dialogues- lossless-iscsi

• Non-Volatile Memory Express (NVMe) • Began as an industry standard solution for efficient PCIe attached non-volatile memory storage (e.g., NVMe PCIe SSDs) • Low-latency and high-IOPS direct-attached NVM storage • NVMe over Fabrics (NVMe-oF) • Built on common NVMe architecture with additional definitions to support message- based NVMe operations • Standardisation of NVMe over a range Fabric types • Initial fabrics; RDMA(RoCE, iWARP, InfiniBand™) and Fibre Channel

• NVMe is a Memory-Mapped, PCIe Model

• Fabrics is a message-based transport; no shared memory

• Fibre Channel uses capsules for both Data and Commands

• NVMe Multi-Queue Interface Model • Single administrative and multiple I/O queues • Host sends NVMe commands over the Submission Queue (SQ) • Controller sends NVMe completions over a paired Completion Queue (CQ) • Transport type-dependent interfaces facilitate the queue operations and NVMe Command Data Transfers

• Architectural Elements Implementation-Dependent Elements • NVM Media and Interface • NVMe Controllers

• NVMe Namespaces

• Fabric Ports

• Subsystem ports are associated with physical Fabric ports

• Multiple NVMe Controllers may be accessed through a single port

• NVMe Controllers are associated with one port

• Fabric Types: PCIe, RDMA (Ethernet RoCE/iWARP, InfiniBand™), Fibre Channel/FCoE

• TCP/IP soon

• nvmexpress.org • NVMe for Absolute Beginners • https://blogs.cisco.com/datacenter/nvme-for-absolute-beginners • Learning NVMe: A Program of Study • https://jmetz.com/2016/08/learning-nvme-a-program-of-study/ • A NVMe Bibliography • https://jmetz.com/2016/08/a-nvme-bibliography/ • Fibre Channel and NVMe • https://jmetz.com/2016/09/fibre-channel-and-nvme-over-fabrics/ • Deep Dive into NVMe over Fibre Channel (FC-NVMe) • https://www.brighttalk.com/webcast/679/265459/dive-deep-into-nvme-over-fibre-channel-fc-nvme • Webinars • The Evolution and Future of NVMe • https://www.brighttalk.com/webcast/12367/290529 • Under the Hood with NVMe over Fabrics • https://www.brighttalk.com/webcast/663/175515

BRKINI-1011 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 79 File Storage: NFS and SMB What is NFS? NFS is a dominant protocol for accessing Network Attached Storage that is File- based (directories, files) vs volume-based (LUN)

BRKINI-1011 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 81 SMB is a dominant protocol for accessing What is SMB? Microsoft-based Network Attached Storage that is Filesystem-based (directories, files) vs volume-based (LUN) SMB Direct is a dominant protocol for accessing Microsoft-based Network Attached Storage that is RDMA-based

More info: Rockin’ and Rollin’ with SMB3 https://www.brighttalk.com/webcast/663/244843 BRKINI-1011 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 82 NFS Network File System (NFS): Overview

• NFS is a distributed filesystem that allows users to access remote filesystems as if they were local • NFS uses a client-server model, in which a server makes directories on its storage accessible to one or more clients; and clients mount the directories to access the files in them • NFS uses RPC (remote procedure calls) to do its work, and TCP/IP as a transport protocol • Hierarchical in nature, where directories (a special type of file) can contain further directories and files. NFS (unlike SMB which is a proprietary protocol) is an industry standard, defined by the IETF

• Workloads • Tend to be shared home directories, data for applications that might be run locally but can run over a network (SQL databases like Oracle for example) and VM datastores • Popularity • NFS and SMB are difficult to size in terms of popularity, but our figures show approx. 50% of the world uses NFS, and the other 50% SMB • VMware support for NFSv4.1 as a client for storing VMDKs • Amazon support for NFSv4.0 in AWS Elastic File System (EFS)

• NFS server directories are exported for use on client systems as a pseudo File System

• Server’s directories are disjointed

• When exporting, server creates a unique file system id (fsid)

• Allows client to access directories from single common root

• Prevents visibility into inaccessible directories

• Advance in Clustering and Massive Scale-Out Storage Reference Slide Access • Described in IETF RFC 5661 • Aggregation of standalone NFS servers, relieves performance issues that are associated with point-to-point connections. • Depends on client understanding how a clustered file system stripes and manages data • No standard storage access protocol; pNFS could be used instead • Flexible, per-file striping patterns • Application SLAs and management policies as well as dynamic load balancing and tiering decisions require per-file control over striping

• NFSv4.1 (pNFS) can aggregate bandwidth Reference Slide NFSv4.1 Client (s) • Modern approach; relieves issues associated with point-to- point connections

pNFS Storage-access protocol protocol

Metadata Server Control protocol Data Servers

Source: SNIA

• SSC permits direct communication with other Reference Slide server to copy data without client involvement

• Removes client bandwidth limitation and possible congestion

• CLONE (internal to server)

• Currently client copies out data and then writes back to the server

• CLONE copies are carried out directly on the server

• COPY (server to server)

• IETF Spec • http://datatracker.ietf.org/wg/nfsv4

• SNIA White Papers • Overview of NFSv4 • http://www.snia.org/sites/default/files/SNIA_An_Overview_of_NFSv4-3_0.pdf • Migrating to NFSv4 • http://www.snia.org/sites/default/files/Migrating_to_NFSv4_v04_-Final.pdf

• Webinars • BrightTalk • https://www.brighttalk.com/search?duration=0..&keywords[]=nfs&q=snia&rank=webcast_relevance • What is NFS • https://www.brighttalk.com/webcast/663/191035 • What’s New with NFS 4.2 • https://www.brighttalk.com/webcast/663/153259

• Overview • Standard way Windows systems share files and folders. SMB 3.0 is the latest version, starting with Windows Server 2012 • Highly optimised for server applications • Characteristics • Speed - designed to be more useful in clusters and wide- area networks • Fault-Tolerance - made to be more resilient to outages • Security - improved encryption capabilities • Workloads • Deep integration with Microsoft Hyper-V and Windows Server Products • Application-specific capabilities for Microsoft SQL Server • Popularity • About 50% adoption

Image source: https://www.samba.org/cifs/docs/what-is-smb.html

Reference Slide • CIFS means SMB as it existed in Windows NT 4 (mid-1990’s!)

• However, the term “CIFS” is sometimes used incorrectly to refer to more recent versions of SMB, like SMB2 or SMB3

• ‘CIFS’ is sometimes used as a marketing term to identify specific products, independent of the SMB version

• Using the term ‘CIFS’ to refer to SMB 2.0 or SMB 3.0 is like...

• Using POP to refer to IMAP (in e-mail protocols)

• Using WEP to refer to WPA (in wireless security)

• Using NFS to refer to NFSv4

• If it says ‘CIFS’ on the box, you don’t know what you’ll get.

• Always look for the full protocol version!

• SNIA

• http://sniaesfblog.org/index.php?s=SMB

• Jose Barreto’s Blog

• https://blogs.technet.microsoft.com/josebda/tag/smb/

• More info: Rockin’ and Rollin’ with SMB3

• https://www.brighttalk.com/webcast/663/244843

• Imagine a grocery store with no labels on • Metadata is the information on the label of any of the cans the cans

• Over time, metadata can be more important than the data itself

Concept borrowed shamelessly from Jeff Lundberg, HDS “The Fundamentals of Object Storage, Part 1”, because it was so brilliant.

• Enables the user to find data based upon Regular Expressions

• Allows you to treat the Cloud not as a large “Object Store” but as a database

• As the size of the Cloud grows, so does your ability to find data

• The better your metadata is, the better your queries can be

• Examples of using complex queries:

• Find objects of a certain age and containing specific meta-data

• Find objects belonging to a person or application that can be removed but including other criteria for exclusion

• Find similar objects and classify them

• Recall: SCSI doesn’t have the ability to “find objects”

BRKINI-1011 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 99 Storing and Retrieving Objects * Object = http://www.example.com/photos/chambers.jpg • Server writes the object to a single node in the cluster, selected via a topology aware algorithm • Algorithms can be optimised to take into account storage node utilisation, performance or other attributes • Based upon a object based policy, the cluster can then perform additional actions on the object • Replicate it to additional nodes or sites, synchronously or asynchronously Rack1 Rack2 Rack3 Rack4 Rack5 Rack6 • Clients can have objects on any or all the storage nodes as per user-defined policies • Object’s replicas, once written are referred to equally, they are not primary, secondary, tertiary • Unique object names enables global access

• Given that the object is accessible via a URL, I can… • Object = http://seth.ceph.cisco.com/photos/cha mbers.jpg • Access it from anywhere • Replicate the object closer to the consuming application • Redirect the application to the closest copy • Prevent access to it from outside of a geographic region

• OSDs: Object Storage Devices • Roughly corresponds to physical disk • Redundancies of OSDs, no need to RAID • Pools • Contain “placement groups” for data • Can be grouped by performance • CRUSH Maps • Identify distribution of objects into OSDs. • Specified on per-pool basis • RADOS • Reliable Autonomic Distributed Object Store • At a practical level this translates to Storing opaque blobs of data (objects) in high performance shared storage • RDB • RADOS Block Device • Striped over Placement Groups, resizable

• Keep at least 3 copies of each file as far away from each other as possible • No RAID • Two types of Swift servers • Proxy server • Maintains mapping of where data resides • Data server • Contains disks, watches for corruption, copies data if disk dies • No central database, computes hash ring to where file should be • Then asks data servers if they have the file • Can be further subdivided into zones • Can be aggregated into regions for globally distributed data sets

Image Source: Seagate https://developers.seagate.com/display/KV/OpenStack+Swift

• S3 = “Simple Storage Service” • Commonly used for backups, archiving, and Web content distribution • “Black box” implementation for customers; details not made public • “Buckets” • User-defined spaces that contain objects • Common uses • Dropbox • Bitcasa • Tumblr • Formspring • Pinterest • Minecraft

• SNIA

• Object Storage 101: The What, How, and Why

• https://www.brighttalk.com/webcast/663/110683

• Object Storage 201 - Understanding Architectural Trade-Offs

• https://www.brighttalk.com/webcast/663/124583

• Visions for Ethernet Connected Drives

• https://www.brighttalk.com/webcast/663/146467

• Things I’ve Written: jmetz.com/bibliography • External Sources • SNIA’s “Everything You Wanted To Know About Storage But Were Too Proud To Ask” series • http://sniaesfblog.org/?p=551 • Fibre Channel • http://fibrechannel.org • NVM Express • http://nvmexpress.org • Storage Performance Benchmarking Series • Part 1: Introduction and Fundamentals: https://www.brighttalk.com/webcast/663/164323/storage-performancebenchmarking-introduction-and-fundamentals • Part 2: Solutions Under Test. https://www.brighttalk.com/webcast/663/164323/storage-performance-benchmarking- introduction-and-fundamentals • Part 3: Block Components. https://www.brighttalk.com/webcast/663/189797 • Part 4: File Components. https://www.brighttalk.com/webcast/663/219127 • Part 5: Workloads. https://www.brighttalk.com/webcast/663/297859

• Understanding the reasons why you want to put a storage solution in place is far more important than how the parts connect together • First things first

• Understanding the strengths/weaknesses of each “layer” will help prevent massive failures in the danger zone • Understanding the combination of technology “sweet spots” and usage requirements can make you a hero • More to come!

• Give us your feedback and receive a Cisco Live 2018 Cap by completing the overall event evaluation and 5 session evaluations. • All evaluations can be completed via the Cisco Live Mobile App.

Don’t forget: Cisco Live sessions will be available for viewing on demand after the event at www.CiscoLive.com/Global.