Nvme Over Fabrics

Why NVMe Over Fibre Channel should be on your Radar!

Tim Jeka FAE [email protected] 414-322-1053

Accelerate with IBM Storage Webinars

The Free IBM Storage Technical Webinar Series Continues in 2018...

Washington Systems Center – Storage experts cover a variety of technical topics.

Audience: Clients who have or are considering acquiring IBM Storage solutions. Business Partners and IBMers are also welcome.

To automatically receive announcements of upcoming Accelerate with IBM Storage webinars, Clients, Business Partners and IBMers are welcome to send an email request to accelerate- join@hursley.ibm.com.

Located in the Accelerate with IBM Storage Blog: https://www.ibm.com/developerworks/mydeveloperworks/blogs/accelerate/?lang=en

Also, check out the WSC YouTube Channel here: https://www.youtube.com/playlist?list=PLSdmGMn4Aud-gKUBCR8K0kscCiF6E6ZYD&disable_polymer=true 2018 Webinars: June 21 – Deep Dive into Spectrum Scale AFM January 9 – DS8880 Easy Tier June 28 - Open System Tape, What’s New January 17 – Start 2018 Fast! What's New for Spectrum Scale V5 and ESS July 19 - Copy Services Manager Version 6.2.2 Technical Update February 8 - VersaStack - Solutions For Fast Deployments August 2 - Back to Basics - TS7700 Concepts and Operations February 16 - TS7700 R4.1 Phase 2 GUI with Live Demo August 7 - IBM/Broadcom: Why NVMe-Over Fiber should be on your February 22 - DS8880 Transparent Cloud Tiering Live Demo radar! March 7 - Spectrum Storage Management, Control, Insights, Foundation; what’s the difference? March 15 - IBM FlashSystem A9000/R and SVC Configuration Best August 30 - Transforming Traditional IT to a Multi-Cloud strategy Practices leveraging IBM Storage - Register Here: March 27 - IBM FlashSystem A9000/R Technical Update https://ibm2.webex.com/ibm2/onstage/g.php?MTID=ed9ba6106502f2abdb1 April 12 - Introducing Spectrum NAS - The Newest Member in the Spectrum 531521d48c7c20 Storage Family April 26 - TS7700 Grid Configuration Changes -- Joins, Merges and Removals May 8 - DS8880 Technical Update June 7 – Economic Value of Response Time © Copyright IBM Corporation 2018. Washington Systems Center - Storage

Today’s Topics

• The Modern SAN

• The Rise of NVMe

• Why Fibre Channel is the Best Fabric for NVMe

• The State of NVMe over Fibre Channel

• POWER and NVMe-OF

• IBM Storage and NVMe-OF

• SAN Health 4.1

• Wrap up and Q & A

A Seismic Shift is Underway in Enterprise Datacenters

• 96% growth in All-flash Arrays, NVMe emerging

– 100x faster than HDD – 1M+ IOPs/SSD NVMe – Bottleneck shifts to the network

– 90% attach rate to FC All-flash arrays Performance

All Flash Arrays PC Client NVMe Server SSD NVMe JBOFs (Enterprise Class) Laptop and client SSDs JBOFs are a new class of are expected to drive Server class NVMe is low latency NVMe scale As NVMe economics volume and NVMe prices emerging for low latency out storage make sense, Enterprise down workloads Class NVMe arrays will become standard

Today © Copyright IBM Corporation 2018. 5 Washington Systems Center - Storage

Digital Era Drives Storage Modernization

• NVMe will become the de facto standard!

Flash NVMe Complexity 87% #1 68% of storage arrays storage protocol say IT is more complex will be flash by in the future than 2 years ago 2020

Modernize your Storage Network

The Future of Storage Protocols White Paper * Gartner, Valdis Filks, Stanley Zaffos, 29 June 2017

“Fibre Channel will remain the “Storage networking “IT leaders planning to upgrade data center protocol of choice investments are becoming a their storage networking need to for the next decade as critical top priority due to use 16Gbps FC today with a performance bottlenecks the adoption of high- migration plan for moving to move out of arrays and into throughput, solid-state, and 32Gbps FC” the storage network” flash storage”

Modernize all aspects of your Data Center

COMPUTE STORAGE IOPs, Latency, Bandwidth.. Oh my!

Flash NVMe What is missing? The Data Path… Disk … is critical to ensure: Performance Security Scalability Visibility IBM 9150 Scales to 10M IOPs

Modernize all aspects of your Data Center

COMPUTE END-to-End Solutions STORAGE FIBRE CHANNEL is a network built for Storage! Flash NVMe

Disk

IBM 9150 Scales to b-type Storage Networking 10M IOPs DATA: Connect IT. Protect IT. Own IT.

Why is All-Flash Array Growth Such A Game Changer? • Why 32Gb is important? Relative performance 32GB is ready for all Flash/NVMe Arrays 8/16/32/128 FC Servers

SSD NVMe Works in Nanoseconds Works in Microseconds Flash in Milliseconds Some servers can do more than 1.6 million transactions per second each Features with many I/Os per transactions 16GB IOPs 10s of Millions Flash run at about 150 IOPS Servers Adapters 16GB delivers 450 Million or about 6,000 IOPS per Tray Frames per second! SSD/Flash run 100K - 4 million IOPS Intel FC SFP+ 1.6 Million IOPs Latency 700NS 32GB IOPs 100s of Millions IOPs will only Increase! POWER 8 FC 780,000 IOPs 32GB delivers 1 Billion Frames/second Latency 780NS Z series 16S+ 320,000 IOPs NVMe Spectrum Scale Testing 15M NVMe Ready IOPS at a latency of around seven POWER9 FC 32GB 10M IOPs microseconds

10,000,000 IOPs….. Really? © Copyright IBM Corporation 2018. IBM FS9150 Scales 10M IOPs 10 Washington Systems Center - Storage

Fibre Channel the Next Generation

• GEN7 which doubles the speeds yet again, is already on the horizon!

• Serial and Parallel Link Speeds… FC 32Gb, 16Gb, 8Gb

Generation 6th Gen 7th Gen 8th Gen

Electrical / Optical Module 32GFC and 64GFC and 128GFC and 128GFCp /SFP+ 256GFCp /SFP+ 512GFCp /SFP+ and QSFP28 and QSFP56 and QSFP112 Electrical Speeds (Gbps) 1 lane of 28.05 1 lanes of 56.1 1 lane of 112.2 4 lanes at 28.05 4 lanes at 56.1 4 lanes at 112.2

SFP+

QSFP28 QSFP56 QSFP112

IT Infrastructure Modernization is Strengthened when Attached with Gen 6 Networking

FC SANs are made for Storage!

Predictable Massive Deep Greater Integrated Performance Scalability Visibility Availability Automation Security

Real-time Insight Rapid Application Granular Application Always-on Self Learning Real-time Ultra-low Latency Delivery Insights & Actions Services Self Healing Encryption

2x 12TB/ 48% 52sec 50% 50x more per fabric reduction in risk of downtime/year fewer faster disaster transactions with for 10s of thousands of maintenance to maintain five-9s of I/O and storage recovery and 2 4 same storage workloads on costs availability expectations network issues business 3 5 compared to scores of arrays1 (vs 95 min average ) continuity 16 Gbps data rate

Exploring NVMe over Fibre Channel

• NVMe over Fibre Channel is relatively new, even thought its component parts are not!

Gen 5 and Gen 6 support FC-NVMe • Fibre Channel has been the leading enterprise storage technology since the mid-1990’s • Speeds of 16Gbps (GEN 5) are widespread. • Gen 6 (2016), delivering twice the speed of GEN 5 selling like hotcakes… • GEN 7, which will double the speeds yet again, is already on the horizon • Fibre Channel does not have a RDMA protocol so FC-NVMe uses FCP for data transfers. • You can deploy NVMe over Fibre Channel on an existing Broadcom FC infrastructure, providing it is relatively up to date! • NVMe-FC is aimed at 16GB, 32GB, and higher speed switches and fabrics

IBM Systems Flash Storage Offerings Portfolio

IBM Power Systems New! Storwize FlashSystem 9100 FlashSystem FlashSystem IBM Elastic DS888xF V5030F / V7000F Models 9110, 9150 A9000 A9000R Storage Server A9P 550K IOPS 9110 225K IOPS A9R 1.3M IOPS 9150 425K IOPS SCALES to 2.5M IOPs Clustered System 9110 900K IOPS 3M IOPS V5030F 130K IOPS 9150 1.7M IOPS V7000F 220K IOPS SCALES to 10M IOPS Enterprise Class, Cloud Service High End Entry / Mid-Range NVMe accelerated Providers Enterprise Multi-Cloud Enabled Big Data Business Critical Scale-out clustering Consolidate file & Simplified management z/OS / AIX Simplified management object workloads Flexible consumption model Power HA Flexible consumption model Faster data analysis Large Grid scale Power i HA Virtualized, enterprise-class, flash-optimized, modular storage Global sharing Full time data reduction Enterprise class heterogeneous data services and selectable data reduction Business critical, deepest integration with z Systems IBM FlashCore™ Technology Optimized Superior performance SVC NVMe FlashCore Module FlashSystem 900 and reliability Superior endurance & better performance Application acceleration • Extreme performance Three-site / Four-site • Hardware Compression Enhanced data storage functions, • FIPS 140-2 replication 900k IOPS • Targeting database economics and flexibility with • Hardware Compression acceleration DS8884F, DS8886F, sophisticated virtualization DS8888F

Flash Technology, Network Design Considerations

• Does the Network meet the Bandwidth and Latency Requirements? • Does the network have Multipath capabilities to provide active- active designs, to enable 100% utilization of all links? • Can the Network handle the dramatic increase in IOP throughput? 32 Gbps HBAs • Does the Network enable availability for high-density and critical Edge Connections workloads? • Storage extension capabilities?

• In-depth visibility to enhance the Flash infrastructure? 32 Gbps SAN Core Fibre Channel • NVMe-oF Ready? Fabric • Like it or not! IT Planners have to evaluate the SAN!

Fibre Channel Flash Storage

FOS v8.2 Summary…. IBM …Not Announced

New Gen 6 Hardware • New ASIC – Condor 4+ - Support for I/O and VM Insight - Supports I/O Insight w/NVMe over Fabrics Gen6 64-port Blade - Supports connectivity to FCoE Severs 64-port Gen6 Blade 128-port Gen6 Enterprise Switch Gen6 G630 128-port FOS v8.2 Enterprise Switch Platform support….. Customer Focus ‒ Support existing Gen 6 and Gen 5 platforms • Ending support FC8-32E, FC8-48E, FC8-64, FX8-24 • FCoE/Cisco UCS support • NVMe Analytics support • FOS RESTful API – PyFOS, Ansible Provisioning • In-flight encryption support for Brocade Switches G620 & G630

Brocade Automation Is Coming…. FOS8.2

• Brocade is starting to close the gap with SAN automation Inventory Monitoring Provisioning

• Configuration to XLS • Port Statistics • Switch • Compare to SAN • Port Errors • Port Health • Violations • Zoning • Reporting • Maps • SAN Extension • Workflow REST API PyFOS Ansible

What is the Fabric Vision technology functionality for NVMe over Fibre Channel?

What tools are used to MONITOR and TROUBLESHOOT IO issues?

MAPS Dashboards IO Insight Automation that VM Insight Automatically detect Customizable Fabric Performance simplifies policy- degraded storage health and Impact Monitoring Seamlessly monitors IO performance based monitoring performance VM performance with integrated device and alerting dashboard, with all Quickly detects and throughout a storage latency and IOPS critical information clearly alerts fabric with standards- monitoring on one screen admins to high based, end-to-end VM levels of latency, tagging helping to identify slow drain devices Fabric Vision

Flow Vision Forward Error Identifies, monitors, Correction and analyzes the Automatically detects performance of and recovers from bit specific flows or frame errors, negating the types need for retransmission COMPASS Buffer Credit ClearLink Simplifies SAN Recovery Diagnostics configuration Automatically recovers Cable and optic and maintains flow control buffer diagnostics that simplify consistency in credit loss at the VC the deployment and & operational behavior level, improving support of large fabrics availability

NVMe IO Insight Capabilities

• Monitor storage IO metrics to gain insights into problems • FOS 8.2 Delivers I/O Insight for NVMe Traffic!

Non-disruptively monitor application flows of interest and obtain device-level IO insight Proactive IO monitoring: – Read/Write statistics – IO per second – Avg/Max IO – Total IOs • Latency metrics – First Response Time – Exchange Completion Time – Pending IO Time No taps, no downtime, no disruption

Acronym Definition NVMe Drivers

Acronym Definition NVMe Drivers FC-NVMe Drivers will be provided by Fibre Channel Vendors

Acronym Definition NVMe Drivers FC-NVMe Drivers will be provided by Fibre Channel Vendors NVMe Subsystem

Acronym Definition NVMe Drivers FC-NVMe Drivers will be provided by Fibre Channel Vendors NVMe Subsystem Non-volatile memory storage device

Acronym Definition NVMe Drivers FC-NVMe Drivers will be provided by Fibre Channel Vendors NVMe Subsystem Non-volatile memory storage device NVMe Controller

Acronym Definition NVMe Drivers FC-NVMe Drivers will be provided by Fibre Channel Vendors NVMe Subsystem Non-volatile memory storage device NVMe Controller A type of controller which supports minimal functionality for discovery of NVMe media controllers NVMe Namespaces Similar to SCSI’s LUN (Logical Unit Number) and Media identifier. The NSID is a set of logical block addresses (LBA) on the NVM media

• NVMe is a communications interface and protocol • NVM Express defines an efficient interface for host software to communicate with a non-volatile memory subsystem over PCI Express (NVMe over PCIe) • Functionally analogous to SAS and SATA but designed to reduce overhead from drivers, OS and application

• NVMe uses PCIe fabric • Multiple devices today in the market • Multiple form factors including 2.5” drives

• NVMe has been designed for high performance • Increased IOPs, bandwidth and lower latency • Exploit Flash and next-generation Non-Volatile Memories • Leverage multicore environments, high I/O parallelism

• Benefit: Lowest Latency for Flash, with higher performance coming …

NVMe and the Data Center

• Building for the Next Generation Flash Networks

Latency Lower TCO

• No translations between protocols • Native protocol designed for flash • Native for NAND • Increased density • Lowest in industry • Lower system power

Scale Fabrics

• Massive parallel I/O capabilities • Can be transported on different • Broad industry support including, interconnects for best performance HW, OS and NAND vendors • FC is the ideal choice (80% of networked flash is deployed on FC)

Where does NVMe performance come from?

SCSI NVMe

CPU CPU CPU CPU NVMe has a greatly reduced and simpler instruction set 34 basic SCSI commands map to just 15 NVMe commands Single queue 64 Commands

NVMe-oF (FC-NVMe)

• Next wave of technology disruption for storage

• NVMe over FC (FC-NVMe)

− Very low latency sub 20 us

− Removes SCSI and SAS/SATA from the IO stack • Shared storage requirement for the enterprise • Ultra High performance/low latency requirements • Expect end to end solutions to begin shipping in 2018 • Broadcom aligning fabric capabilities to align with this technology • FC-NVMe will run on existing Gen 5/6 fabrics • Gen6 for Analytics • NVMe & SCSI coexist in the same server and on the same FC SAN • Low risk based on proven storage networks

NVMe Over Fibre Channel

• Same Transport - Just A Different Protocol Protocol Header (FCP, FICON, NVMe, …) • Fabric Services Payload • Flow control (B2B credits) • Discovery Name server • Zoning and Security • FC metrics still in tact • Analytics to examine protocol • Ability to run both SCSI and NVMe side-by- side over fibre channel! • Sounds like a SAN!

NVMe over FC Integrated Network Sensors

End-to-End Monitoring

Storage Server Array

Infrastructure Infrastructure Infrastructure infrastructure Application Application Health Performance Availability utilization Performance baselining

Enterprise NVMe Requires a Network

• Many options, plenty of confusion

• Networked storage is a must for large customers NVMe Server Software needing scalability, availability, and reliability Server Transport Abstraction • Fibre Channel is the transport for the vast majority of today’s all flash arrays. • Ethernet will also be supported, however there are some major

caveats:

Channel

FCoE

iWARP

iNVMe RoCEv2

• FCoE and RoCEv2 require a lossless network. Infiniband Fibre • RoCEv2, iWARP and InfiniBand are RDMA-based but not compatible with each other Storage Transport Abstraction • NVMe over TCP (iNVMe) proposed by Intel, Facebook,

NVMe SSDs

End-To-End NVMe Technologies

• NVMe over Fabrics scales to the Datacenter (Rows of Server Racks)

Servers Storage Fabric All Flash Arrays

iWARP iWARP NVMe SATA Windows Software

Current RoCEv2 RoCEv2 SAS NVMe Flash VMware Software Array

FC FC S FC S C Linux C S S I … I FC NVMe FC Software NVMe NVMe NVMe PCIe Software Optimized Flash Optimized Connection Array Technologies

Let the NVMe-Over Fabrics War Begin There is only one purpose-built fabric for storage: Fibre Channel Fibre Channel Ethernet-based Fabrics “Other” Fabric

Fibre Channel RoCEv2 iWARP Infiniband • Brocade, Qlogic • Mellanox, QLogic, EMC • Intel, Chelsio • Mellanox Emulex • DCB Ethernet • Standard Ethernet • Infiniband fabric • Lower latency • Low latency • Higher latency • Lower latency • Low risk dual proto • IBTA standard • Increasingly niche • Niche FCP & FCNVMe • IO Insight, AMP

NVMe over different Fabrics

• NVMe’s benefit is a simpler stack than SCSI

• NVMe over Fibre Channel extends that simplicity

• But NVMe over RDMA fabrics means complexity

• RDMA was originally defined for InfiniBand

• Making RDMA work over IP adds extra network layers

• Extra network layers complicate the protocol stack

• Using RDMA contradicts the NVMe benefit

• Even though special RNICs or TOEs manage most of the special protocol layers, they still translate to extra work for configuration, management and troubleshooting

NVMe over Fabrics – Enterprise Options Enterprise Protocol Latency Scalable Footprint RoCE Dominant v2 Fibre Channel Lower Yes Storage Fabric RoCEv2 iWARP Low Yes Negligible (Mellanox) Fibre iWARP (Intel) Med Yes Negligible Channel FCoE (Cisco) Low Med Limited Soft NVMe over tcp/ip High Limited None FCoE (iNVMe) – FB iNVMe Infiniband/ IB OmniPath Lowest Limited Almost none (Mellanox/Intel) NVMe Universe

NVMe over Fibre Channel – Today; an evolutionary enterprise transition © Copyright IBM Corporation 2018. 44 Washington Systems Center - Storage

Why Ethernet is More Challenging for NVMe

• Competing incompatible protocols: RoCEv2, iWARP, FCoE, iNVMe

Rip out and Incompatible Lack granular Not built for replace protocols network visibility storage

Net new dedicated No interoperability, Can’t see problems No storage discovery adds network required must choose your when sampling deployment and protocol now limited packets configuration complexity

Ethernet is a general purpose network, not optimized for storage

Investment Protection: No Rip Out and Replace

• Concurrent Traffic Dual Emulex Gen 6 HBA Emulex HBAs by • Run NVMe & SCSI concurrently on same Gen 5 HBAs SAN & HBAs with familiar fabric services. SCSI • The Fibre Channel fabric must be Gen 5 (16 SCSINVMe NVMeSCSI Gbps) or better yet Gen 6 (32 Gbps) • Servers using NVMe over Fibre Channel need Gen 6 (HBAs); Gen 6 HBAs also work with Gen 5 fabrics. 55% • You need a storage device that supports FC- lower NVMe frame types. This can be an FC- latency NVMe-enabled array NVMe over FC, Natively • Existing Gen 5 and 6 SANs can run NVMe SCSI over FC, Translated without code changes or disruptions • Clone & Remap your existing SCSI luns to NVMe Traffic NVMe NSIDs. SCSI Traffic

http://www.nvmexpress.org/wp-content/uploads/NVMe_Over_Fabrics.pdf © Copyright IBM Corporation 2018. 47 Washington Systems Center - Storage Storage Networking must keep pace and ensure Summary – FC-NVMe! Connectivity is not the bottleneck for the data center

FC-NVMe

Increased Virtualization Density Low Latency, High Throughput “NVMe” Over Fibre Channel Non-Volatile Memory “Express”

TransportMore Content NVMe Video, BigNatively Data over Fibre Channel Low Latency

Reliable,Greater OPEX Secure, Efficiency Leverage Existing InvestmentsHigh Speedin Fibre Flash ChannelStorage Available

Industry’s First Technology Demonstration Emulex - Broadcom

Fibre Channel and NVMe –Trusted meets Fast!

Increased Virtualization Density Low Latency, High Throughput “SCSI” Over Fibre Channel Non-Volatile Memory “Express”

#1More in ContentStorage Video, Connectivity Big Data Low Latency, No bottleneck

Reliable,Greater OPEX Secure, Efficiency Available Concurrency,High Speed Efficiency Flash Storage

Billions Invested, SAN&AFA Future of All Flash Storage

FC-NVMe in Action! – No translations

FC NVMe FC SCSI SCSI over FC, Translated

Brocade IBM SAN GEN6 32GB SCSI to NVMe

Direct memory access Direct memory access Into (out of) adapter Transferred over a out of (into) adapter SAN Fabric

FC-NVMe NVMe

Exploiting Dual Protocol FCP and FC-NVMe Best Practices… Dual-protocol infrastructure (Recommended) • Deploy NVMe based arrays Servers – Leverage existing infrastructure SCSI-on-FC – Easily supported dual infrastructure FC-NVMe GEN6 HBAs – How long will the transition take? – Avoid risks • Incremental Migration – Applications dictate how individual volumes can be migrated – Changes can be rolled back easily without disruption to hardware or cabling. DUAL Protocol SAN arrays Existing 32Gb FC fabric Existing enterprise • FCP & NVMe over FC can both leverage FC SCSI-on-FC zoning Storage infrastructure FC-NVME – Only SAN Zoning improves security – Zoning restricts devices from accessing network areas that should not be visiting! • Discovery – Emulex create drivers that leverage FCP for device discovery, then check those devices for SCSI on FC Arrays FC NVMe Arrays FC-NVMe traffic support. Washington Systems Center - Storage

Where to start? The Lab!…. It’s Alive! • Steps to establish a FC-NVMe test lab. Servers SCSI-on-FC FC-NVMe GEN6 HBAs – Set up a single server with an internal drive, single switch, and a single FC-NVMe enabled array

– Connect the server to an IP network for access to other servers.

– Explore the HBA config and storage array options you’ve read so much about! DUAL Protocol SAN arrays Existing 32Gb FC fabric Existing enterprise – Configure a volume to use the NVMe array SCSI-on-FC Storage infrastructure FC-NVME – Copy a file from the internal drive to the NVMe volume and back again.

– Run your performance testing application (Iometer) to benchmark your NVMe volume. Compare it to your internal volume. SCSI on FC Arrays • Lather, rinse, and repeat until your paranoid nature is FC NVMe Arrays satisfied! Washington Systems Center - Storage

Visibility Non-disruptively monitor application flows of interest and obtain device-level IO insight

Emulex HBAs Real-time and historical metrics: NVMe SCSI SCSI

• Read/write I/O latency and first response times (including both average and maximum values) • Read/write IOPS transfer rate • I/O queue depth (pending IOs) Status First Response • All metrics retained based on data size Read Command • 2 year historical metric retention via Brocade Data Network Advisor First Response and Status

NVMe Traffic SCSI Traffic Washington Systems Center - Storage

Benefits of NVMe over Fibre Channel The only purpose-built fabric for storage • Performance and Stability of a Proven Storage Technology Low Latency Extend NVMe Simple Deployment Faster than Networking Beyond single Rack Ethernet

Utilizes existing GEN 6 supports 32Gb Optimized for FC infrastructure and 128Gb today storage workloads concurrently running maximum performance NVMe & SCSI

Purpose Built for Storage; No “Rip-Replace” Deployment; Higher Performance IBM POWER, Storage, NVMe and Broadcom Solution Overview

POWER9 Scale-out Servers…. 10M IOPs?

The most reliable and resilient entry level cloud-enabled servers….. NVMe Ready…announced mid-March S924/H924 2-socket 4U server, is designed to meet highest performance and security with a memory footprint of up to 4TB, an industry leader in the 2 socket market

S914 1-socket 4U server is the price attractive entry offering into the POWER9 family of servers. Industry leading integrated security and reliability as well cloud enabled out of the box with integrated PowerVM technology

S922/H922 2-socket 2U server is designed to meet highest performance and security in a dense form factor with a memory footprint of up to 4TB, an industry leader in the 2 socket market

L922 2-socket 2U server is designed to meet highest performance and security for Linux workloads in a dense form factor with a memory footprint of up to 4TB, an industry leader in the 2 socket market Washington Systems Center - Storage PCIe3 X8 32Gb/s 2-PORT Fibre adapter with POWER8 and POWER9

IBM SAN24B-6 SAN64B-6

IBM Feature Codes : EN1A and EN1B Broadcom’s LPe32002-series SAN256B-6 Scales to 10M IOPs LPe32002 PCIe HBA SAN512B-6 SAN Support 2 Ports 32Gb/s FC using SR optics 32 Gb/s Supported in the following Ports SFP+ Fabric Configurations: LC Type Connections ‘switch’ attach mode… Auto negotiates speeds 32,16,8, Gb/s (Brocade Fabric Only) POWER9 - https://www.ibm.com/support/knowledgecenter/9223-42H/p9hcd/fcen1a.htm https://www.broadcom.com/products/storage/fibre-channel-host-bus-adapters/lpe32002-m2/

POWER8 - RPQ 8A2420 - PCIe3 32Gb/s 2-PORT ADR available 06-29-17 https://www-01.ibm.com/common/ssi/ShowDoc.wss?docURL=/common/ssi/rep_rp/0/ENUS8A2420/index.html&lang=en&request_locale=en https://www.ibm.com/support/knowledgecenter/9223-42H/p9hcd/fcen1a.htm Washington Systems Center - Storage

IBM Power Systems Storage Options with Broadcom

32GB 6-29-17 POWER9 Announced RPQ 8A2420 - PCIeG3 32GB 2PORT LP ADR POWER8 Enterprise Vs Mid View Power System PCIe3 32Gb/s 2-PORT IBM 32Gb SAN Switches Fibre adapter (s) IBM i DS8880 AIX SVC/Storwize Linux IBM SAN24B-6 NPIV Any supported storage VIOS including IBM Flash Systems NPIV FlashSystem V9000 NPIV Benefits of a 32Gb Fibre Channel SAN FlashSystem 900 • High Bandwidth XIV NPIV • Lowest Latency SAN64B-6 NPIV • Highest Performance FlashSystem FS9100 • Scalability Scales to 10M IOPs A9000r • Resource Consolidation SAN256B-6 • Manageability SAN512B-6 • Scales to 10M IOPs 32GB 6-29-17 POWER9 Announced © Copyright IBM Corporation 2018. RPQ 8A2420 - PCIeG3 32GB 2PORT LP ADR POWER8 IBM NVMe Spectrum Announements Currently, NVMe is the new hype that will change the Industry

NEW IBM FlashSystem 9100 • NVMe-Accelerated Enterprise Flash Array – 100% NVMe end-to-end - Industry Leading Performance and Scale - NVMe IBM FlashCore Modules and NVMe industry standard SSD - Agility, Availability and Security - Supports Physical, Virtual and Docker Environments • IBM Storage Insights: AI-empowered predictive analytics, storage resource management, and support platform delivered over the cloud • Includes Spectrum Storage Software for array management, data reuse, modern data protection, disaster recovery, and containerization - IBM Spectrum Virtualize - IBM Spectrum Copy Data Management FlashSystem 9100 - IBM Spectrum Protect Plus - IBM Spectrum Virtualize for Public Cloud - IBM Spectrum Connect

NEW IBM FlashSystem 9100 Multi-Cloud Solutions • Solution blueprints for Data Reuse, Protection and Efficiency • Solution blueprints for Business Continuity and Data Reuse • Solution blueprints for Private Cloud Flexibility and Data Protection © Copyright IBM Corporation 2018 For IBM and IBM Business Partners Only 60 Washington Systems Center - Storage IBM Storage and SDI

1.25M IOPS, 17GB/s per Transformation of • Scale out for up to 10 Million IOPS, rack unit performance and 136GB/sec in 8U capacity density • Up to 8PB of NVMe data storage in 8U IBM FlashSystem • Assumes 5:1 Data reduction 9110 and 9150 • 96 x 19.2TB NVMe FCM for 1.8PB Raw Up to 1PB Designed for NVMe data storage per rack unit Performance and density to Modernize and Transform business efficiency Up to 230TB raw per rack unit

• Software Defined Storage (SDS) NVMe Use Cases • Use NVMe drives as a caching medium, either on the host or in the storage system, to reduce the read latency for cache-friendly workloads • Local read-only cache (LROC) function of IBM Spectrum Scale is already capable of using NVMe drives in this way • Adopt NVMe media • Take advantage of NVMe over Fabrics as a protocol for host attachment • Use NVMe over Fabrics as a protocol for back-end storage access • Similar approach is already used today by IBM Spectrum Accelerate, which uses the user-space I/O paradigm to access back-end storage over RDMA in the FlashSystem A9000 and A9000R

• IBM Systems 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 System x3650 M4

8 16 • Elastic Storage Server (ESS) 9 17 EXP3524 • Built with IBM Spectrum Scale software and will benefit in the same way using NVMe media as a caching medium • FlashSystem 9100 • Integration of NVMe within the controller and NVMe over Fabrics technologies for host to controller connectivity • FlashSystem 900/A9000, Spectrum Virtualize family and DS8000 • Integration of NVMe over Fabrics technologies for host to controller connectivity © Copyright IBM Corporation 2018. 63 Washington Systems Center - Storage IBM Storage & SDI Driven by Data, Unlocked by Software NVMe over Ethernet/FC ready • Upgrade to software with support for NVMe-oF 4Q18 • Configure new NVMe hosts and map volumes • 25GbE iSCSI NIC FlashSystem V9000 to these hosts using w/iWarp RDMA NVMe • 25GbE iSCSI NIC Migrate existing workload w/RoCE RDMA • by reconfiguring hosts so • 16Gb FC/FC-NVMe HBA that existing volumes are Storwize V7000 • Provision storage to hosts now accessed using using SCSI NVMe • Create volumes and map Use same hardware volumes to hosts using • simultaneously for NVMe SCSI and SCSI for a simple SAN Volume Controller migration Washington Systems Center - Storage IBM Storage & SDI Driven by Data, Unlocked by Software Spectrum Accelerate Solutions

• Upgrade to software with support for NVMe- FlashSystem A9000 oF4Q18 • Configure new NVMe hosts and map volumes to these hosts using NVMe Migrate existing workload • 16Gb FC/FC-NVMe HBA • by reconfiguring hosts so that existing volumes are 1.Provision storage to hosts now accessed using NVMe using SCSI Use same hardware 2.Create volumes and map • simultaneously for NVMe volumes to hosts using and SCSI for a simple SCSI migration FlashSystem A9000R

65 Washington Systems Center - Storage

Spectrum Scale and NVMe

• At the Flash Memory Summit (FMS) 2017, it was announced that an NVMe over Fabrics demonstration, that included IBM’s Spectrum Scale, was named the 2017 recipient for “Most Innovative Flash Memory Business Application.” • IBM is a winner in this particular Best of Show category for the second year in a row • The solution can deliver 15 million IOPS at a latency of around seven microseconds • Spectrum Storage has to keep up with the I/O (not capacity), of bleeding edge storage technology (NVMe). Why SAN Health? New San Health 4.1

What is Broadcom’s SAN Health?

SAN Health discovers and provides information about: • Brocade SAN Switches, as well as, legacy m-type SAN Switches • Storage Products (EMC, HDS, HPE, IBM, NetApp, etc.) • Multiple protocols (FCP, FC-NVMe, FICON) • Cisco MDS SAN Switches • HBAs (Emulex, QLogic, etc.)

• Takes only minutes to install and run…. • All components in a NVMe-oF infrastructure! Washington Systems Center - Storage

SAN Health Audit Process….made easier!

Run the Audit Tool Download the report

Upload the Audit Explore the report

Your report is automatically generated and stored into your MyBrocade account for retrieval Washington Systems Center - Storage

Brocade’s SAN Health

SAN Health is a free utility available from www.brocade.com/SANHealth

When to use it • Keeping Accurate SAN records • Pre and Post FLASH engagement • Refresh opportunities • Get Ready for NVMe-oF

SAN Health provides • Check on suitability for FLASH • NVMe-OF 8.2 support SAN Health can be run against • Comprehensive Documentation • Any Brocade B-Series SAN Switch • Performance Graphs • Brocade M-Series SAN Switches • Detailed Topology Diagrams • Cisco MDS SAN Switches • Highlighting of Potential Problems Washington Systems Center - Storage

Roadmap discussion

• SAN Health 4.1 (Available)

• Broadcom Release Cuts users over to the new email addresses, logo etc

• FOS 8.2 Changes Support

• SAN Health 5.0 (Working on it starting in July)

• Relatively simple, quick and free!

• Little effort for a reasonable amount of information in return……. Washington Systems Center - Storage

SAN Health Version 5 New functionalities

• SAN Health moves to a 64 bit windows 10 style app using Visual Studio 2015 and NET46.

• Auto discovers Virtual Fabric Designs

• Version 5 automatically discovers Mutliprotocol Routers and Access Gateways just from connecting from any seed switch in the SAN.

• Dropping support for McData and FOS less than V5. San Health 4.0.7 will continue support for McData equipment.

• SAN Health V5 takes advantage of new multi-threading functionality to eliminate many steps from the audit and make the report more intuitive to the end user.

• SAN Health V5 new performance capture method. Washington Systems Center - Storage

SAN Update

Questions: Email: [email protected] Downloads and more information: www.broadcom.com/sanhealth Uploads [email protected] New Online Help: SAN Health = http://community.broadcom.com/docs/DOC-2662

Are you or your customers among the 48,000 users benefitting from this? 1,800+ reports encompassing 3 million+ switch ports are generated every week! Washington Systems Center - Storage Summary….. FC-NVMe-oF Top 5 Reasons Fibre Channel might be the right Choice!

• Dedicated Storage Network…. SANs are designed for Storage!

• Run NVMe and SCSI side-by-side • Fibre Channel is Robust and Battle Harden for Discovery and Name services • Zoning and security

• Integrated Qualification and Support Sounds like Fibre Channel GEN6 SAN to me…….