ULLtraDIMM™ SSD Overview
Rob Callaghan June 9th, 2014
c 1 A Global Leader in Flash Storage Solutions
Rankings Trailing 4 Qtr Financials* Global Operations Leading Retail Brand°
$6.2B Revenue #1 Global Retail $3.6B Net Cash* Revenue $0.7B R&D Investment 5,500 Employees† Share
SanDisk Client All Leading & Retail SSDs Smartphone Approved & Tablet Supplier to Qualified at Manufacturers All Leading PC 6 of the Top 7 use SanDisk Manufacturers Server & Storage OEMs
Enterprise SSDs and Storage Software
*Financials as of Q4, ‘13. Net Cash = [Cash + cash equivalents + short-term & long-term marketable securities] less [debt at maturity value] as of the end of Q4, ‘13. †Headcount as of Jan., ‘14. NPD Estimate, Nov., ‘13. Estimates of the memory card & USB markets from NPD (Nov. ‘13) and GfK Retail and Technology, Sep., ‘13. 2 Enabling Flash Storage from Wafer to Software
NAND TECH NAND DIE SCALE ASSY, TEST & CONTROLLER FLASH MGMT SSD SOFTWARE PACKAGING
Close to Half of Industry Bit Output World-Leading Innovator + Together with manufacturing 4,900 Patents partner Toshiba Fabs: World class NAND capacity
1991 2013
Patents as of Oct., ‘13; NPD Estimate, Nov., ‘13. Gartner: NAND Flash Supply & Demand, WW 1Q ‘12-4Q ‘14, 3Q ’13. Update Dec., ‘13. 3 The Path to Ultra Low Latency & Scalable Performance
DDR 1’s
10’s PCIe
100’s Latency (µsec) speed memory bus speed memory
1000’s on high the Flash Storage SAS/SATA/FC
1,000,000 100 100,000 IOPS
4 Creating a New Storage Interface
This is a This is DRAM with SATA DIMM battery backup
Flash Flash Flash Flash
Flash Flash Flash Flash
This is DDR3 directly to flash
5 Scalable I/O Performance, Constant Latency 1400 100 90 1200 3200G 80 1000 70
60 800 50 Demonstrated 600 At Open Compute (K) IOPS 1600G 40 Summit in Latency (us) Latency 30 January 2014 400 800G 20 200 400G 10
0 0 IBM 1 2 4 8 3650 2 socket Server Number of ULLtraDIMMTM Devices Running RHAT 6.3
6 The World’s First Flash Storage Device on the Memory Channel
Enterprise Class Memory Channel Reliability, Endurance Cost Effective Interface & Support
7 ULLtraDIMM Combines MCS with Storage
Guardian Technology ™ Ultra Low Latency, High Performance . Enterprise level endurance with MLC . 7µs write latency . 5 year warranty . 135K Read IOPS . 45K Write IOPS
Enterprise Class Reliability Scalable, Cost Effective Media . Back up power circuitry . 200, 400 GB . Full Data Path Protection . Scalable architecture . 2M hrs MTBF . Latest flash technology (19nm MLC)
Memory Channel Interface . DDR3 protocol . Configured as block device (through device driver)
8 Guardian TechnologyTM is Key Enabler
Base 19nm MLC NAND – 0.5 DWPD
Deep Flash Understanding Characterization Flash Capability Up to 10 0.5 DWPD DWPD
Use Each Flash Aggregated Flash Device to its Full Management Capability
19nm Value Based Optimizing Flash MLC NAND Advanced Signal High Performance Settings Over Processing Product Life High Endurance Enterprise Class SSD Endurance enhanced MLC NAND – 10 DWPD
9 ULLtraDIMM Series
ULLtraDIMM Ultra low latency + Usage Model scalable IOPS 200GB Capacity 400GB Form Factor RDIMM Endurance (Random) 10 DWPD MTBF 1.5M Hours Warranty 5 years
10 10 Utilizing Existing Server Infrastructure DDR3 Interface & Protocol Scalable I/O & Capacity
Example: IBM x3850 X6 server
. Scalable up to 32 ULLtraDIMM’s . Up to 12.8TB capacity . >4.5M IOPS in a single server
11 IBM Redpaper
Near Linear Scaling
Lower is better
12 Software Architecture
User space Management Applications Software
Kernel space OS Stack Block Layer
MCS Kernel Driver BIOS/UEFI
MCS Firmware Hardware Flash Controller Firmware
Diablo SanDisk OEM 3rd Party
13 ULLtraDIMM Population Schema
. Operates from 800 – 1,600 MT/s Mem Core0 Core1 Core0 Core1 Mem Chan Core2 Core3 Core2 Core3 Chan Mem Core4 Core5 Core4 Core5 Mem Chan Chan – Supports 1600MT/s Core6 Core7 Core6 Core7 Mem Mem Chan CPU0 CPU1 Chan – Interoperable with standard Mem Mem Chan Chan RDIMM in the same channel
Mem Mem Chan Chan . Can be populated to support many Mem Mem Chan CPU2 CPU3 Chan Mem Core0 Core1 Core0 Core1 Mem different performance points Chan Core2 Core3 Core2 Core3 Chan Mem Core4 Core5 Core4 Core5 Mem Chan Core6 Core7 Core6 Core7 Chan – Minimum Requirement = 1 RDIMM in the system Examples . Recommended population: DIMM # of # of Flash DRAM Read Write – 1 ULLtraDIMM per channel slots UD’s RDIMMs Capacity Capacity IOPS IOPS 24 2 22 800GB 352GB 300K 130K – Matched with 1 RDIMM per 24 4 20 1.6TB 320GB 600K 260K channel 24 8 16 3.2TB 256GB 1.2M 520K
14 Bus Utilization Q&A . Does ULLtraDIMM utilize the memory bus for FTL or other flash management?
– No, Using its embedded processors, ULLtraDIMM performs all flash management on the device itself and only transfers on DDR3 bus after its internal data processing is complete houg. • Analogous to a host bus adapter
. Does the ULLtraDIMM slow down the memory bus
– No, but ULLtraDIMM is limited by memory population guidelines (no different than an RDIMM)
– Memory controller will map out a slow device if it does not meet expected performance
– It is possible but can be mitigated by adding more parallel data processing by adding additional ULLtraDIMMs
. Can the ULLtraDIMM run slower than optimal under situations where its internal buffer gets saturated?
– It is possible but can be mitigated by adding more devices for additional parallel processing
15 SanDisk BIOS Modification for UlltraDIMMTM
. Why BIOS modifications are required for UlltraDIMMTM (UD) – Need to manage UD differently when detected (i.e. not as a standard DRAM) – Memory reservation (E820) and type (MTRR) tables need UD awareness – Initialization of UD hardware – ACPI table updates . BIOS modification guide contains following 4 high-level steps – MRC changes: disable memory test and interleaving of UD – E820 table: UD address space marked as reserved – MTRR table: UD address space marked as write-back – Training Sequence : SanDisk UD Training to be uploaded into UEFI – ACPI table: entries needed to expose UD to the OS device driver
16 CPU Utilization
. SanDisk is in the process of finalizing the CPU usage based on production level hardware and software . General guideline is to assign one Hyper thread for Two ULLtraDIMMs (or ~200K to 270K iops max)
Core Core Core Core
H1 H2 H1 H2 H1 H2 H1 H2
Core Core Core Core
H1 H2 H1 H2 H1 H2 H1 H2
17 ULLtraDIMM Series
ULLtraDIMM Usage Model Ultra low latency + scalable IOPS Capacity 200GB, 400GB Form Factor RDIMM Endurance (Random) 10 DWPD MTBF 1.5M Hours Warranty 5 years
Software Drivers Linux RedHat Linux SuSe Microsoft Windows Vmware ESX 6.3 SLES 11 SP1 2008R2 5.1U1 6.4 SLES 11 SP2 2012 5.1U2 6.5 SLES 11 SP3 2012 R2 5.5
18 Benefits to Applications
Financial Services Database/Cloud Virtualization Blade Server In Memory Compute*
. Block device . Block device/ . Block device . Block device . Memory . Low, predictable memory . Increased VMs per . Utilizes empty extension latency extension* Node DIMM slots . Reduce response . . Fast Interactive Increase . Faster response . Enables high times for analytics Data Analysis Transactions per times per VM density storage queries Second blades . Memcached . * Future roadmap consolidation
19 Benefits to Applications
Latency Write Read Bandwidth IOPS Good for sensitive intensive intensive intensive intensive Ulltradimm
OLTP database No SQL database Virtual Desktop High Frequency Trading Web 2.0
. Lowest latency SSD solution in the market . Near linear Scalability in IOPS and bandwidth with parallel processing architecture
20 E-trading Performance Using ULLtraDIMM
. Use ULLtraDIMM as primary storage for messages and events
. Architecture Enables: – Reduced Transaction Latency – Increased Transaction Throughput – Lower Transaction Latency Jitter
. ULLtraDIMM latency approaches RAMDisk performance using low cost 19nm MLC NAND
21 Reduced Latency Enables Real-Time Analytics
The Application Has Become the Bottleneck in E-Trading
22 Example: Blade Servers
. Requires dedicated blade server slot . Decreases server efficiency . Custom Form Factor
. Enables high density, high performance storage . Block and memory expansion storage . Improves blade server efficiency
23 Summary . First enterprise-class SSD that utilizes the memory bus . ULLtraDIMM takes advantage of the 12GB/s memory bus lanes . Lowest latency SSD in the market . Parallel architecture allows high IOPS and bandwidth performance
24 Thank You!
SanDisk is a trademark of SanDisk Corporation, registered in the United States and other countries. ULLtraDIMM is a trademark of SanDisk Enterprise IP LLC. Other brand names mentioned herein are for identification purposes only and may be the trademarks of their respective holder(s). *1 GB = 1,000,000,000 bytes. Actual user capacity less.
c 25