Storage in the DIMM Socket

StoragePRESENTATION in the TITLE DIMM GOES HERE Socket

Adrian Proctor Vice President, Marketing Viking Tec hno logy SNIA Legal Notice

The material contained in this tutorial is copyrighted by the SNIA unless otherwise noted. Member companies and individual members may use this material in presentations and literature under the following conditions: Any slide or slides used must be reproduced in their entirety without modification The SNIA must b e ack nowl ed ged as th e source of any mat eri al used i n th e b od y o f any document containing material from these presentations. This presentation is a project of the SNIA Education Committee. Neither the author nor the ppygresenter is an attorney and nothing in this presentation is intended to be, or should be construed as legal advice or an opinion of counsel. If you need legal advice or a legal opinion please contact your attorney. The information presented herein represents the author' s personal opinion and current understanding of the relevant issues involved. The author, the presenter, and the SNIA do not assume any responsibility or liability for damages arising out of any reliance on or use of this information. NO WARRANTIES, EXPRESS OR IMPLIED. USE AT YOUR OWN RISK.

As data sets continue to grow, IT managers have begun seeking out new ways for memoryygypy technology to be deployed in the data center in order to take greater advantage of the performance and latency benefits. Non–Volatile DIMMs, or NVDIMMs, provide a persistent memory solution with the endurance and performance of DRAM coupled with the non–volatility associtdiated w ithtith storage. This tutorial will provide a general overview of this emerging technology and how it plays in the data center. You will learn what an NVDIMM is, how it works, where it fits and why system architects should consider them for their next generation enterprise server and storage designs.

The Memoryyg / Storage Problem (y)(Latency) New Memory Technology Roadmap NVDIMMs – What they are NVDIMMs – How they work NVDIMMs – System Considerations NVDIMM Performance NVDIMM Ecosystem Summary

NON-VOLATILE MEMORY HIERARCHY VOLATILE MEMORY HIERARCHY

10K ns 100ns

PCIe SSD DRAM 100K ns 10ns SAS SSD 10M ns CPU Cache 1ns

HDD

 As CPU technology scales with Moore’s Law, memory IO creates significant performance bottlenecks  The latency gap in memory / storage hierarchy needs to be bridged  NVDIMM offers a solution today (Storage at DRAM Latency) Shining Light on the DIMM Slot © 2014 Storage Networking Industry Association. All Rights Reserved. Memory/Storage Hierarchy

. Data-Intensive Applications Need Fast Access To Storage . Larggpe Performance Gap Between Main Memor y And HDD . SSDs Have Narrowed The Gap, But a Big Gap Still Exists . Until an “SCM” becomes viable for mainstream adoption (2020?) Performance Gap

CPU MAINNew Memory SSD HDD CACHE MEMORYTechnology MRAM / ReRAM

SRAM DRAM NAND Magnetic

0 1 2 3 4 5 6

Volatile DRAM Non-Volatile Memory

yy . MRAM SCM . PCM . ReRAM apabilit CC

NVDIMM elative RR

DDR4 DDR3 . SliScaling PblProblems . MFG Concerns <2x nm

2012 2014 2016 2020

Shining Light on the DIMM Slot 7 NVDIMM - SNIA © 2014 Storage Networking Industry Association. All Rights Reserved. WHAT THE INDUSTRY WANTS FROM MEMORY (THE HOLY GRAIL) . Infinite Endurance . Lowest Latency . High Capacity . Non-Vo la tile . Low Power NVDIMM DOES MOST OF THIS TODAY… . Scalability . Low Cost

Reside on the Memory Channel (DDR3/DDR4) Retain data in the event of an unexpected power loss Combines mature memory technologies (DRAM and Flash) Requires independent power source to ensure persistence Fits well with the NVM Programming Model (as precursor to SCM) Delivers new levels of storage performance Databases can run faster and recover more quickly Can enhance both SSD endurance and reliability

NORMAL OPERATION: 1. During normal operation, the NVDIMM appears like a standard DDR3 DRAM module. » DRAM Latency (nanoseconds) » DRAM Endurance (practically infinite) » DRAM Bandwidth (12GB/s per NVDIMM)

POWER-FAIL - DRAM SAVE to FLASH: 1. Power-Fail Event: The NVDIMM isolated from the BUS. All data (incl. ECC) in the DRAM is SAVED to onboard Flash via the NVDIMM Logic. Super Capacitors provide hold up power to the module during this operation. 2. When the SAVE completes. The NVDIMM module is then shut down.

POWER RESUME: 1. When power to the system is returned, Super Capacitors are re-charged & the data is RESTORED back from FLASH into the DRAM.

NORMAL STATE: 1. Once all data is RESTORED back into the DRAM, the NVDIMM is ready for I/O transactions with the host system. 2. Host system finishes BOOT and normal NVDIMM operation continues.

The “Pieces of the Puzzle” that are required for NVDIMM Integration

System Support H/W Trigger Mechanical System Management (ADR) (Power Health) (Power Source)

“NVDIMM –A”Aware” Application NVDIMM BIOS

“…every 100ms of latency cost them 1% in sales”

“…an extra 500ms in search page generation time dropped traffic by 20%”

“…a broker could lose $4M per millisecond if their electronic trading platform is 5ms behind the competition”

Ecosystem performance gap between compute & storage

CPU • Nanoseconds latency (1000x faster than Flash) NVDIMM DRAM

(Latency) • 1.4 million IOPS (3x better) anoseconds ee NN

PERFORMANCE GAP Performanc

PCIe SSD SSD

HDD milliseconds

1GB – 32GB 60GB – 4TB STORAGE

Benchmark: VDBENCH, Platform: Intel Sandybridge, Linux, Two DDR3-1333 NVDIMMs as interleaved pair (channel interleaving), PRAMFS vs. SATA SSD as Linux block device

Performance vs. Cost ($ per I/O) trade-off

HDD

COST ($) COST SSD

PCIe SSD

PERFORMANCE / COST GAP NVDIMM

DRAM

*Cost per PB written: PERFORMANCE Best in Class SSD: $100.00 / PB versus. NVDIMM: $0.40 (250x cost savings) Reference: http://www.vikingtechnology.com/uploads/NVDIMM_Technical_Comparison.pdf Shining Light on the DIMM Slot Volatile Non-Volatile 18 NVDIMM - SNIA © 2014 Storage Networking Industry Association. All Rights Reserved. NVDIMM ADOPTION

The flow of NVDIMM evolution and adoption

Standards NVDIMM JEDEC Vendors

Motherboard Platform Integrated BIOS ODM’s, OEM’s Solutions

Shining Light on the DIMM Slot 19 NVDIMM - SNIA © 2014 Storage Networking Industry Association. All Rights Reserved. Innovation and Alternate Memory Bus Device Options …. The NVDIMM-SIG is in the process of developing taxonomy to helppygpg the industry distinguish product categories

DRAM Modules NVDIMM MCS (ULLtraDIMM) SATADIMM HDIMM (Hybrid DIMM) MRAM, ReRAM

 Flash is cheaper than DRAM $/GB  NVDIMM has 1000x lower latency than Flash  DRAM has practically infinite endurance

 Hyperscale want “Dense & Cheap” (WORM)  Financial want low & predictable latency  Storage wants higher I/O performance & increased data security  No individual “BEST” choice – There are OPTIONS….

Standard Servers become Highly Flexible There is a solution for whatever the Application Demands • Highest Performing Storage - NVDIMM • High Capacity Flash – PCIe SSD • Lower La tency SSD – ULLtra DIMM

NVDIMM Special Interest Group (SIG) formed January 2014 Orgggpanized under the SNIA Solid State Storage Initiative to help: Accelerate awareness and adoption of NVDIMMs Vendors collaborate to broaden industry support and knowledge SNIA’ s history of developing standards and providing education: The NVM Programming Model Technical Working Group Ideal venue for NVDIMM SIG support NVDIMM SIG will: Educate on how system vendors can design in NVDIMM Communicate industry standards as they evolve Develop market understanding of NVDIMM technology Communicate how new programming models help deliver value

The SNIA Education Committee would like to thank the following individuals for their contributions to this Tutorial. Authorship History Additional Contributors

Original Author: Adrian Proctor 3/2014

Please send any questions or comments regarding this SNIA Tutorial to [email protected]