Flash Drives: Block Storage or Memory? Tony Roug, Virident Director Solution Architecture 2 ©2012 Virident Systems Inc. Confidential and Proprietary Agenda • Storage Class Memory: The Bridge • Access Models: The Architecture • Access Capabilities: The Value ©2012 Virident Systems Inc. Confidential and Proprietary 3 Storage Class Memory Storage Class Memory (SCM) CPU CPU Disk SATA/SAS PCIe Memory- SSDs SSDs class SCM DRAM DRAM DRAM DRAM SATA/SAS PCIe Memory-class Attribute Disk DRAM SSDs SSD SCM Capacity (GB) 100’s-1000’s 100’s 100’s-1000’s 100’s-1000’s 10’s-100’s Read 10’s ms, 100’s us, 5010 –’ s75 us, us, 200100 – ’400s ns, ns, performance ~100 MB/s ~100’s MB/s 21 –’s 4GB/s GB/s 21 ’–s 5 GB/s GB/s ~100 ns, Write 10’s ms 100’s us ~25~10 –’ s50 us, us, 1~5’s us, 10’s GB/s performance ~100 MB/s ~100 MB/s ~0.5~1 ’–s 1+GB/s GB/s 12’-s5 GB/sGB/s PCIe SSDs established, Memory-class emerging ©2012 Virident Systems Inc. Confidential and Proprietary 4 Agenda • Storage Class Memory: The Bridge • Access Models: The Architecture • Access Capabilities: The Value ©2012 Virident Systems Inc. Confidential and Proprietary 5 Block versus Memory access Application Migrated Performance Application Perceived Performance Storage SSDs PCIe SSDs Memory Class SCM Datacenter SCM-aware Applications cache cache SCM Applications opt opt cache File File SCM File SCM Direct Direct System System System Direct OS Block Drivers Block Drivers SCM Drivers Storage SATA, SAS, FC PCIe Hardware PCIe, Memory Optimization required for applications to realize memory class benefit ©2012 Virident Systems Inc. Confidential and Proprietary 6 Open industry directions Storage SSDs PCIe SSDs Memory Class SCM Datacenter flashcach SCM-aware Applications cache cache SCM Applications e opt opt cache bcache File File SCM SNIAFile NVMeSCM Direct Direct System System ProgrammingSystem TWGDirect OS NVMExpres s Block Drivers Block Drivers NVMExpressSCM Drivers SCSIExpres s Storage SATA, SAS, FC Hardware PCIe SFF -8639 PCIe, Memory Industry agreement for optimized architecture in place ©2012 Virident Systems Inc. Confidential and Proprietary 7 Agenda • Storage Class Memory: The Bridge • Access Models: The Architecture • Access Capabilities: The Value ©2012 Virident Systems Inc. Confidential and Proprietary 8 Example PCIe Block Capabilities: Virident User Level FlashMAX • File open and close, file read and write, raw read and write User Block IO • Multiple threads through libaio libaio O_Direct File System • Direct DMA from user to board/board to user through o_direct • CLI: monitor, diag, beacon, secure erase Kernel Block IO Kernel Level Kernel Block IO Virtual to • block register: register, unregister, open, release, ioctrl MetaData Wear Level • block transfer: request, queue Physical • DMA scatter/gather: blk_rq_map_sg Rate Limit DMA Setup GC • rate limit: guaranteed minimum performance at 100% capacity • wear leveling: identification/adaptation to app access patterns PCIe I/F Board Level PCIe I/F • partition options: 512 and 4K sector size DMA Engines Req/Resp Queues • write acceleration mode: maxperformance, maxcapacity RAID 5 • reliability option: raid5 enabled/disabled SRAM ECC SRAM ECC Flash I/F … Flash I/F FLASH FLASH FLASH FLASH PCIe SSD optimizations within a block level interface ©2012 Virident Systems Inc. Confidential and Proprietary 9 NVM Express: “memory class” capabilities? Function NVMExpress 1.0 Capability Block Compatibility Basic Features Read, Write Read, Write Flush Sync Cache Format Format ACID Fused Operations Compare and Write Operations Atomic Write Unit New ACID Capabilities Write Atomicity Performance, Queues/PRP Endurance, Data Set Management: Incompressible, Access Size, New Performance and Efficiency Write Prepared, Sequential Read/Write, Access Freq, Features Access Latency Capabilities Compare Namespaces Deallocate TRIM/UNMAP Power Management Start Stop Data End-to-end Protection + Metadata DIF/DIX protection Format Secure Erase Secure Erase Write uncorrectable Write Long Write protect WRPROTECT ©2012New Virident Systemscapabilities Inc. Confidential and makeProprietary “memory10 class” access possible, but not easy Enabling SCM: Making it easy 1. streamline the application access path: e.g. RDMA access SCM-aware 2. offer improved semantics: e.g. atomic-write SCM Applications 3. simplify caching implementations: e.g. key-value cache store, User 4. construct cluster-level solutions: e.g. synchronize cross-server activities and SCM File SCM orchestrate server to SAN data movement System Direct 5. emulate memory: e.g. mmap and malloc 6. expose internals of flash management: e.g. Kernel optimized file system with snapshot, cross-card data transfers, trading capacity vs performance NVM Express Driver NVM Express Create industry standard capabilities enabling memory class access ©2012 Virident Systems Inc. Confidential and Proprietary 11 Summary • Storage Class Memory: • First level value enabled by block mode • Access Models: • NVM Express enables memory class usage models • Access Capabilities: • Industry agreement on application enabling technology launched • SNIA Programming TWG ©2012 Virident Systems Inc. Confidential and Proprietary 12 Thanks Tony Roug Phone: 4085829647 Email: [email protected] ©2012 Virident Systems Inc. Confidential and Proprietary 13 .