Chungbuk National University Reducing Write Amplification of Flash Storage through Cooperative Data Management with NVM 32nd International Conference on Massive Storage Systems and Technology (MSST) May, 2016 Eunji Lee, Chungbuk Natational University Julie Kim, Ewha University Hyokyung Bahn, Ewha University Sam H. Noh, UNIST 1 Chungbuk National University Write Amplification in SSD Undesirable phenomenon associated with flash memory Number of writes to storage is higher than the number of writes issued by a host Key aspect limiting stable performance and endurance of SSD Performance is fluctuating! Source: Radian Memory Systems 2 Chungbuk National University Write Amplification in SSD Garbage collection is performed to recycle used blocks Copy out valid pages in a victim block into a free block write (B’ F’ G’ H’) write (B’ F’ G’ H’) + write (A C D E ) in GC A E B’ B’ A B F F’ F’ C C G G’ G’ D D H H’ H’ E Write 4 Blocks Write 8 Blocks 2x Writes! Write Amplification Factor : 2.0 3 Chungbuk National University Workload and WAF Relationship Analyze WAF with respect to workload characteristics Generate two synthetic workloads using SSDsim Sequential writes Random writes 7.0 7.0 6.0 6.0 6x writes! 5.0 5.0 4.0 4.0 WAF 3.0 No write amplification 3.0 2.0 2.0 1.0 1.0 0.0 0.0 4GB 8GB 16GB 32GB 64GB 4GB 8GB 16GB 32GB 64GB SSD Capacity SSD Capacity Random updates incur the dispersed distribution of the valid pages 4 Chungbuk National University Workload and WAF Relationship Real workload is a mixture of random and sequential accesses Observe WAF varying the level of randomness 1.4 1.35 1.3 1.25 1.2 A low level of randomness 1.15 WAF amplifies writes! 1.1 1.05 1 100% 100% random random r:s = 9:1 r:s = 8:2 r:s = 7:3 r:s = 6:4 r:s = 5:5 r:s = 4:6 r:s = 3:7 r:s = 2:8 r:s = 1:9 Sequential sequential Real workloads are likely to be between two opposites 5 Chungbuk National University NVM Technology Becoming increasingly viable as leading semi-conductor manufacturers are eagerly investing in it . Diablo Technologies, Memory 1 . All-flash system memory module . 4TB Memory . Intel and Micron, 3DXpoint . All-new memory technology . 8x-10x denser than DRAM . 1000 times less latency than NAND Fast, scalable, and persistent memory is being realized in computer systems 6 Chungbuk National University Cooperative Data Management (CDM) Goal: Reduce WAF by taking advantage of non- volatility of caches Using NVM as a storage cache – promising option Volatile Cache Non-Volatile Cache A C A C B’ D B’ D A E B’ A E B’ B F F’ B F F’ C G G’ C G G’ D H H’ D H H’ Flash Storage Flash Storage 7 Chungbuk National University Cooperative Data Management (CDM) In traditional systems, all valid pages in a victim block should be copied into a free block during GC 4 block writes ! Volatile Cache Non-Volatile Cache A C A C B’ D B’ D A E B’ A A E B’ B F F’ C B F F’ C G G’ D C G G’ D H H’ E D H H’ Flash Storage Flash Storage 8 Chungbuk National University Cooperative Data Management (CDM) CDM skips the copying of valid pages in GC if the data exist in non-volatile cache Only one block write! Volatile Cache Non-Volatile Cache A C A C B’ D B’ D A E B’ A NO A E B’ E B F F’ C B F F’ C G G’ D NO C G G’ D H H’ E NO D H H’ Flash Storage Flash Storage 9 Chungbuk National University Cooperative Data Management (CDM) Finite state diagram . “Removable” state - Can be erased if the data needs to be copied into another block - Or it is same as the valid state . Data is set to “Removable” when it is cached in a non-volatile cache 10 Chungbuk National University Getting Real: Issues with CDM Implementation of CDM has several issues depending on the architectures Feasible architectures . NV-cache as a page cache in host . NV-cache as an in-storage cache Host’s page cache Host’s page cache NV-cache DRAM-cache NV-cache Flash Storage Flash Storage 11 Chungbuk National University 1. NV-cache as a host cache Issue 1. Consistency Updating data in a cache touches a final copy Crash during update results in inconsistent data Volatile Cache Non-Volatile Cache C’ crash crash A C A C’ B’ D B’ D B’ A B’ E F’ C F’ G’ D G’ H’ E H’ Flash Storage Flash Storage 12 Chungbuk National University 1. NV-cache as a host cache Issue 1. Consistency Solution is associated with specific file system implementations Data consistency is managed in a file system layer with different techniques File systems should be redesigned considering a way of handling data in CDM 13 Chungbuk National University Case Study: Ext4 with CDM Finite State Diagram Introduce additional states of cached data to determine whether its another copy remains in storage Update data with a copy- on-write technique if the cached data serves as a final copy 14 Chungbuk National University 1. NV-cache as a host cache Issue 2. Communication Overhead Events in cache and storage should be notified to each other synchronously . e.g., Garbage Collection, Erase of a block, Cached data update, etc. Designing a new interface is no longer a big deal . Recent packet-based interfaces like NVM-e are easy to piggyback additional information on the original data However, frequent communication for additional information transfer can be a burden . Consider finding a way to relieve the overhead as a future work 15 Chungbuk National University 2. NV-cache as an in-storage cache More deployable architecture No consistency issue . File systems assume that the data in storage can become inconsistent if a system crashes during updates Not serious communication overhead . Sharing information inside a storage Host’s page cache device is much cheaper and easier than synchronizing storage with a host cache DRAM-cache Development can be achieved by a single party - storage manufacturer NV-cache . No need to change file systems Flash Storage 16 Chungbuk National University Performance Evaluation Trace-driven simulation with SSDsim . Developed by MSR as an extension of Disksim . Emulate SLC NAND Flash 17 Chungbuk National University Performance Evaluation Implement the in-storage NV-cache module and modify a storage controller to support CDM Compare with a NVM-basic model . Manage NVM like a volatile cache . Cache data on access and evict it with LRU policy 18 Chungbuk National University Write Amplification Factor CDM reduces WAF by 2.1-17.6% and 4.3-38.2% in JEDEC and OLTP workloads 19 Chungbuk National University Response Time Average response time is improved by 9.7% and 20.3% on average in JEDEC and OLTP 20 Chungbuk National University Standard Deviation of Response Time Reduce standard deviation of response time by 31% and 39% on average in JEDEC and OLTP Relieve the performance fluctuating 21 Chungbuk National University Thank you! [email protected] http://oslab.cbnu.ac.kr 22.