Chungbuk National University
Reducing Write Amplification of Flash Storage through Cooperative Data Management with NVM
32nd International Conference on Massive Storage Systems and Technology (MSST) May, 2016
Eunji Lee, Chungbuk Natational University Julie Kim, Ewha University Hyokyung Bahn, Ewha University Sam H. Noh, UNIST
1 Chungbuk National University Write Amplification in SSD
Undesirable phenomenon associated with flash memory Number of writes to storage is higher than the number of writes issued by a host Key aspect limiting stable performance and endurance of SSD
Performance is fluctuating!
Source: Radian Memory Systems 2 Chungbuk National University Write Amplification in SSD
Garbage collection is performed to recycle used blocks Copy out valid pages in a victim block into a free block
write (B’ F’ G’ H’) write (B’ F’ G’ H’) + write (A C D E ) in GC
A E B’ B’ A B F F’ F’ C C G G’ G’ D D H H’ H’ E Write 4 Blocks Write 8 Blocks
2x Writes! Write Amplification Factor : 2.0 3 Chungbuk National University Workload and WAF Relationship
Analyze WAF with respect to workload characteristics Generate two synthetic workloads using SSDsim
Sequential writes Random writes 7.0 7.0 6.0 6.0 6x writes! 5.0 5.0 4.0 4.0
WAF 3.0 No write amplification 3.0 2.0 2.0 1.0 1.0 0.0 0.0 4GB 8GB 16GB 32GB 64GB 4GB 8GB 16GB 32GB 64GB SSD Capacity SSD Capacity Random updates incur the dispersed distribution of the valid pages 4 Chungbuk National University Workload and WAF Relationship
Real workload is a mixture of random and sequential accesses Observe WAF varying the level of randomness
1.4 1.35
1.3 1.25 1.2 A low level of randomness 1.15
WAF amplifies writes! 1.1
1.05 1 100% 100% random
random r:s = 9:1 r:s = 8:2 r:s = 7:3 r:s = 6:4 r:s = 5:5 r:s = 4:6 r:s = 3:7 r:s = 2:8 r:s = 1:9 Sequential sequential Real workloads are likely to be between two opposites
5 Chungbuk National University NVM Technology
Becoming increasingly viable as leading semi-conductor manufacturers are eagerly investing in it . Diablo Technologies, Memory 1 . All-flash system memory module . 4TB Memory . Intel and Micron, 3DXpoint . All-new memory technology . 8x-10x denser than DRAM . 1000 times less latency than NAND
Fast, scalable, and persistent memory is being realized in computer systems
6 Chungbuk National University Cooperative Data Management (CDM)
Goal: Reduce WAF by taking advantage of non- volatility of caches Using NVM as a storage cache – promising option
Volatile Cache Non-Volatile Cache
A C A C B’ D B’ D
A E B’ A E B’ B F F’ B F F’ C G G’ C G G’ D H H’ D H H’
Flash Storage Flash Storage
7 Chungbuk National University Cooperative Data Management (CDM)
In traditional systems, all valid pages in a victim block should be copied into a free block during GC 4 block writes !
Volatile Cache Non-Volatile Cache
A C A C B’ D B’ D
A E B’ A A E B’ B F F’ C B F F’ C G G’ D C G G’ D H H’ E D H H’
Flash Storage Flash Storage
8 Chungbuk National University Cooperative Data Management (CDM)
CDM skips the copying of valid pages in GC if the data exist in non-volatile cache Only one block write!
Volatile Cache Non-Volatile Cache
A C A C B’ D B’ D
A E B’ A NO A E B’ E B F F’ C B F F’ C G G’ D NO C G G’ D H H’ E NO D H H’
Flash Storage Flash Storage 9 Chungbuk National University Cooperative Data Management (CDM)
Finite state diagram . “Removable” state - Can be erased if the data needs to be copied into another block - Or it is same as the valid state . Data is set to “Removable” when it is cached in a non-volatile cache
10 Chungbuk National University Getting Real: Issues with CDM
Implementation of CDM has several issues depending on the architectures Feasible architectures . NV-cache as a page cache in host . NV-cache as an in-storage cache
Host’s page cache Host’s page cache
NV-cache DRAM-cache
NV-cache Flash Storage Flash Storage
11 Chungbuk National University 1. NV-cache as a host cache
Issue 1. Consistency Updating data in a cache touches a final copy Crash during update results in inconsistent data
Volatile Cache Non-Volatile Cache
C’ crash crash A C A C’ B’ D B’ D
B’ A B’ E F’ C F’ G’ D G’ H’ E H’
Flash Storage Flash Storage
12 Chungbuk National University 1. NV-cache as a host cache
Issue 1. Consistency Solution is associated with specific file system implementations Data consistency is managed in a file system layer with different techniques File systems should be redesigned considering a way of handling data in CDM
13 Chungbuk National University Case Study: Ext4 with CDM
Finite State Diagram Introduce additional states of cached data to determine whether its another copy remains in storage Update data with a copy- on-write technique if the cached data serves as a final copy
14 Chungbuk National University 1. NV-cache as a host cache
Issue 2. Communication Overhead Events in cache and storage should be notified to each other synchronously . e.g., Garbage Collection, Erase of a block, Cached data update, etc. Designing a new interface is no longer a big deal . Recent packet-based interfaces like NVM-e are easy to piggyback additional information on the original data However, frequent communication for additional information transfer can be a burden . Consider finding a way to relieve the overhead as a future work 15 Chungbuk National University 2. NV-cache as an in-storage cache
More deployable architecture No consistency issue . File systems assume that the data in storage can become inconsistent if a system crashes during updates Not serious communication overhead . Sharing information inside a storage Host’s page cache device is much cheaper and easier than synchronizing storage with a host cache DRAM-cache Development can be achieved by a single party - storage manufacturer NV-cache . No need to change file systems Flash Storage
16 Chungbuk National University Performance Evaluation
Trace-driven simulation with SSDsim . Developed by MSR as an extension of Disksim . Emulate SLC NAND Flash
17 Chungbuk National University Performance Evaluation
Implement the in-storage NV-cache module and modify a storage controller to support CDM
Compare with a NVM-basic model . Manage NVM like a volatile cache . Cache data on access and evict it with LRU policy
18 Chungbuk National University Write Amplification Factor
CDM reduces WAF by 2.1-17.6% and 4.3-38.2% in JEDEC and OLTP workloads
19 Chungbuk National University Response Time
Average response time is improved by 9.7% and 20.3% on average in JEDEC and OLTP
20 Chungbuk National University Standard Deviation of Response Time
Reduce standard deviation of response time by 31% and 39% on average in JEDEC and OLTP Relieve the performance fluctuating
21 Chungbuk National University Thank you!
[email protected] http://oslab.cbnu.ac.kr
22