Chungbuk National University

Reducing of Flash Storage through Cooperative Data Management with NVM

32nd International Conference on Massive Storage Systems and Technology (MSST) May, 2016

Eunji Lee, Chungbuk Natational University Julie Kim, Ewha University Hyokyung Bahn, Ewha University Sam H. Noh, UNIST

1 Chungbuk National University Write Amplification in SSD

 Undesirable phenomenon associated with  Number of writes to storage is higher than the number of writes issued by a host  Key aspect limiting stable performance and endurance of SSD

Performance is fluctuating!

Source: Radian Memory Systems 2 Chungbuk National University Write Amplification in SSD

 Garbage collection is performed to recycle used blocks  Copy out valid pages in a victim block into a free block

write (B’ F’ G’ H’) write (B’ F’ G’ H’) + write (A C D E ) in GC

A E B’ B’ A B F F’ F’ C C G G’ G’ D D H H’ H’ E Write 4 Blocks Write 8 Blocks

2x Writes! Write Amplification Factor : 2.0 3 Chungbuk National University Workload and WAF Relationship

 Analyze WAF with respect to workload characteristics  Generate two synthetic workloads using SSDsim

Sequential writes Random writes 7.0 7.0 6.0 6.0 6x writes! 5.0 5.0 4.0 4.0

WAF 3.0 No write amplification 3.0 2.0 2.0 1.0 1.0 0.0 0.0 4GB 8GB 16GB 32GB 64GB 4GB 8GB 16GB 32GB 64GB SSD Capacity SSD Capacity Random updates incur the dispersed distribution of the valid pages 4 Chungbuk National University Workload and WAF Relationship

 Real workload is a mixture of random and sequential accesses  Observe WAF varying the level of randomness

1.4 1.35

1.3 1.25 1.2 A low level of randomness 1.15

WAF amplifies writes! 1.1

1.05 1 100% 100% random

random r:s = 9:1 r:s = 8:2 r:s = 7:3 r:s = 6:4 r:s = 5:5 r:s = 4:6 r:s = 3:7 r:s = 2:8 r:s = 1:9 Sequential sequential Real workloads are likely to be between two opposites

5 Chungbuk National University NVM Technology

 Becoming increasingly viable as leading semi-conductor manufacturers are eagerly investing in it . Diablo Technologies, Memory 1 . All-flash system memory module . 4TB Memory . and Micron, 3DXpoint . All-new memory technology . 8x-10x denser than DRAM . 1000 times less latency than NAND

 Fast, scalable, and persistent memory is being realized in computer systems

6 Chungbuk National University Cooperative Data Management (CDM)

 Goal: Reduce WAF by taking advantage of non- volatility of caches  Using NVM as a storage cache – promising option

Volatile Cache Non-Volatile Cache

A C A C B’ D B’ D

A E B’ A E B’ B F F’ B F F’ C G G’ C G G’ D H H’ D H H’

Flash Storage Flash Storage

7 Chungbuk National University Cooperative Data Management (CDM)

 In traditional systems, all valid pages in a victim block should be copied into a free block during GC  4 block writes !

Volatile Cache Non-Volatile Cache

A C A C B’ D B’ D

A E B’ A A E B’ B F F’ C B F F’ C G G’ D C G G’ D H H’ E D H H’

Flash Storage Flash Storage

8 Chungbuk National University Cooperative Data Management (CDM)

 CDM skips the copying of valid pages in GC if the data exist in non-volatile cache  Only one block write!

Volatile Cache Non-Volatile Cache

A C A C B’ D B’ D

A E B’ A NO A E B’ E B F F’ C B F F’ C G G’ D NO C G G’ D H H’ E NO D H H’

Flash Storage Flash Storage 9 Chungbuk National University Cooperative Data Management (CDM)

 Finite state diagram . “Removable” state - Can be erased if the data needs to be copied into another block - Or it is same as the valid state . Data is set to “Removable” when it is cached in a non-volatile cache

10 Chungbuk National University Getting Real: Issues with CDM

 Implementation of CDM has several issues depending on the architectures  Feasible architectures . NV-cache as a page cache in host . NV-cache as an in-storage cache

Host’s page cache Host’s page cache

NV-cache DRAM-cache

NV-cache Flash Storage Flash Storage

11 Chungbuk National University 1. NV-cache as a host cache

 Issue 1. Consistency  Updating data in a cache touches a final copy  Crash during update results in inconsistent data

Volatile Cache Non-Volatile Cache

C’ crash crash A C A C’ B’ D B’ D

B’ A B’ E F’ C F’ G’ D G’ H’ E H’

Flash Storage Flash Storage

12 Chungbuk National University 1. NV-cache as a host cache

 Issue 1. Consistency  Solution is associated with specific implementations  Data consistency is managed in a file system layer with different techniques  File systems should be redesigned considering a way of handling data in CDM

13 Chungbuk National University Case Study: Ext4 with CDM

 Finite State Diagram  Introduce additional states of cached data to determine whether its another copy remains in storage  Update data with a copy- on-write technique if the cached data serves as a final copy

14 Chungbuk National University 1. NV-cache as a host cache

 Issue 2. Communication Overhead  Events in cache and storage should be notified to each other synchronously . e.g., Garbage Collection, Erase of a block, Cached data update, etc.  Designing a new interface is no longer a big deal . Recent packet-based interfaces like NVM-e are easy to piggyback additional information on the original data  However, frequent communication for additional information transfer can be a burden . Consider finding a way to relieve the overhead as a future work 15 Chungbuk National University 2. NV-cache as an in-storage cache

 More deployable architecture  No consistency issue . File systems assume that the data in storage can become inconsistent if a system crashes during updates  Not serious communication overhead . Sharing information inside a storage Host’s page cache device is much cheaper and easier than synchronizing storage with a host cache DRAM-cache  Development can be achieved by a single party - storage manufacturer NV-cache . No need to change file systems Flash Storage

16 Chungbuk National University Performance Evaluation

 Trace-driven simulation with SSDsim . Developed by MSR as an extension of Disksim . Emulate SLC NAND Flash

17 Chungbuk National University Performance Evaluation

 Implement the in-storage NV-cache module and modify a storage controller to support CDM

 Compare with a NVM-basic model . Manage NVM like a volatile cache . Cache data on access and evict it with LRU policy

18 Chungbuk National University Write Amplification Factor

 CDM reduces WAF by 2.1-17.6% and 4.3-38.2% in JEDEC and OLTP workloads

19 Chungbuk National University Response Time

 Average response time is improved by 9.7% and 20.3% on average in JEDEC and OLTP

20 Chungbuk National University Standard Deviation of Response Time

 Reduce standard deviation of response time by 31% and 39% on average in JEDEC and OLTP  Relieve the performance fluctuating

21 Chungbuk National University Thank you!

[email protected]  http://oslab.cbnu.ac.kr

22