Flash Memory Overview

Steven Swanson Welcome to the Data Age

• The world processed 9 Zeabytes of data in 2008* • Acquiring data is easy • Extracng knowledge is hard – Storage performance is major boleneck – Solid-state storage can help

*hp://hmi.ucsd.edu Hardware/Soware Programming Prototyping interfaces

Data ECC & Device Security Characterizaon The Flash Juggernaut Flash is Fast! Hard Drives PCIe-Flash 2007

Lat.: 7.1ms 68us BW: 2.6MB/s 250MB/s 1x 104x 1x 96x • Random 4KB Reads from user space Flash Operaons

Read 0V 1V

5V 0V 20V 20V Erase Floang Gate Program

0V 0V Organizing Flash Cells into Chips

Select transistors

One block

One page

Data storage

Select Transistors

One NAND chain Organizing Flash Cells into Chips

One Block

Peripheral Logic • ~16K blocks/chip • ~16-64Gbits/chip Flash Operaons

Page: 0 1 2 3 4 n-4 n-3 n-2 n-1 n SLC: Single Level Cell Block 0 … Block 1 … == 1 bit Block 2 …

… … … MLC: Mul Level Cell Block n … == 2 bits Erase Program Blocks Pages TLC: Triple Level Cell

== 3 bits Single-Level Cell

Endurance: 100,000 Cycles Data retenon: 10 years Read Latency: 25us Program Latency: 100-200us Likelihood

== 1 bit 1 0

Charge on the floating gate Mul-Level Cell (2 bits)

Endurance: 5000-10,000 Cycles Data retenon: 3-10 years Read Latency: 25-37us Program Latency: 600-1800us Likelihood

== 2 bits

11 10 00 01

Charge on the floating gate Triple-level Cell (3bits)

Endurance: ~500-1000 Cycles Data retenon: 3 years Read Time: 60-120us Program Time: 500-6500us == 3 bits Likelihood

111 110 100 101 001 000 010 011

Charge on the floating gate Flash Failure Mechanisms

• Program/Erase (PE) Wear – Permanent damaged to the gate oxide at each flash cell – Caused by high program/erase voltages – Damage causes charge to leak off the floang gate • Program disturb – Data corrupon caused by interference from programming adjacent cells. – No permanent damage Making Disks out Flash Chips

Peripheral Logic

Read Pages Write Pages Read Erase Blocks Write Hierarchical addresses Flat address space PE Wear No wear limitaons Wring Data

SSD Maintain a map between “virtual” logical block addresses and “physical” flash locaons. Wring more data…

When you overwrite data, it goes to a new locaon. Flash Translaon Layer (FTL) User Soware • Logical Block Address Flash • Write pages in order FTL • Erase/Write granularity • Wears out FTL Flash • Logical à Physical map • Wear leveling • Power cycle recovery Centralized FTL State

Map Write Point

LBA Physical Page Address 101001011010001 0 Block 5 Page 7 010100100101011 101010110101001 2k Block 27 Page 0 111111111111111 4k Block 10 Page 2 111111111111111 111111111111111

Block Info Table Next Sequence Number: 12

Block Erased Erase Valid Page Sequence Bad Block Count Count Number Indicator 0 False 3 15 5 False 1 True 7 0 - False 2 False 0 4 9 False Read

Soware 1. Read Data at LBA 2k

2. Map

LBA Physical Page Address FTL 0 Block 5 Page 7 2k Block 27 Page 0 4k Block 10 Page 2

Flash 3. Flash Operaon Write – Mid Block

Write 0101101011001010 to LBA 2k Write Point = Block 2, Page 5

Map 1010010111010101 0101001010111011 LBA Physical Page Address 1010101101001010 0 Block 5 Page 7 2k Block 0 Page 0 4k Block 10 Page 2

Next Sequence Number: 12 Block Info Table

Block Erased Erase Valid Page Sequence Bad Block Count Count Number Indicator 0 False 3 15 5 False 1 True 7 0 - False 2 False 0 4 9 False Write – Mid Block

Write 0101101011001010 to LBA 2k Write Point = Block 2, Page 5 Write Point = Block 2, Page 6

Map 1010010111010101 0101001010111011 LBA Physical Page Address 1010101101001010 0101101011001010 0 Block 5 Page 7 2k Block 0 2 Page 0 5 4k Block 10 Page 2

Next Sequence Number: 12 Block Info Table

Block Erased Erase Valid Page Sequence Bad Block Count Count Number Indicator 0 False 3 15 14 5 False 1 True 7 0 - False 2 False 0 4 5 9 False Write – Block Jump (1)

Write 0101001010100110 to LBA 2k Write Point = Block 2, Page 63 Write Point = Block 1, Page 0

Map 0101011010101010 1010010111010101

1010001010111010 0101001010111011 LBA Physical Page Address 0101011010010101 1010101101001010 0101110100101000 0 Block 5 Page 7 0101011010100111 0101110100010110 2k Block 0 Page 5 1011101000101010 0101101001101010 4k Block 0 Page 2 Block 1

Next Sequence Number: 12 Block Info Table

Block Erased Erase Valid Page Sequence Bad Block Count Count Number Indicator 0 False 3 15 5 False 1 True 7 0 - False 2 False 0 4 9 False Write – Block Jump (1)

Write 0101001010100110 to LBA 2k Write Point = Block 2, Page 63 Write Point = Block 1, Page 0

Map 0101011010101010 1010010111010101

1010001010111010 0101001010111011 LBA Physical Page Address 0101011010010101 1010101101001010 0101110100101000 0101001010100110 0 Block 5 Page 7 0101011010100111 0101110100010110 2k Block 0 2 Page 5 63 1011101000101010 0101101001101010 4k Block 0 Page 2 Block 1

Next Sequence Number: 12 Block Info Table

Block Erased Erase Valid Page Sequence Bad Block Count Count Number Indicator 0 False 3 15 14 5 False 1 True 7 0 - False 2 False 0 4 5 9 False Write – Block Jump (2)

Write 1101000101101001 to LBA 4k Write Point = Block 1, Page 0 Write Point = Block 1, Page 1

Map 0101011010101010 1010010111010101

1010001010111010 0101001010111011 LBA Physical Page Address 0101011010010101 1010101101001010 0101110100101000 0101001010100110 0 Block 5 Page 7 0101011010100111 0101110100010110 2k Block 2 Page 63 1011101000101010 0101101001101010 4k Block 0 Page 2

Next Sequence Number: 12 Block Info Table

Block Erased Erase Valid Page Sequence Bad Block Count Count Number Indicator 0 False 3 14 5 False 1 True 7 0 - False 2 False 0 5 9 False Write – Block Jump (2)

Write 1101000101101001 to LBA 4k Write Point = Block 1, Page 0 Write Point = Block 1, Page 1

Map 0101011010101010 1010010111010101

1010001010111010 0101001010111011 LBA Physical Page Address 0101011010010101 1010101101001010 0101110100101000 0101001010100110 0 Block 5 Page 7 1101000101101001 0101011010100111 0101110100010110 2k Block 2 Page 63 1011101000101010 0101101001101010 4k Block 0 1 Page 2 0

Next Sequence Number: 12 13 Block Info Table

Block Erased Erase Valid Page Sequence Bad Block Count Count Number Indicator 0 False 3 14 13 5 False 1 T F 7 0 1 12 False 2 False 0 5 9 False Erase Block Info Table

Block Erased Erase Valid Page Sequence Bad Block Count Count Number Indicator 0 False 3 13 5 False 1 False 7 1 12 False 2 False 0 3 9 False

Move Valid Pages

1010010111010101 0101011010101010 Block 2 0101001010111011 1010001010111010

0101011010010101 1010101101001010

0101110100101000

1101000101101001

0101011010100111

0101110100010110

1011101000101010 Erase Block Info Table

Block Erased Erase Valid Page Sequence Bad Block Count Count Number Indicator 0 False 3 13 5 False 1 False 7 1 12 False 2 False 0 3 0 9 False

Move Valid Pages

1010010111010101 0101011010101010 Block 2 0101001010111011 1010001010111010 Update: 0101011010010101 1010101101001010 0101110100101000 1010001010111010 •Map 1101000101101001 1101000101101001 • 0101011010100111 0101011010100111 Valid Pg Counts 0101110100010110 •etc. 1011101000101010 Erase Block Info Table

Block Erased Erase Valid Page Sequence Bad Block Count Count Number Indicator 0 False 3 13 5 False 1 False 7 1 12 False 2 F T 01 0 - False

Move Valid Pages

Block 2 1010010111010101 0101001010111011 Update: 1010101101001010 1010001010111010 •Map 1101000101101001

0101011010100111 •Valid Pg Counts •etc. Distributed FTL State Metadata

Typical Page LBA ECC Erase Erase Sequence Bad Block Out of Band EDC Count Seal Number Indicator Data 2k 3 False 5 False

Data Only Necessary in First and Last Page of Block

Summary Page Typical Block Physical Logical Block Page 0 Page 1 Page Address Page 2 0 10k . . 1 32k . 2 14k Page n Power Cycle

Map Write Point

LBA Physical Page Address 1010010111010101 Scan each block: 0101001010111011 1. Summary page 0 Block 5 Page 7 1010101101001010 2. First Page 2k Block 27 Page 0 3. All Pages 4k Block 10 Page 2

Block Info Table Next Sequence Number: 12

Block Erased Erase Valid Page Sequence Bad Block Count Count Number Indicator 0 False 3 15 5 False 1 True 7 0 0 False 2 False 0 4 9 False