18. MEMORY HIERARCHY

COS375 / ELE375 3/12/2021

Mohammad Shahrad

Acknowledgements: David August Margaret Martonosi Gennady Pekhimenko Princeton Graduate College Forbes College

Springdale Golf Course

1 2 Engineering Library at the Friend Center

3 ~40 m / ~130 ft

4 Back to CPU

5 CPU and Memory

Processor Memory

6 CPU, Memory, and Storage Processor Memory (DRAM)

Storage (SSD) (HDD)

7 Memory vs. Storage (High-Level)

RAM: Random Access Memory • Volatile: retains data only if receiving power • Faster • Higher cost per bit

Storage • Non-volatile: retains data even in the absence of a power source • Slower • Cheaper cost per bit

8 Ideal Memory

Memory • Zero access latency • Infinite capacity • Zero cost • Infinite bandwidth

Attribution: Gennady Pekhimenko 9 Challenge 1: Bigger is slower!

Q. How long does it take for you to open Q. How long does it take for you to open page 98 of the same book when stored on page 98 of the book you hold in hand? a shelf in this library?

10 Dynamic Random Access Memory (DRAM)

Image source: https://www.allaboutcircuits.com/technical-articles/introduction-to-dram-dynamic-random-access-memory/ 11 Dynamic Random Access Memory (DRAM)

A larger DRAM is slower: Larger decoders needed to select cells Longer paths

Image source: https://www.allaboutcircuits.com/technical-articles/introduction-to-dram-dynamic-random-access-memory/ 12 DRAM Cost Trend

Source: https://aiimpacts.org/trends-in-dram-price-per-gigabyte/ 13 Challenge 2: Faster is more expensive!

14 Static Random Access Memory (SRAM)

Much larger circuit than DRAM Much faster access time

15 DRAM vs. SRAM

16 Cost-Performance Tradeoff – Which one is better?

Fast Memory Processor (SRAM)

Low capacity

Slow Memory Processor (DRAM)

Slow

Processor

17 On-Chip vs. Off-Chip – Which one is better?

Processor

Processor

One can dedicate extra CPU transistors for having on-chip SRAM.

18 Size-Performance Tradeoff – Which one is better?

Processor

Processor

Break the same SRAM into multiple chunks and use the smaller ones more often to get performance gains.

19 Memory Hierarchy

Fast/ Small cheaper perbit cheaper faster per bit per faster

Big / Slow

Attribution: Gennady Pekhimenko 20 21 To-do Items

Ø No class on Monday.

Ø No office hour during the recess.

See you on Wednesday!

22