18. MEMORY HIERARCHY COS375 / ELE375 3/12/2021 Mohammad Shahrad Acknowledgements: David August Margaret Martonosi Gennady Pekhimenko Princeton Graduate College Forbes College Springdale Golf Course 1 2 Engineering Library at the Friend Center 3 ~40 m / ~130 ft 4 Back to CPU 5 CPU and Memory Processor Memory 6 CPU, Memory, and Storage Processor Memory (DRAM) Storage (SSD) (HDD) 7 Memory vs. Storage (High-Level) RAM: Random Access Memory • Volatile: retains data only if receiving power • Faster • Higher cost per bit Storage • Non-volatile: retains data even in the absence of a power source • Slower • Cheaper cost per bit 8 Ideal Memory Memory • Zero access latency • Infinite capacity • Zero cost • Infinite bandwidth Attribution: Gennady Pekhimenko 9 Challenge 1: Bigger is slower! Q. How long does it take for you to open Q. How long does it take for you to open page 98 of the same book when stored on page 98 of the book you hold in hand? a shelf in this library? 10 Dynamic Random Access Memory (DRAM) Image source: https://www.allaboutcircuits.com/technical-articles/introduction-to-dram-dynamic-random-access-memory/ 11 Dynamic Random Access Memory (DRAM) A larger DRAM is slower: Larger decoders needed to select cells Longer paths Image source: https://www.allaboutcircuits.com/technical-articles/introduction-to-dram-dynamic-random-access-memory/ 12 DRAM Cost Trend Source: https://aiimpacts.org/trends-in-dram-price-per-gigabyte/ 13 Challenge 2: Faster is more expensive! 14 Static Random Access Memory (SRAM) Much larger circuit than DRAM Much faster access time 15 DRAM vs. SRAM 16 Cost-Performance Tradeoff – Which one is better? Fast Memory Processor (SRAM) Low capacity Slow Memory Processor (DRAM) Slow Processor 17 On-Chip vs. Off-Chip – Which one is better? Processor Processor One can dedicate extra CPU transistors for having on-chip SRAM. 18 Size-Performance Tradeoff – Which one is better? Processor Processor Break the same SRAM into multiple chunks and use the smaller ones more often to get performance gains. 19 Memory Hierarchy Fast/ Small cheaper bit per faster per bit per faster Big / Slow Attribution: Gennady Pekhimenko 20 21 To-do Items Ø No class on Monday. Ø No office hour during the recess. See you on Wednesday! 22.