DOCSLIB.ORG

Memory Hierarchy Design Overview Locality of Reference Cache

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Memory Hierarchy Design Overview Locality of Reference Cache Cache Systems CPU Main CPU Main Memory Memory 400MHz 10MHz Cache 10MHz Memory Hierarchy Design Bus 66MHz Bus 66MHz Chapter 5 and Appendix C Data object Block transfer transfer Main CPU Cache Memory 1 4 Overview Example: Two-level Hierarchy Access Time • Problem T1+T2 –CPU vs Memory performance imbalance • Solution – Driven by temporal and spatial locality – Memory hierarchies • Fast L1, L2, L3 caches • Larger but slower memories • Even larger but even T slower secondary storage 1 • Keep most of the action in the higher levels 0 Hit ratio 1 2 5 Locality of Reference Basic Cache Read Operation • Temporal and Spatial • Sequential access to memory • CPU requests contents of memory location • Unit-stride loop (cache lines = 256 bits) • Check cache for this data for (i = 1; i < 100000; i++) • If present, get from cache (fast) sum = sum + a[i]; • If not present, rea d requ ire d bloc k from main memory to cache • Non-unit stride loop (cache lines = 256 bits) • Then deliver from cache to CPU for (i = 0; i <= 100000; i = i+8) • Cache includes tags to identify which block sum = sum + a[i]; of main memory is in each cache slot 3 6 1 Elements of Cache Design Number of Caches • Cache size • Increased logic density => on-chip cache • Line (block) size – Internal cache: level 1 (L1) • Number of caches – External cache: level 2 (L2) • Unified cache • Mapping function – Balances the load between instruction and data fetches – Block placement – Only one cache needs to be designed / implemented – Block identification • Split caches (data and instruction) • Replacement Algorithm – Pipelined, parallel architectures • Write Policy 7 10 Cache Size Mapping Function • Cache size << main memory size • Cache lines << main memory blocks • Small enough • Direct mapping – Minimize cost – Maps each block into only one possible line – Speed up access (less gates to address the cache) – (block address) MOD (number of lines) – Keep cache on chip • Fully associative • Large enough – Block can be placed anywhere in the cache – Minimize average access time • Set associative • Optimum size depends on the workload – Block can be placed in a restricted set of lines • Practical size? – (block address) MOD (number of sets in cache) 8 11 Line Size Cache Addressing • Optimum size depends on workload • Small blocks do not use locality of reference Block address Block offset principle Tag Index • Larger blocks reduce the number of blocks – Replacement overhead Block offset – selects data object from the block Tag Cache Main Memory • Practical sizes? Index – selects the block set Tag – used to detect a hit 9 12 2 Direct Mapping Replacement Algorithm • Simple for direct-mapped: no choice • Random – Simple to build in hardware •LRU Associativity Two-way Four-way Eight-way Size LRU Random LRU Random LRU Random 16KB 5.18% 5.69% 4.67% 5.29% 4.39% 4.96% 64KB 1.88% 2.01% 1.54% 1.66% 1.39% 1.53% 256KB 1.15% 1.17% 1.13% 1.13% 1.12% 1.12% 13 16 Associative Mapping Write Policy • Write is more complex than read – Write and tag comparison can not proceed simultaneously – Only a portion of the line has to be updated • Write policies – Write through – write to the cache and memory – Write back – write only to the cache (dirty bit) • Write miss: – Write allocate – load block on a write miss – No-write allocate – update directly in memory 14 17 K-Way Set Associative Mapping Alpha AXP 21064 Cache CPU 21 8 5 Address Tag Index offset Data data In out Valid Tag Data (256) Write buffer =? Lower level memory 15 18 3 Write Merging Cache Performance Improvements Write address V V V V • Average memory-access time 100 1 000 = Hit time + Miss rate x Miss penalty 104 1 0 00 • Cache optimizations 108 1 0 0 0 – Reducing the miss rate 112 1 0 0 0 – Reducing the miss penalty Write address V V V V – Reducing the hit time 100 11 11 00 00 00 00 00 00 19 22 DECstation 5000 Miss Rates Example 30 Which has the lower average memory access time: 25 A 16-KB instruction cache with a 16-KB data cache or 20 A 32-KB unified cache Instr. Cache % 15 Data Cache Hit time = 1 cycle Unified 10 Miss ppyenalty = 50 cy cles Load/store hit = 2 cycles on a unified cache 5 Given: 75% of memory accesses are instruction references. 0 1 KB 2 KB 4 KB 8 KB 16 KB 32 KB 64 KB 128 KB Overall miss rate for split caches = 0.75*0.64% + 0.25*6.47% = 2.10% Cache size Miss rate for unified cache = 1.99% Average memory access times: Direct-mapped cache with 32-byte blocks Split = 0.75 * (1 + 0.0064 * 50) + 0.25 * (1 + 0.0647 * 50) = 2.05 Percentage of instruction references is 75% 20 Unified = 0.75 * (1 + 0.0199 * 50) + 0.25 * (2 + 0.0199 * 50) = 2.24 23 Cache Performance Measures Cache Performance Equations • Hit rate: fraction found in that level CPUtime = (CPU execution cycles + Mem stall cycles) * Cycle time – So high that usually talk about Miss rate Mem stall cycles = Mem accesses * Miss rate * Miss penalty – Miss rate fallacy: as MIPS to CPU performance, CPUtime = IC * (CPIexecution + Mem accesses per instr * Miss rate * • Average memory-access time Miss penalty) * Cycle time = Hit time + Miss rate x Miss penalty (ns) Misses per instr = Mem accesses per instr * Miss rate • Miss penalty: time to replace a block from lower CPUtime = IC * (CPIexecution + Misses per instr * Miss penalty) * level, including time to replace in CPU Cycle time – access time to lower level = f(latency to lower level) – transfer time: time to transfer block =f(bandwidth) 21 24 4 Reducing Miss Penalty Critical Word First and Early Restart • Multi-level Caches • Critical Word First: Request the missed • Critical Word First and Early Restart word first from memory • Priority to Read Misses over Writes • Early Restart: Fetch in normal order, but as •Merggging Write Buffers soon as the requested word arrives, send it to CPU • Victim Caches 25 28 Multi-Level Caches Giving Priority to Read Misses over Writes • Avg mem access time = Hit time(L1) + Miss Rate SW R3, 512(R0) (L1) X Miss Penalty(L1) • Miss Penalty (L1) = Hit Time (L2) + Miss Rate LW R1, 1024 (R0) (L2) X Miss Penalty (L2) LW R2, 512 (R0) • Avg mem access time = Hit Time (L1) + Miss Rate (()(L1) X (Hit Time ( ()L2) + Miss Rate ( ()L2) X • Direct-mapped, write-through cache Miss Penalty (L2) mapping 512 and 1024 to the same block • Local Miss Rate: number of misses in a cache divided by the total number of accesses to the and a four word write buffer cache • Will R2=R3? • Global Miss Rate: number of misses in a cache divided by the total number of memory accesses • Priority for Read Miss? generated by the cache 26 29 Performance of Multi-Level Caches Victim Caches 27 30 5 Reducing Miss Rates: 1. Larger Block Size Types of Cache Misses • Compulsory • Effects of larger block sizes – First reference or cold start misses – Reduction of compulsory misses • Capacity • Spatial locality – Working set is too big for the cache – Increase of miss penalty (transfer time) – Fully associative caches – Re duct ion o f num ber o f bloc ks • Conflict (collision) • Potential increase of conflict misses – Many blocks map to the same block frame (line) • Latency and bandwidth of lower-level memory – Affects – High latency and bandwidth => large block size • Set associative caches • Direct mapped caches • Small increase in miss penalty 31 34 Miss Rates: Absolute and Distribution Example 32 35 Reducing the Miss Rates 2. Larger Caches 1. Larger block size • More blocks 2. Larger Caches • Higher probability of getting the data 3. Higher associativity • Longer hit time and higher cost 4. Pseudo-associative caches • Primarily used in 2nd level caches 5. Compiler optimizations 33 36 6 3. Higher Associativity 5. Compiler Optimizations • Eight-way set associative is good enough • Avoid hardware changes • 2:1 Cache Rule: • Instructions – Miss Rate of direct mapped cache size N = – Profiling to look at conflicts between groups of Miss Rate 2-way cache size N/2 instructions • Higher Associativity can increase • Data – Clock cycle time – Loop Interchange: change nesting of loops to access data in order stored in memory – Hit time for 2-way vs. 1-way external cache +10%, – Blocking: Improve temporal locality by accessing internal + 2% “blocks” of data repeatedly vs. going down whole columns or rows 37 40 4. Pseudo-Associative Caches Loop Interchange /* Before */ • Fast hit time of direct mapped and lower conflict for (j = 0; j < 100; j = j+1) misses of 2-way set-associative cache? for (i = 0; i < 5000; i = i+1) • Divide cache: on a miss, check other half of cache x[i][j] = 2 * x[i][j]; to see if there, if so have a pseudo-hit (slow hit) /* After */ Hit time for (i = 0; i < 5000; i = i+1) for (j = 0; j < 100; j = j+1) Pseudo hit time Miss penalty x[i][j] = 2 * x[i][j]; • Drawback: •Sequential accesses instead of striding through memory every 100 words; improved spatial locality – CPU pipeline design is hard if hit takes 1 or 2 cycles – Better for caches not tied directly to processor (L2) •Same number of executed instructions – Used in MIPS R1000 L2 cache, similar in UltraSPARC 38 41 Pseudo Associative Cache Blocking (1/2) /* Before */ for (i = 0; i < N; i = i+1) CPU for (j = 0; j < N; j = j+1){ Address r = 0; Data Data for (k = 0; k < N; k = k+1) in out r = r + y[i][k]*z[k][j]; 1 1 Data x[i][j] = r; Tag }; =? •Two Inner Loops: –Read all NxN elements of z[] 3 2 2 –Read N elements of 1 row of y[] repeatedly –Write N elements of 1 row of x[] =? Write buffer •Capacity Misses a function of N & Cache Size: –3 NxNx4 => no capacity misses Lower level memory –Idea: compute on BxB submatrix that fits 39 42 7 Blocking (2/2) 2.
Load more

Recommended publications
  • Embedded DRAM
    Embedded DRAM Raviprasad Kuloor Semiconductor Research and Development Centre, Bangalore IBM Systems and Technology Group DRAM Topics Introduction to memory DRAM basics and bitcell array eDRAM operational details (case study) Noise concerns Wordline driver (WLDRV) and level translators (LT) Challenges in eDRAM Understanding Timing diagram – An example References Slide 1 Acknowledgement • John Barth, IBM SRDC for most of the slides content • Madabusi Govindarajan • Subramanian S. Iyer • Many Others Slide 2 Topics Introduction to memory DRAM basics and bitcell array eDRAM operational details (case study) Noise concerns Wordline driver (WLDRV) and level translators (LT) Challenges in eDRAM Understanding Timing diagram – An example Slide 3 Memory Classification revisited Slide 4 Motivation for a memory hierarchy – infinite memory Memory store Processor Infinitely fast Infinitely large Cycles per Instruction Number of processor clock cycles (CPI) = required per instruction CPI[ ∞ cache] Finite memory speed Memory store Processor Finite speed Infinite size CPI = CPI[∞ cache] + FCP Finite cache penalty Locality of reference – spatial and temporal Temporal If you access something now you’ll need it again soon e.g: Loops Spatial If you accessed something you’ll also need its neighbor e.g: Arrays Exploit this to divide memory into hierarchy Hit L2 L1 (Slow) Processor Miss (Fast) Hit Register Cache size impacts cycles-per-instruction Access rate reduces Slower memory is sufficient Cache size impacts cycles-per-instruction For a 5GHz
    [Show full text]
  • Chapter 6 : Memory System
    Computer Organization and Architecture Chapter 6 : Memory System Chapter – 6 Memory System 6.1 Microcomputer Memory Memory is an essential component of the microcomputer system. It stores binary instructions and datum for the microcomputer. The memory is the place where the computer holds current programs and data that are in use. None technology is optimal in satisfying the memory requirements for a computer system. Computer memory exhibits perhaps the widest range of type, technology, organization, performance and cost of any feature of a computer system. The memory unit that communicates directly with the CPU is called main memory. Devices that provide backup storage are called auxiliary memory or secondary memory. 6.2 Characteristics of memory systems The memory system can be characterised with their Location, Capacity, Unit of transfer, Access method, Performance, Physical type, Physical characteristics, Organisation. Location • Processor memory: The memory like registers is included within the processor and termed as processor memory. • Internal memory: It is often termed as main memory and resides within the CPU. • External memory: It consists of peripheral storage devices such as disk and magnetic tape that are accessible to processor via i/o controllers. Capacity • Word size: Capacity is expressed in terms of words or bytes. — The natural unit of organisation • Number of words: Common word lengths are 8, 16, 32 bits etc. — or Bytes Unit of Transfer • Internal: For internal memory, the unit of transfer is equal to the number of data lines into and out of the memory module. • External: For external memory, they are transferred in block which is larger than a word.
    [Show full text]
  • Memory Systems
    Memory Systems Patterson & Hennessey Chapter 5 1 Memory Hierarchy Primary Memory Secondary Secondary CPU Cache Main Memory Memory ---- Memory Memory (Disk) (Archival) Registers MC MM MD MA Memory Content: MC ⊆ MM ⊆ MD ⊆ MA Memory Parameters: Other Factors: •Access Time: increase with distance from CPU • Energy consumption •Cost/Bit: decrease with distance from CPU • Reliability •Capacity: increase with distance from CPU • Size/density 2 § 5.1 Introducion Memory Technology Static RAM (SRAM) 0.5ns – 2.5ns, $2000 – $5000 per GB Memory Hierarchy Dynamic RAM (DRAM) CPU 50ns – 70ns, $20 – $75 per GB Increasing distance Level 1 from the CPU in Magnetic disk access time Levels in the Level 2 5ms – 20ms, $0.20 – $2 per GB memory hierarchy Ideal memory Level n Access time of SRAM Capacity and cost/GB of disk Size of the memory at each level 3 Locality of Reference Programs access a small proportion of their address space at any time If an item is referenced: - temporal locality: it will tend to be referenced again soon - spatial locality: nearby items will tend to be referenced soon Why does code have locality? Our initial focus: two levels (upper, lower) block: minimum unit of data transferred hit: data requested is in the upper level miss: data requested is not in the upper level 4 Taking Advantage of Locality Memory hierarchy Store everything on disk Copy recently accessed (and nearby) items from disk to smaller DRAM memory Main memory Copy more recently accessed (and nearby) items from DRAM to smaller/faster SRAM memory
    [Show full text]
  • A Novel Cache Architecture for Multicore Processor Using Vlsi Sujitha .S, Anitha .N
    International Journal of Scientific & Engineering Research, Volume 4, Issue 6, June-2013 178 ISSN 2229-5518 A Novel Cache Architecture For Multicore Processor Using Vlsi Sujitha .S, Anitha .N Abstract— Memory cell design for cache memories is a major concern in current Microprocessors mainly due to its impact on energy consumption, area and access time. Cache memories dissipate an important amount of the energy budget in current microprocessors. An n-bit cache cell, namely macrocell, has been proposed. This cell combines SRAM and eDRAM technologies with the aim of reducing energy consumption while maintaining the performance. The proposed M-Cache is implemented in processor architecture and its performance is evaluated. The main objective of this project is to design a newly modified cache of Macrocell (4T cell). The speed of these cells is comparable to the speed of 6T SRAM cells. The performance of the processor architecture including this modified-cache is better than the conventional Macrocell cache design. Further this Modified Macrocell is implemented in Mobile processor architecture and compared with the conventional memory. Also a newly designed Femtocell is compared with both the macrocell and Modified m-cache cell. Index Terms— Femtocell. L1 cache, L2 cache, Macrocell, Modified M-cell, UART. —————————— —————————— 1 INTRODUCTION The cache operation is that when the CPU requests contents of memory location, it first checks cache for this data, if present, get from cache (fast), if not present, read PROCESSORS are generally able to perform operations required block from main memory to cache. Then, deliver from cache to CPU. Cache includes tags to identify which on operands faster than the access time of large capacity block of main memory is in each cache slot.
    [Show full text]
  • Memory Hierarchy Summary
    Carnegie Mellon Carnegie Mellon Today DRAM as building block for main memory The Memory Hierarchy Locality of reference Caching in the memory hierarchy Storage technologies and trends 15‐213 / 18‐213: Introduction to Computer Systems 10th Lecture, Feb 14, 2013 Instructors: Seth Copen Goldstein, Anthony Rowe, Greg Kesden 1 2 Carnegie Mellon Carnegie Mellon Byte‐Oriented Memory Organization Simple Memory Addressing Modes Normal (R) Mem[Reg[R]] • • • . Register R specifies memory address . Aha! Pointer dereferencing in C Programs refer to data by address movl (%ecx),%eax . Conceptually, envision it as a very large array of bytes . In reality, it’s not, but can think of it that way . An address is like an index into that array Displacement D(R) Mem[Reg[R]+D] . and, a pointer variable stores an address . Register R specifies start of memory region . Constant displacement D specifies offset Note: system provides private address spaces to each “process” . Think of a process as a program being executed movl 8(%ebp),%edx . So, a program can clobber its own data, but not that of others nd th From 2 lecture 3 From 5 lecture 4 Carnegie Mellon Carnegie Mellon Traditional Bus Structure Connecting Memory Read Transaction (1) CPU and Memory CPU places address A on the memory bus. A bus is a collection of parallel wires that carry address, data, and control signals. Buses are typically shared by multiple devices. Register file Load operation: movl A, %eax ALU %eax CPU chip Main memory Register file I/O bridge A 0 Bus interface ALU x A System bus Memory bus I/O Main Bus interface bridge memory 5 6 Carnegie Mellon Carnegie Mellon Memory Read Transaction (2) Memory Read Transaction (3) Main memory reads A from the memory bus, retrieves CPU read word xfrom the bus and copies it into register word x, and places it on the bus.
    [Show full text]
  • Problem 1 : Locality of Reference. [10 Points] Problem 2 : Virtual Memory
    CMU 18–746 Storage Systems Spring 2010 Solutions to Homework 0 Problem 1 : Locality of reference. [10 points] Spatial locality is the concept that objects logically “close” in location to an object in the cache are more likely to be accessed than objects logically “far.” For example, if disk block #3 is read and placed in the cache, it is more likely that disk block #4 will be referenced soon than block #31,415,926 because blocks #3 and #4 are close to each other on the disk. Temporal locality is the concept that items more recently accessed in the cache are more likely to be accessed than older, “stale” blocks. For example, if disk blocks are read and placed in the cache in the order #271, #828, #182, then it is more likely that disk block #182 will be referenced again soon than block #271. Another way of thinking of this is that a block is likely to be accessed multiple times in a short span of time. Problem 2 : Virtual memory. [20 points] (a) Swapping is the process of evicting a virtual memory page from physical memory (writing the contents of the page to a backing store [for example, a disk]) and replacing the freed memory with another page (either a newly created page, or an existing page read from the backing store). In essence, one virtual memory page is “swapped” with another. Swapping is initiated whenever all physical page frames are allocated and a virtual memory reference is made to a page that’s not in memory.
    [Show full text]
  • Week 6: Assignment Solutions
    Week 6: Assignment Solutions 1. Which of the following is true for a memory hierarchy? a. It tries to bridge the processor-memory speed gap. b. The speed of the memory level closest to the processor has the highest speed. c. The capacity of the memory level farthest away from the processor is the largest. d. It is based on the principle of locality of reference. All the answers are true. This follows from the basic definition of memory hierarchy design. The fastest and smallest memory module is closest to the processor. The overall access time of the memory system is close to the access time of the fastest memory level. This is achieved by exploiting locality of reference. 2. Which of the following statements are false? a. Temporal locality arises because of loops in a program. b. Spatial locality arises because of loops in a program. c. Temporal locality arises because of sequential instruction execution. d. Spatial locality arises because of sequential instruction execution. Correct answers are (b) and (c). Temporal locality says that an accessed word will be accessed again in the near future, and this is due to the presence of loops. Spatial locality says that if a work is accessed, then words in the neighborhood will also be accessed in the near future. This happens due to sequential program execution. 3. Assume that a read request takes 50 nsec on a cache miss and 5 nsec on a cache hit. While running a program, it is observed that 80% of the processor’s read requests result in a cache hit.
    [Show full text]
  • Slap: a Split Latency Adaptive Vliw Pipeline Architecture Which Enables On-The-Fly Variable Simd Vector-Length
    © 20XX IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. arXiv:2102.13301v1 [cs.AR] 26 Feb 2021 SLAP: A SPLIT LATENCY ADAPTIVE VLIW PIPELINE ARCHITECTURE WHICH ENABLES ON-THE-FLY VARIABLE SIMD VECTOR-LENGTH Ashish Shrivastava?y, Alan Gatherer?z, Tong Sunk, Sushma Wokhluk, Alex Chandrak ?Wireless Access Lab, Futurewei Technologies Inc ySenior Member, IEEE zFellow, IEEE ABSTRACT Over the last decade the relative latency of access to shared memory by multicore increased as wire resistance domi- nated latency and low wire density layout pushed multi-port memories farther away from their ports. Various techniques were deployed to improve average memory access latencies, such as speculative pre-fetching and branch-prediction, often leading to high variance in execution time which is unac- ceptable in real-time systems. Smart DMAs can be used to directly copy data into a layer-1 SRAM, but with over- head. The VLIW architecture, the de-facto signal-processing engine, suffers badly from a breakdown in lock-step exe- cution of scalar and vector instructions. We describe the Split Latency Adaptive Pipeline (SLAP) VLIW architecture, a cache performance improvement technology that requires zero change to object code, while removing smart DMAs and their overhead. SLAP builds on the Decoupled Access and Execute concept by 1) breaking lock-step execution of func- tional units, 2) enabling variable vector length for variable data-level parallelism, and 3) adding a novel triangular-load mechanism.
    [Show full text]
  • The Memory System
    The Memory System Fundamental Concepts Some basic concepts Maximum size of the Main Memory byte-addressable CPU-Main Memory Connection Memory Processor k-bit address bus MAR n-bit data bus Up to 2k addressable MDR locations Word length = n bits Control lines ( R / W , MFC, etc.) Some basic concepts(Contd.,) Measures for the speed of a memory: . memory access time. memory cycle time. An important design issue is to provide a computer system with as large and fast a memory as possible, within a given cost target. Several techniques to increase the effective size and speed of the memory: . Cache memory (to increase the effective speed). Virtual memory (to increase the effective size). The Memory System Semiconductor RAM memories Internal organization of memory chips Each memory cell can hold one bit of information. Memory cells are organized in the form of an array. One row is one memory word. All cells of a row are connected to a common line, known as the “word line”. Word line is connected to the address decoder. Sense/write circuits are connected to the data input/output lines of the memory chip. Internal organization of memory chips (Contd.,) 7 7 1 1 0 0 W0 • • • FF FF A 0 W • • 1 • A 1 Address • • • • • • Memory decoder cells A • • • • • • 2 • • • • • • A 3 W • • 15 • Sense / Write Sense / Write Sense / Write R / W circuit circuit circuit CS Data input /output lines: b7 b1 b0 SRAM Cell Two transistor inverters are cross connected to implement a basic flip-flop. The cell is connected to one word line and two bits lines by transistors T1 and T2 When word line is at ground level, the transistors are turned off and the latch retains its state Read operation: In order to read state of SRAM cell, the word line is activated to close switches T1 and T2.
    [Show full text]
  • 14. Caches & the Memory Hierarchy
    14. Caches & The Memory Hierarchy 6.004x Computation Structures Part 2 – Computer Architecture Copyright © 2016 MIT EECS 6.004 Computation Structures L14: Caches & The Memory Hierarchy, Slide #1 Memory Our “Computing Machine” ILL XAdr OP JT 4 3 2 1 0 PCSEL Reset RESET 1 0 We need to fetch one instruction each cycle PC 00 A Instruction Memory +4 D ID[31:0] Ra: ID[20:16] Rb: ID[15:11] Rc: ID[25:21] 0 1 RA2SEL + WASEL Ultimately RA1 RA2 XP 1 Register WD Rc: ID[25:21] WAWA data is 0 File RD1 RD2 WE WE R F Z JT loaded C: SXT(ID[15:0]) (PC+4)+4*SXT(C) from and IRQ Z ID[31:26] ASEL 1 0 1 0 BSEL results Control Logic stored to ALUFN ASEL memory BSEL A B MOE WD WE MWR ALUFN ALU MWR Data MOE OE PCSEL Memory RA2SEL Adr RD WASEL WDSEL WERF PC+4 0 1 2 WD S E L 6.004 Computation Structures L14: Caches & The Memory Hierarchy, Slide #2 Memory Technologies Technologies have vastly different tradeoffs between capacity, access latency, bandwidth, energy, and cost – … and logically, different applications Capacity Latency Cost/GB Processor Register 1000s of bits 20 ps $$$$ Datapath SRAM ~10 KB-10 MB 1-10 ns ~$1000 Memory DRAM ~10 GB 80 ns ~$10 Hierarchy Flash* ~100 GB 100 us ~$1 I/O Hard disk* ~1 TB 10 ms ~$0.10 subsystem * non-volatile (retains contents when powered off) 6.004 Computation Structures L14: Caches & The Memory Hierarchy, Slide #3 Static RAM (SRAM) Data in Drivers 6 SRAM cell Wordlines Address (horizontal) 3 Bitlines (vertical, two per cell) Address decoder 8x6 SRAM Sense array amplifiers 6 6.004 Computation Structures Data out L14: Caches & The Memory Hierarchy, Slide #4 SRAM Cell 6-MOSFET (6T) cell: – Two CMOS inverters (4 MOSFETs) forming a bistable element – Two access transistors Bistable element bitline bitline (two stable states) stores a single bit 6T SRAM Cell Vdd GND “1” GND Vdd “0” Wordline N access FETs 6.004 Computation Structures L14: Caches & The Memory Hierarchy, Slide #5 SRAM Read 1 1.
    [Show full text]
  • Unstructured Computations on Emerging Architectures
    Unstructured Computations on Emerging Architectures Dissertation by Mohammed A. Al Farhan In Partial Fulfillment of the Requirements For the Degree of Doctor of Philosophy King Abdullah University of Science and Technology Thuwal, Kingdom of Saudi Arabia May 2019 2 EXAMINATION COMMITTEE PAGE The dissertation of M. A. Al Farhan is approved by the examination committee Dissertation Committee: David E. Keyes, Chair Professor, King Abdullah University of Science and Technology Edmond Chow Associate Professor, Georgia Institute of Technology Mikhail Moshkov Professor, King Abdullah University of Science and Technology Markus Hadwiger Associate Professor, King Abdullah University of Science and Technology Hakan Bagci Associate Professor, King Abdullah University of Science and Technology 3 ©May 2019 Mohammed A. Al Farhan All Rights Reserved 4 ABSTRACT Unstructured Computations on Emerging Architectures Mohammed A. Al Farhan his dissertation describes detailed performance engineering and optimization Tof an unstructured computational aerodynamics software system with irregu- lar memory accesses on various multi- and many-core emerging high performance computing scalable architectures, which are expected to be the building blocks of energy-austere exascale systems, and on which algorithmic- and architecture-oriented optimizations are essential for achieving worthy performance. We investigate several state-of-the-practice shared-memory optimization techniques applied to key kernels for the important problem class of unstructured meshes. We illustrate
    [Show full text]
  • The Automatic Improvement of Locality in Storage Systems
    The Automatic Improvement of Locality in Storage Systems Windsor W. Hsuyz Alan Jay Smithz Honesty C. Youngy yStorage Systems Department Almaden Research Center IBM Research Division San Jose, CA 95120 fwindsor,[email protected] zComputer Science Division EECS Department University of California Berkeley, CA 94720 fwindsorh,[email protected] Report No. UCB/CSD-03-1264 July 2003 Computer Science Division (EECS) University of California Berkeley, California 94720 The Automatic Improvement of Locality in Storage Systems Windsor W. Hsuyz Alan Jay Smithz Honesty C. Youngy yStorage Systems Department zComputer Science Division Almaden Research Center EECS Department IBM Research Division University of California San Jose, CA 95120 Berkeley, CA 94720 fwindsor,[email protected] fwindsorh,[email protected] Abstract 80 Random I/O (4KB Blocks) ) s Sequential I/O Disk I/O is increasingly the performance bottleneck in r u o 60 H computer systems despite rapidly increasing disk data ( k s transfer rates. In this paper, we propose Automatic i D e r Locality-Improving Storage (ALIS), an introspective i t n 40 E storage system that automatically reorganizes selected d a disk blocks based on the dynamic reference stream to e R o t increase effective storage performance. ALIS is based e 20 m on the observations that sequential data fetch is far more i T efficient than random access, that improving seek dis- tances produces only marginal performance improve- 0 ments, and that the increasingly powerful processors and 1991 1995 1999 2003 large memories in storage systems have ample capacity Year Disk was Introduced to reorganize the data layout and redirect the accesses so as to take advantage of rapid sequential data trans- Figure 1: Time Needed to Read an Entire Disk as a fer.
    [Show full text]