DOCSLIB.ORG

Improving Processor Efficiency by Exploiting Common-Case Behaviors of Memory Instructions

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Improving Processor Efficiency by Exploiting Common-Case Behaviors of Memory Instructions IMPROVING PROCESSOR EFFICIENCY BY EXPLOITING COMMON-CASE BEHAVIORS OF MEMORY INSTRUCTIONS A Thesis Presented to The Academic Faculty by Samantika Subramaniam In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the School of Computer Science Georgia Institute of Technology May 2009 IMPROVING PROCESSOR EFFICIENCY BY EXPLOITING COMMON-CASE BEHAVIORS OF MEMORY INSTRUCTIONS Approved by: Professor Gabriel H. Loh, Advisor Professor Nathan Clark School of Computer Science School of Computer Science Georgia Institute of Technology Georgia Institute of Technology Professor Milos Prvulovic Professor Hsien-Hsin S. Lee School of Computer Science School of Electrical and Computer Georgia Institute of Technology Engineering Georgia Institute of Technology Professor Hyesoon Kim Aamer Jaleel, PhD School of Computer Science Intel Corporation Georgia Institute of Technology Date Approved: 10 December 2008 To my family, for their unconditional love and support iii ACKNOWLEDGEMENTS I take this opportunity to express my deepest gratitude to Professor Gabriel H. Loh, my thesis adviser for his unwavering support, immense patience and excellent guidance. Col- laborating with him has been a tremendous learning experience and has been instrumental in shaping my future research career. I will especially remember him for his endless energy and superb guidance and thank him for continually bringing out the best in me. I thank Professor Milos Prvulovic for helping me throughout this journey, by being an excellent teacher and a helpful co-author as well as for providing me very useful feedback on my research ideas, presentations, thesis and general career decisions. I thank Professor Hsien- Hsien S. Lee for his excellent insights and tips throughout our research collaboration. He has always been a wealth of information to me and has helped me become a better re- searcher. I thank Professor Hyesoon Kim for her invaluable suggestions during my thesis proposal, dissertation writing and for always responding promptly to my requests. I thank Professor Nathan Clark for agreeing to serve on my thesis committee and for his feedback to help improve my work. I thank Aamer Jaleel from Intel Corporation for taking the time to be the external member on my thesis committee. I would like to thank my friends, Kiran, Guru, Ioannis D, Rahul, Jon, Yuejien, Richard, Mrinmoy, Dong Hyuk, Dean, Gopi, Ioannis S, Carina and countless other friends and col- leagues that I have met at Georgia Tech and who have supported me throughout. I thank Susie McClain, Deborah Mitchell, Becky Wilson and Alan Glass for their assistance in ad- ministrative matters and for providing me guidance throughout my time at Georgia Tech. I thank the various program committees and reviewers for their constructive feedback on my research papers. I thank Intel Corporation for research equipment, awarding me a fel- lowship to pursue my graduate studies and for hiring me as an intern with the MTL group in Santa Clara. This experience contributed greatly to my research career. I would also like to thank Georgia Tech and the College of Computing for giving me the opportunity and iv the infrastructure to pursue my graduate studies at this prestigious university. I thank my parents, Mohana and Seetharam Subramaniam for being my pillars of strength throughout this journey. I thank my sister Menaka, my brother-in-law Vikram, my niece Kavya, my grandmother Visalakshi, my grandaunt Jayalakshmi, my mother-in-law Mangala and other family members for being tremendous sources of support and encour- agement. I thank my late grandfather, Ramachandran, for his support and blessings all my life. Finally I would like to thank Sankar, my husband, for his endless love and support throughout my graduate career. v TABLE OF CONTENTS DEDICATION . iii ACKNOWLEDGEMENTS . iv LIST OF TABLES . xi LIST OF FIGURES . xii SUMMARY . xvi I INTRODUCTION . 1 1.1 Motivation . 1 1.2 Efficiency of Current Processors . 1 1.3 Memory Instruction Processing . 2 1.3.1 Conservativeness in Memory Instruction Processing . 2 1.3.2 Scalability Concerns . 4 1.3.3 Multi-threaded Microarchitectures . 5 1.4 Memory Instruction Behaviors . 6 1.4.1 Predictability in Memory Dependencies . 6 1.4.2 Predictability in Data Forwarding . 6 1.4.3 Predictability in Instruction Criticality . 7 1.4.4 Conservative Allocation and Deallocation Policies . 8 1.5 Scope and Overview of the Dissertation . 8 1.5.1 Scope . 8 1.5.2 Overview of Dissertation . 9 II AN OVERVIEW OF MEMORY INSTRUCTION BEHAVIOR . 11 2.1 Speculative Memory Disambiguation and Memory Dependencies . 11 2.2 Load and Store Queues and Data Forwarding Patterns . 13 2.3 Instruction Criticality and Criticality-Aware Load Execution . 15 2.4 Allocation and Deallocation Constraints Imposed on Load and Store Queues 16 2.5 Applying Characteristics of Memory Behavioral Patterns to Improve Pro- cessor Efficiency . 18 vi III STORE VECTORS: DESIGNING A SCALABLE, EFFICIENT, AND HIGH- PERFORMANCE MEMORY DEPENDENCE PREDICTOR . 19 3.1 Introduction . 19 3.2 Background . 20 3.2.1 Memory Dependence Predictors . 20 3.2.2 Store Sets . 22 3.2.3 Scheduling Structures . 26 3.3 Store Vector Dependence Prediction . 27 3.3.1 Step-by-Step Operation . 28 3.3.2 Example . 30 3.4 Evaluation . 32 3.4.1 Methodology . 32 3.4.2 Performance Results . 33 3.5 Optimizations . 40 3.5.1 Sensitivity to Hardware Budget . 40 3.5.2 Reduction of Store Vector Length . 41 3.5.3 Most Recently Conflicting store . 43 3.5.4 Store Window . 44 3.5.5 Tagged SVT . 45 3.5.6 Effect of Incorporating Control Flow Information . 46 3.6 Why Do Store Vectors Work? . 47 3.7 Implementation Complexity . 50 3.8 Conclusions . 52 IV PEEP: APPLYING PREDICTABILITY OF MEMORY DEPENDENCES IN SI- MULTANEOUS MULTI-THREADED PROCESSORS . 54 4.1 Introduction . 54 4.2 Background . 56 4.2.1 SMT Fetch Policies and Resource Stalls . 56 4.2.2 Memory Dependences and SMT . 58 4.3 Adapting SMT Fetch Policies to Exploit Memory Dependences . 60 4.3.1 Proactive Exclusion . 61 vii 4.3.2 Leniency and Early Parole . 63 4.4 Evaluation . 66 4.4.1 Methodology . 66 4.4.2 Proactive Exclusion Only . 68 4.4.3 Proactive Exclusion with Early Parole (PEEP) . 69 4.4.4 Fairness Results . 72 4.5 PEEP on an SMT Processor without Speculative Memory Disambiguation 74 4.6 Sensitivity Analysis . 76 4.6.1 Sensitivity to Multithreading . 76 4.6.2 Scalability of PEEP . 78 4.6.3 PEEP Combined with Other Fetch Policies . 78 4.6.4 Delay Sensitivity . 79 4.7 Evaluating PEEP in an Industrial Simulator . 81 4.8 Conclusions . 84 V FIRE-AND-FORGET: IMPLEMENTING SCALABLE STORE TO LOAD FOR- WARDING . 85 5.1 Introduction . 85 5.2 Background . 86 5.2.1 Conventional Load and Store Queues . 86 5.2.2 Load Re-Execution . 87 5.2.3 Other Related Work . 91 5.3 Fire-and-Forget Store-to-Load Forwarding . 94 5.3.1 Load Queue Index Prediction . 94 5.3.2 Complete Store Queue Elimination . 97 5.3.3 Speculative Memory Cloaking . 99 5.3.4 Corner Cases and Loose Ends . 100 5.4 Evaluation . 100 5.4.1 Methodology . 101 5.4.2 Performance Results . 103 5.4.3 Scalability of FnF: Performance analysis over a medium configuration104 5.5 Specific Case Studies . 104 viii 5.5.1 A Good Example . 104 5.5.2 A Neutral Example . 105 5.5.3 An Extreme Example . 106 5.5.4 A Misbehaving Example . 107 5.6 Power, Area and Latency considerations . 107 5.7 Speculative Stores to Improve FnF Efficiency . 108 5.8 FnF ROB: Load/Store Scheduling without a Store Queue or a Load Queue 114 5.9 NoSQ . 117 5.10 Conclusions . 119 VI LCP: LOAD CRITICALITY PREDICTION FOR EFFICIENT LOAD INSTRUC- TION PROCESSING . 120 6.1 Introduction . 120 6.2 Background . 122 6.2.1 Instruction (and Load) Criticality . 122 6.2.2 Predicting Instruction Criticality . 123 6.3 Our Load-Criticality Predictor . 125 6.3.1 Basic Operational Description . 125 6.3.2 Example . 126 6.3.3 Issue-Rate Filtering of the Predictor . ..
Load more

Recommended publications
  • Instruction Level Parallelism -- Hardware Speculation and VLIW (Static Superscalar)
    Lecture 17: Instruction Level Parallelism -- Hardware Speculation and VLIW (Static Superscalar) CSE 564 Computer Architecture Summer 2017 Department of Computer Science and Engineering Yonghong Yan [email protected] www.secs.oakland.edu/~yan Topics for Instruction Level Parallelism § ILP Introduction, Compiler Techniques and Branch Prediction – 3.1, 3.2, 3.3 § Dynamic Scheduling (OOO) – 3.4, 3.5 and C.5, C.6 and C.7 (FP pipeline and scoreboard) § Hardware Speculation and Static Superscalar/VLIW – 3.6, 3.7 § Dynamic Scheduling, Multiple Issue and Speculation – 3.8, 3.9 § ILP Limitations and SMT – 3.10, 3.11, 3.12 2 REVIEW 3 Not Every Stage Takes only one Cycle § FP EXE Stage – Multi-cycle Add/Mul – Nonpiplined for DIV § MEM Stage 4 Issues of Multi-Cycle in Some Stages § The divide unit is not fully pipelined – structural hazards can occur » need to be detected and stall incurred. § The instructions have varying running times – the number of register writes required in a cycle can be > 1 § Instructions no longer reach WB in order – Write after write (WAW) hazards are possible » Note that write after read (WAR) hazards are not possible, since the register reads always occur in ID. § Instructions can complete in a different order than they were issued (out-of-order complete) – causing problems with exceptions § Longer latency of operations – stalls for RAW hazards will be more frequent. 5 Hazards and Forwarding for Longer- Latency Pipeline § H 6 Problems Arising From Writes § If we issue one instruction per cycle, how can we avoid structural
    [Show full text]
  • Software-Hardware Cooperative Memory Disambiguation
    Software-Hardware Cooperative Memory Disambiguation Ruke Huang, Alok Garg, and Michael Huang Department of Electrical & Computer Engineering University of Rochester fhrk1,garg,[email protected] Abstract order execution of these instructions do not violate the program se- mantics. Without the a priori knowledge of which load instructions In high-end processors, increasing the number of in-flight in- can execute out of program order and not violate program seman- structions can improve performance by overlapping useful process- tics, conventional implementations simply buffer all in-flight load ing with long-latency accesses to the main memory. Buffering these and store instructions and perform cross-comparisons during their instructions requires a tremendous amount of microarchitectural re- execution to detect all violations. The hardware uses associative sources. Unfortunately, large structures negatively impact proces- arrays with priority encoding. Such a design makes the LSQ proba- sor clock speed and energy efficiency. Thus, innovations in effective bly the least scalable of all microarchitectural structures in modern and efficient utilization of these resources are needed. In this paper, out-of-order processors. In reality, we observe that in many appli- we target the load-store queue, a dynamic memory disambiguation cations, especially array-based floating-point applications, a signif- logic that is among the least scalable structures in a modern micro- icant portion of memory instructions can be statically determined processor. We propose to use software assistance to identify load not to cause any possible violations. Based on these observations, instructions that are guaranteed not to overlap with earlier pend- we propose to use software analysis to identify certain memory in- ing stores and prevent them from competing for the resources in the structions to bypass hardware memory disambiguation.
    [Show full text]
  • MIPS, HPS, Two-Level Branch Prediction, and Compressed Code RISC Processor
    Awards ................................................................................................................................................................ Common Bonds: MIPS, HPS, Two-Level Branch Prediction, and Compressed Code RISC Processor ONUR MUTLU ETH Zurich RICH BELGARD ......We are continuing our series of of performance optimization on the com- sions made in the MIPS project that retrospectives for the 10 papers that piler and keep the hardware design sim- passed the test of time. The retrospective received the first set of MICRO Test of ple. The compiler is responsible for touches on the design tradeoffs made to Time (“ToT”) Awards in December generating and scheduling simple instruc- couple the hardware and the software, 2014.1,2 This issue features four retro- tions, which require little translation in the MIPS project’s effect on the later spectives written for six of the award- hardware to generate control signals to development of “fabless” semiconductor winning papers. We briefly introduce control the datapath components, which companies, and the use of benchmarks these papers and retrospectives and in turn keeps the hardware design simple. as a method for evaluating end-to-end hope that you will enjoy reading them as Thus, the instructions and hardware both performance of a system as, among much as we have. If anything ties these remain simple, whereas the compiler others, contributions of the MIPS project works together, it is the innovation they becomes much more important (and that have stood the test of time. delivered by taking a strong position in likely complex) because it must schedule the RISC/CISC debates of their decade. instructions well to ensure correct and High-Performance Systems We hope the IEEE Micro audience, espe- high-performance use of a simple pipe- The second retrospective addresses three cially younger generations, will find the line.
    [Show full text]
  • Memory Dependence Prediction Methods Study and Improvement Proposals
    Barcelona School of Informatics (FIB) Master in Information Technology Final Master Project Report Memory Dependence Prediction Methods Study and Improvement Proposals Otto Fernando Pflücker López March 2011 Tutor: Gladys Miriam Utrera Iglesias Supervisor: Adrián Cristal Table of Contents Abstract 6 Chapter 1. Work Presentation 7 1.1 Introduction 7 1.2 Problem Description 7 1.3 Problem Objectives 8 1.4 Time scheduling 8 1.5 Document organization 9 Chapter 2. Background 10 2.1 Introduction 10 2.2 Comparison criteria 10 2.3 Load prediction methods classification 10 2.4 Miss-prediction recovery 11 2.5 Memory dependence predictors 11 2.5.1 Store-load pair dependence predictor (1997) 11 2.5.2 Store barrier cache (1997) 12 2.5.3 Store-sets predictor (1998) 12 2.5.4 Inclusive and exclusive collision predictor (1999) 12 2.5.5 Enhanced store-sets predictor (1999) 13 2.5.6 Color sets (2002) 13 2.5.7 Store distance (2006) 13 2.5.8 Store vector (2006) 14 2.5.9 Synchronizing store-sets (2006) 14 2.5.10 Counting dependence predictors (2008) 15 2.6 Other load dependence methods 15 2.6.1 Last value predictor (1996) 15 2.6.2 Memory renaming (1997) 15 2.6.3 Stride predictor (1997) 15 2.6.4 Context predictors (2000) 16 2.7 Predictors used in manufactured architectures 16 2.7.1 The Alpha load-wait table 16 2.7.2 The P6 processor pessimistic predictor 17 2.8 Summary 17 Chapter 3. Evaluation framework 18 3.1 The M5 simulator 18 3.2 Framework configuration 18 Chapter 4.
    [Show full text]
  • CHAPTER 15 Exploiting Load/Store Parallelism Via Memory
    CHAPTER 15 Exploiting Load/Store Parallelism via Memory Dependence Prediction Since memory reads or loads are very frequent, memory latency, that is the time it takes for memory to respond to requests can impact performance significantly. Today, reading data from main memory requires more than 100 processor cycles while in 'typical” programs about one in five instructions reads from mem- ory. A naively built multi-GHz processor that executes instructions in-order would thus have to spent most of its time simply waiting for memory to respond to requests. The overall performance of such a processor would not be noticeably better than that of a processor that operated with a much slower clock (in the order of a few hundred MhZ). Clearly, increasing processor speeds alone without at the same time finding a way to make memories respond faster makes no sense. The memory latency problem can be attacked directly by reducing the time it takes memory to respond to read requests. Because it is practically impossible to build a large, fast and cost effective memory, it is impossible to make memory respond fast to all requests. How- ever, it is possible to make memory respond faster to some requests. The more requests it can process faster, the higher the overall performance. This is the goal of traditional memory hierarchies where a collection of faster but smaller memory devices, commonly referred to as caches, is used to provide faster access to a dynamically changing subset of memory data. Given the limited size of caches and imperfections in the caching policies, memory hierarchies provide only a partial solution to the memory latency problem.
    [Show full text]
  • Advanced Speculation to Increase the Performance of Superscalar Processors Kleovoulos Kalaitzidis
    Advanced speculation to increase the performance of superscalar processors Kleovoulos Kalaitzidis To cite this version: Kleovoulos Kalaitzidis. Advanced speculation to increase the performance of superscalar processors. Performance [cs.PF]. Université Rennes 1, 2020. English. NNT : 2020REN1S007. tel-03033709 HAL Id: tel-03033709 https://tel.archives-ouvertes.fr/tel-03033709 Submitted on 1 Dec 2020 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. THÈSE DE DOCTORAT DE L’UNIVERSITÉ DE RENNES 1 ÉCOLE DOCTORALE N° 601 Mathématiques et Sciences et Technologies de l’Information et de la Communication Spécialité : Informatique Par « Kleovoulos KALAITZIDIS » « Advanced Speculation to Increase the Performance of Superscalar Processors » Thèse présentée et soutenue à Rennes, le 06 Mars 2020 Unité de recherche : Inria Rennes - Bretagne Atlantique Thèse N° : Rapporteurs avant soutenance : Pascal Sainrat Professeur, Université Paul Sabatier, Toulouse Smail Niar Professeur, Université Polytechnique Hauts-de-France, Valenciennes Composition du Jury : Président : Sebastien Pillement Professeur, Université de Nantes Examinateurs : Alain Ketterlin Maître de conférence, Université Louis Pasteur, Strasbourg Angeliki Kritikakou Maître de conférence, Université de Rennes 1 Dir. de thèse : André Seznec Directeur de Recherche, Inria/IRISA Rennes Dedicated to my beloved parents, my constant inspiration in life.
    [Show full text]
  • Address-Indexed Memory Disambiguation and Store-To-Load Forwarding
    Address-Indexed Memory Disambiguation and Store-to-Load Forwarding Sam S. Stone, Kevin M. Woley, Matthew I. Frank Department of Electrical and Computer Engineering University of Illinois, Urbana-Champaign {ssstone2,woley,mif}@uiuc.edu Copyright c 2005 IEEE. Reprinted from Proceedings, 38th International Symposium on Microarchitecture, Nov 2005, 171-182. This material is posted here with permission of the IEEE. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to [email protected]. By choosing to view this document, you agree to all provisions of the copyright laws protecting it. Address-Indexed Memory Disambiguation and Store-to-Load Forwarding Sam S. Stone, Kevin M. Woley, Matthew I. Frank Department of Electrical and Computer Engineering University of Illinois, Urbana-Champaign {ssstone2,woley,mif}@uiuc.edu Abstract ward values from in-flight stores to dependent loads, and This paper describes a scalable, low-complexity alter- to buffer stores for in-order retirement. As instruction win- native to the conventional load/store queue (LSQ) for su- dows expand and the number of in-flight loads and stores perscalar processors that execute load and store instruc- increases, the latency and dynamic power consumption of tions speculatively and out-of-order prior to resolving their store-to-load forwarding and memory disambiguation be- dependences. Whereas the LSQ requires associative and come increasingly critical factors in the processor’s perfor- age-prioritized searches for each access, we propose that mance.
    [Show full text]
  • Advanced Computer Architecture II Review of 752 Iron Law Iron Law
    ECE/CS 757: Advanced Review of 752 Computer Architecture II • Iron law Instructor:Mikko H Lipasti • Beyond pipelining • Superscalar challenges Spring 2009 • Instruction flow • Register data flow University of Wisconsin‐Madison • Memory Dataflow • Modern memory interface Lecture notes based on slides created by John Shen, Mark Hill, David Wood, Guri Sohi, and Jim Smith, Natalie Enright Jerger, and probably others Iron Law Iron Law Time Processor Performance = --------------- • Instructions/Program Program – Instructions executed, not static code size – Determined by algorithm, compiler, ISA Instructions Cycles Time = X X • Cycles/Instruction Program Instruction Cycle – Determined by ISA and CPU organization (code size) (CPI) (cycle time) – Overlap among instructions reduces this term • Time/cycle Architecture --> Implementation --> Realization – Determined by technology, organization, clever circuit design Compiler Designer Processor Designer Chip Designer Our Goal Pipelined Design • Motivation: • Minimize time, which is the product, NOT – Increase throughput with little increase in hardware. isolated terms Bandwidth or Throughput = Performance • Common error to miss terms while devising optimizations • Bandwidth (BW) = no. of tasks/unit time • For a system that operates on one task at a time: – E.g. ISA change to decrease instruction count – BW = 1/delay (latency) – BUT leads to CPU organization which makes clock • BW can be increased by pipelining if many operands exist which need the same operation, i.e. many repetitions of slower the same task are to be performed. • Bottom line: terms are inter‐related • Latency required for each task remains the same or may even increase slightly. ECE 752: Advanced Computer Architecture I 1 Ideal Pipelining Example: Integer Multiplier Comb. Logic L n Gate Delay BW = ~(1/n) n n L -- Gate L -- Gate 2 Delay 2 Delay BW = ~(2/n) n n n L -- Gate L -- Gate L -- Gate 3 Delay 3 Delay 3 Delay BW = ~(3/n) • Bandwidth increases linearly with pipeline depth [Source: J.
    [Show full text]