Download File
Total Page:16
File Type:pdf, Size:1020Kb
Finding, Measuring, and Reducing Inefficiencies in Contemporary Computer Systems Melanie Kambadur Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Graduate School of Arts and Sciences COLUMBIA UNIVERSITY 2016 c 2016 Melanie Kambadur All Rights Reserved ABSTRACT Finding, Measuring, and Reducing Inefficiencies in Contemporary Computer Systems Melanie Kambadur Computer systems have become increasingly diverse and specialized in recent years. This complexity supports a wide range of new computing uses and users, but is not without cost: it has become difficult to maintain the efficiency of contemporary general purpose comput- ing systems. Computing inefficiencies, which include nonoptimal runtimes, excessive energy use, and limits to scalability, are a serious problem that can result in an inability to ap- ply computing to solve the world's most important problems. Beyond the complexity and vast diversity of modern computing platforms and applications, a number of factors make improving general purpose efficiency challenging, including the requirement that multiple levels of the computer system stack be examined, that legacy hardware devices and soft- ware may stand in the way of achieving efficiency, and the need to balance efficiency with reusability, programmability, security, and other goals. This dissertation presents five case studies, each demonstrating different ways in which the measurement of emerging systems can provide actionable advice to help keep general purpose computing efficient. The first of the five case studies is Parallel Block Vectors, a new profiling method for understanding parallel programs with a fine-grained, code-centric perspective aids in both future hardware design and in optimizing software to map better to existing hardware. Second is a project that defines a new way of measuring application interference on a datacenter's worth of chip-multiprocessors, leading to improved schedul- ing where applications can more effectively utilize available hardware resources. Next is a project that uses the GT-Pin tool to define a method for accelerating the simulation of GPGPUs, ultimately allowing for the development of future hardware with fewer inefficien- cies. The fourth project is an experimental energy survey that compares and combines the latest energy efficiency solutions at different levels of the stack to properly evaluate the state of the art and to find paths forward for future energy efficiency research. The final project presented is NRG-Loops, a language extension that allows programs to measure and intelligently adapt their own power and energy use. Table of Contents List of Figures iv List of Tables xi 1 Introduction 1 1.1 Computing Diversity . .1 1.2 Hardware-Software Mismatches Cause Efficiency Problems . .3 1.3 Why Efficiency is (Still) Important . .4 1.4 Considerations Besides Efficiency . .5 1.5 Measurement's Role in Improving Efficiency . .7 1.6 Summary of Contributions . .8 1.7 Dissertation Outline . .9 2 Background: Measuring the Intersection of Hardware and Software 11 2.1 Computer Systems . 11 2.2 An Overview of Computing Analyses . 16 2.3 Dynamic Performance Analyses . 18 3 Parallel Block Vectors 23 3.1 Introduction . 24 3.2 Parallel Block Vector Profiles . 26 3.3 Harmony: Efficient Collection of PBVs . 30 3.4 Architectural Design Applications of PBVs . 35 3.5 Pinpointing Software Performance Issues with PBVs . 43 i 3.6 Related Work . 51 3.7 Limitations and Future Work . 52 3.8 Discussion . 54 4 Datacenter-Wide Application Interference 56 4.1 Introduction . 57 4.2 Complexities of Interference in a Datacenter . 58 4.3 A Methodology for Measuring Interference in Live Datacenters . 64 4.4 Applying the Measurement Methodology . 69 4.5 Performance Opportunities . 75 4.6 Related Work . 78 4.7 Limitations and Future Work . 80 4.8 Discussion . 81 5 Fast Computational GPGPU Design 83 5.1 Introduction . 84 5.2 Background . 86 5.3 Tracing GPU Programs with GT-Pin . 89 5.4 A Study of Large OpenCL Applications . 92 5.5 Selecting GPU Simulation Subsets . 96 5.6 Related Work . 106 5.7 Limitations and Future Work . 108 5.8 Discussion . 109 6 Energy Efficiency Across the Stack 110 6.1 Introduction . 111 6.2 Background on Energy Management . 114 6.3 Experimental Design and Methodology . 116 6.4 System-Level Results . 121 6.5 Application-Level Energy Management . 132 6.6 Related Work . 139 ii 6.7 Limitations and Future Work . 140 6.8 Discussion . 140 7 NRG-Loops 142 7.1 Introduction . 142 7.2 NRG-Loops . 147 7.3 NRG-RAPL . 150 7.4 Case Studies . 154 7.5 Related Work . 163 7.6 Limitations and Future Work . 164 7.7 Discussion . 165 8 Conclusions 166 8.1 Summary of Findings . 166 8.2 Looking Forward . 168 Bibliography 171 Appendix Acronyms 204 iii List of Figures 3.1 Parallel block vector for matrix multiplication. For each basic block in an application, top, the profile, bottom, indicates the block execution frequency at each possible thread count (i.e., degree of parallelism). ................. 28 3.2 Harmony instrumentation points. Profiler action is taken upon various runtime events. Careful engineering offloads expensive work to the least frequent events, in particular program start and finish which do not overlap with the execution of the program itself. This results in minimal profiling work at the most frequent events (i.e., basic block executions), reducing the profiling overhead and minimizing perturbation. .................................... 29 3.3 Direct instrumentation example. Each basic block is augmented to record its execution at the current degree of parallelism. The additional three instruction use only one register and do not induce any register spills. ............... 32 3.4 Thread library wrapper example. Here the instrumentation decrements and increments the effective thread count upon upon entry to and exit from of a blocking call respectively. .................................. 32 3.5 Low overhead of instrumentation. Program slowdown due to profile collection ranges from 2% to 44% with an average overhead of 18%. ............. 34 3.6 Parallel block vectors for Parsec. These heatmaps are a visualization of the profiles produced by Harmony. For the given application, they show the number of times (shading) each static block (row) was executed at each degree of parallelism (column). ...................................... 36 iv 3.7 Classifying basic blocks by parallelism. These graphs show the percentage of blocks which execute only serially (serial), blocks which execute both serially and in parallel (mixed), and blocks which only execute in parallel (parallel) for each application, for both nominal and effective thread counting, and for both static and dynamic block executions. ............................. 36 3.8 Opcode mix by class. Instruction mixes for the entire program compared with the mixes for each basic block class (serial, parallel, and mixed). In all applications, the instruction mixes for both purely serial and purely parallel blocks differ significantly from whole program mixes. ............................. 37 3.9 Memory interaction by class. The proportion of memory operations for serial and parallel basic blocks differ from the proportion in the program as a whole. .. 39 3.10 Hottest blocks are not always the most parallel blocks. Each static basic block's weighted average nominal thread count was calculated and then plotted against its total number of dynamic executions. The graphs show that the hottest blocks are primarily split between those that execute only serial and those that execute near the max degree of parallelism. .................... 41 3.11 Few basic blocks represent large portions of serial and parallel runtime. For basic blocks that were determined by parallel block vectors to always execute serially (left) or in parallel (right), percentages of runtime execution are attributed to static basic blocks. For most applications, a small number of blocks represents a large fraction of the total runtime. ......................... 42 3.12 ParaShares rank basic blocks to identify those with the greatest impact on parallel execution, weighting each block by the runtime parallelism exhibited by the application each time the block was executed. ............... 43 3.13 ParaShare rankings identify important blocks to target for multithreaded performance optimizations. These graphs show the ParaShare percentages (or- dered from greatest to least share) of all the basic blocks in eight benchmark appli- cations. ....................................... 45 v 3.14 Robustness of the metrics. Runtimes and basic block execution counts can change across program trials, but the differences are small relative to differences in ParaShares collected across varying thread counts or input sizes. ......... 48 3.15 ParaShares versus unweighted rankings in top 20 blocks. ParaShares do not always highlight new `hot' blocks, but can often significantly impact the relative importance of a block versus dynamic instruction count rankings not weighted by parallelism. ..................................... 49 3.16 ParaShares pinpoint inefficiencies that lead to significant opportunities for optimization. With the extremely targeted profiling provided by ParaShares, we were able to improve benchmark performance by up to 92% through source code changes less than 10 lines long. ........................... 50 4.1 Datacenter machines are filled with applications. Profiling 1000 12-core, 24 hyperthread Google servers running production workloads, we found the average machine had more than 14 of the 24 hyperthreads