UC Berkeley UC Berkeley Electronic Theses and Dissertations Title Bounds on the Energy Consumption of Computational Kernels Permalink https://escholarship.org/uc/item/4v32d7hh Author Gearhart, Andrew Scott Publication Date 2014 Peer reviewed|Thesis/dissertation eScholarship.org Powered by the California Digital Library University of California Bounds on the Energy Consumption of Computational Kernels by Andrew Scott Gearhart A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Computer Science and the Designated Emphasis in Computational Science and Engineering in the Graduate Division of the University of California, Berkeley Committee in charge: Professor James W. Demmel, Chair Professor Katherine A. Yelick Professor Tarek I. Zohdi Fall 2014 Bounds on the Energy Consumption of Computational Kernels Copyright 2014 by Andrew Scott Gearhart 1 Abstract Bounds on the Energy Consumption of Computational Kernels by Andrew Scott Gearhart Doctor of Philosophy in Computer Science and the Designated Emphasis in Computational Science and Engineering University of California, Berkeley Professor James W. Demmel, Chair As computing devices evolve with successive technology generations, many machines target either the mobile or high-performance computing/datacenter environments. In both of these form factors, energy consumption often represents the limiting factor on hardware and software effi- ciency. On mobile devices, limitations in battery technology may reduce possible hardware ca- pability due to a tight energy budget. On the other hand, large machines such as datacenters and supercomputers have budgets directly related to energy consumption and small improvements in energy efficiency can significantly reduce operating costs. Such challenges have influenced re- search upon the impact of applications, operating and runtime systems upon energy consumption. Until recently, little consideration was given to the potential energy efficiency of algorithms them- selves. A dominant idea within the high-performance computing (HPC) community is that applications can be decomposed into a set of key computational problems, called kernels. Via automatic perfor- mance tuning and new algorithms for many kernels, researchers have successfully demonstrated performance improvements on a wide variety of machines. Motivated by the large and increas- ingly growing dominant cost (in time and energy) of moving data, algorithmic improvements have been attained by proving lower bounds on the data movement required to solve a computational problem, and then developing communication-optimal algorithms that attain these bounds. This thesis extends previous research on communication bounds and computational kernels by presenting bounds on the energy consumption of a large class of algorithms. These bounds apply to sequential, distributed parallel and heterogeneous machine models and we detail methods to further extend these models to larger classes of machines. We argue that the energy consumption of computational kernels is usually predictable and can be modeled via linear models with a handful of terms. Thus, these energy models (and the accompanying bounds) may apply to many HPC applications when used in composition. 2 Given energy bounds, we analyze the implications of such results under additional constraints, such as an upper bound on runtime, and also suggest directions for future research that may aid future development of a hardware/software co-tuning process. Further, we present a new model of energy efficiency, Cityscape, that allows hardware designers to quickly target areas for improve- ment in hardware attributes. We believe that combining our bounds with other models of energy consumption may provide a useful method for such co-tuning; i.e. to enable algorithm and hard- ware architects to develop provably energy-optimal algorithms on customized hardware platforms. i Now this is not the end. It is not even the beginning of the end. But it is, perhaps, the end of the beginning. - Sir Winston Churchill, 1942 ii Contents Contents ii List of Figures v List of Tables vii 1 Introduction 1 1.1 Communication Now Dominates Performance Costs . 1 1.2 Energy Efficiency at the Algorithm Level . 2 1.3 Thesis Goals and Contributions . 2 1.4 Thesis Organization . 3 2 Energy Consumption and Computing 5 2.1 Power vs. Energy . 6 2.2 Phase-based Execution of Applications . 7 2.3 Key Consumers of Energy on Desktops and Server Nodes . 9 Energy Consumption in CMOS Logic . 10 Other Hardware Components . 14 2.4 Network Energy Consumption on Distributed Parallel Machines . 15 3 Machine Models for Runtime and Energy 17 3.1 Problems, Algorithms, and Implementations . 17 3.2 Machine Models . 18 Sequential Machine Model (S) . 19 Distributed Parallel Machine Model 1 (DP1) . 21 Model Compositions and Distributed Parallel Model 2 (DP2) . 21 Heterogeneous Machine Model (H) . 23 3.3 Problems of Particular Focus . 25 Matrix-vector multiplication . 26 Matrix-matrix Multiplication . 29 O(n2) n-body problem . 35 3.4 Model Validation . 37 iii Performance Counter Measurement . 37 Measuring Power and Energy . 40 Sequential Model (S) . 40 Fitting the Model via Least Squares . 42 Distributed Parallel Models . 53 Heterogeneous Model . 53 3.5 Parameter Estimation for Machines and Implementations and Related Work . 58 4 Bounds on Communication, Runtime and Energy for Specific Algorithms 61 4.1 Communication Lower Bounds for Sequential and Distributed Parallel Machines . 61 Lower Bounds on the DP Models that Include Link Contention . 64 4.2 Energy Lower Bounds for Specific Algorithms . 69 O(n3) Classical Matrix Multiplication . 69 Strassen and Strassen-like Matrix Multiplication . 72 Matrix-vector multiplication . 74 O(n2) n-body problem . 75 4.3 Bounds on Heterogeneous Machines . 78 Input/Output Dominated Lower Bounds . 80 Loomis-Whitney Dominated Lower Bound . 82 4.4 Optimal Heterogeneous Algorithms . 84 Heterogeneous Matrix-Vector Multiplication . 84 Heterogeneous O(n3) Matrix-Matrix Multiplication . 86 5 Bounds on Communication, Runtime and Energy for Programs that Access Arrays 90 5.1 Bounds on Programs that Reference Arrays . 90 Sequential Model . 95 Distributed Parallel Model 1 . 95 Distributed Parallel Model 2 . 96 Heterogeneous Model . 97 Example: Energy Lower Bound for Matrix-matrix Multiplication . 98 5.2 Perfect Strong Scaling in the Distributed Machine Model . 99 6 Applications of Bounds on Specific Machine Models 105 6.1 Overview . 105 6.2 Example Machines for Analysis . 108 6.3 Classical O(n3) Matrix-matrix Multiplication . 111 6.4 O(n2) n-body problem . 117 6.5 Programs that access arrays with subsets of the iteration variables . 122 7 Implications for Hardware Designs 130 7.1 Introduction . 130 7.2 Cityscape Model of Energy Efficiency . 132 iv 7.3 Financial cost/Job (Cjob) ...............................136 7.4 Further Directions . 141 8 Conclusions 147 Bibliography 150 v List of Figures 2.1 Tradeoff between minimizing energy or power . 6 2.2 Typical Scientific Code Power Trace . 8 2.3 Power phases of matmul benchmark on Sandy Bridge-EP . 8 2.4 Power phases of heterogeneous matmul on Sandy Bridge-EP and Nvidia K20 . 9 2.5 CMOS inverter with n-type (nMOS) and p-type (pMOS) transistors indicated . 11 2.6 Input/Output wattage curves for a Dell DH350E-S0 power supply for 2U servers [1] . 15 3.1 Relationship between hardware, algorithm and implementation . 18 3.2 Serial (S) and Distributed Parallel (DP) machine models . 19 3.3 Composition of Sequential (S) and Distributed Parallel (DP) Machine Models . 22 3.4 Heterogeneous machine model . 24 3.5 Compressed Sparse Row (CSR) storage format . 28 3.6 Two-dimensional block cyclic distribution of a matrix on a 2-by-2 processor grid . 31 3.7 Processor grids for 3D and 2.5D matrix-matrix multiplication [126] . 32 3.8 Breadth-First or Depth-First traversals of recursion tree [98] . 35 3.9 Data layouts for 1D, 1.5D and 2D n-body algorithms . 36 3.10 O(n2) n-body algorithm with and without a cutoff distance . 37 3.11 Counting cache misses during array copy on Sandy Bridge-EP . 39 3.12 Inaccurate floating point operation counts on Sandy Bridge-EP . 39 3.13 Typical wall power sample windows for several sparse matrix-vector multiplication problems . 41 3.14 Sandy Bridge-EP: Flop/Word ratios for double-precision sparse matrix-vector multi- plication (DSPMV) . 45 3.15 Sandy Bridge-EP: Modeled (no row scaling) double-precision matrix-matrix multipli- cation (DGEMM) . 49 3.16 Sandy Bridge-EP: Modeled (no row scaling) double-precision dense matrix-vector multiplication (DGEMV) . 50 3.17 Sandy Bridge-EP: Modeled (no row scaling) double-precision sparse matrix-vector multiplication (DSPMV) . 51 3.18 Heterogeneous machine for validation . 53 3.19 Runtime impact of scaling either Host or GPU SGEMM size . 54 3.20 Runtime impact of scaling either Host or GPU SGEMV size . 56 vi 4.1 Communication bounds for Strassen’s algorithm on d-dimensional tori. The lower plot is log-log, while the upper is linear on the y-axis. Horizontal lines in the lower plot correspond to perfect strong scaling. 67 4.2 Example of heterogeneous matrix-vector data partitioning with 4 processors . 85 4.3 Heterogeneous matrix-matrix computation example execution on 4 processors . 89 5.1 Relationship between the per-processor and contention communication lower bounds, with labels on each region indicating lower bound dominance. F and M are constants. 93 6.1 Energy costs as node count and memory are scaled . 106 6.2 Effect of constraints on energy efficiency . 107 6.3 2.5D O(n3) Matrix-matrix Multiplication: Effect of replicating memory on energy efficiency . 117 6.4 CA O(n2) n-body: Effect of replicating memory on energy efficiency . 123 6.5 3-Body Problem: Effect of replicating memory on energy efficiency . 129 7.1 Example Cityscape Model for O(n3) Matrix-matrix multiplication . 135 7.2 Cjob with various parameter sets for Algorithm 11 . 142 7.3 Sequential Machine with 3 levels of fast memory . 143 vii List of Tables 3.1 Computational motifs as described in [9] .