Models and Metrics to Enable Energy-Efficiency Optimizations Suzanne Rivoire, Stanford University Mehul A. Shah and Parthasarathy Ranganathan, Hewlett-Packard Laboratories Christos Kozyrakis, Stanford University Justin Meza, University of California, Los Angeles Power consumption and energy efficiency are important factors in the initial design and day-to-day management of computer systems.Researchers and system designers need benchmarks that characterize energy efficiency to evaluate systems and identify promising new technologies.To predict the effects of new designs and configurations,they also need accurate methods of modeling power consumption. n recent years, the power consumption of servers • circuit techniques such as disabling the clock signal and data centers has become a major concern. to a processor’s unused parts; According to the US Environmental Protection • architectural techniques such as replacing complex Agency, enterprise power consumption in the US uniprocessors with multiple simple cores; and doubled between 2000 and 2006 (www.energystar. • support for multiple low-power states in processors, Igov/ia/partners/prod_development/downloads/EPA_ memory, and disks. Datacenter_Report_Congress_Final1.pdf), and will dou- ble again in the next five years. Server power consump- At the system level, the latter approach requires poli- tion not only directly affects a data center’s electricity cies to intelligently exploit these low-power states for costs, but also necessitates the purchase and operation energy savings. Across multiple systems in a cluster or of cooling equipment, which can consume from data center, these policies can involve dynamically adapt- one-half to one watt for every watt of server power ing workload placement or power provisioning to meet consumption. specific energy or thermal goals.1 All of these power-related costs can potentially exceed To facilitate these optimizations, we need metrics to the cost of purchasing hardware. Moreover, the envi- define energy efficiency, which will help designers com- ronmental impact of data center power consumption is pare designs and identify promising energy-efficient tech- receiving increasing attention, as is the effect of escalat- nologies. We also need models to predict the effects of ing power densities on the ability to pack machines into dynamic power management policies, particularly over a data center.1 many systems. Unlike the significant body of work on The two major and complementary ways to approach power management and optimization, there has been this problem involve building energy efficiency into the relatively little focus on metrics and models. initial design of components and systems, and adaptively We address the challenges in defining metrics for energy managing the power consumption of systems or groups efficiency with a specific case study on JouleSort, which of systems in response to changing conditions in the provides a complete, full-system benchmark for energy workload or environment. Examples of the former efficiency across a variety of system classes.2 The approach include “Approaches to Power Modeling in Computer Systems” 0018-9162/07/$25.00 © 2007 IEEE P u b l i s h e d b y t h e I E E E C o m p u t e r S o c i e t y December 2007 39 Approaches to Power Modeling in Computer Systems Power models are fundamental to energy-efficiency Power models used in simulators of proposed hard- research, whether the goal is to improve the compo- ware trade speed and portability for increased accuracy, nents’ and systems’ design or to efficiently use existing relying on detailed knowledge of component architec- hardware. Developers use these models offline to evalu- ture and circuit technology. Wattch1 is a widely used ate proposed designs, and online in policies to exploit CPU power model that estimates the power costs of component power modes within a system, or to effi- accessing different parts of the processor and combines ciently distribute work across several systems. this information with activity counts from a performance An ideal power model has several properties. First, it simulator to yield power estimates. Similar models have must be accurate. It should also be portable across a been proposed for other components, including mem- variety of existing and future hardware designs, and ory, disks, and networking, as well as complete applicable under a variety of workloads. Finally, it should systems.2,3 These simulators are highly accurate, but also be cost-effective in its hardware and software require- closely tied to specific systems and simulation infrastruc- ments and execute swiftly. tures that are much slower than actual hardware. Models used in online power- management policies, for which 8 speed is a first-class constraint, cannot rely on such detailed simu- 7 ) t n lation. Using real-time system e c r 6 events instead of simulated activity e p ( r counts addresses this drawback. o 5 r r Frank Bellosa proposed using e ppred, t = 14.45 + 23.61 * ucpu, t – 0.85 * umem, t + 22.32 * udisk, t + 0.03 * unet, t n o 4 i processor performance counter t c i d registers to provide on-the-fly e r 3 p power characterization of real sys- e g 4 a 2 tems. His simple and portable r e v model used the counts of instruc- A 1 tions executed and memory 0 accesses to drive the selection of a Matrix Stream SPECjbb SPECweb SPECint SPECfp processor’s frequency states. More Benchmark detailed and processor-specific performance-counter-based mod- Figure A.Accuracy of the model created by Mantis9 for a low-power blade.The equation els have been developed to model shows the power predicted at a given time t as a function of CPU utilization,number of both power5 and thermal6 proper- memory and disk accesses,and number of network accesses sampled at that time.Each uti- ties. Finally, since researchers devel- lization input u is given as a percentage of its maximum value.The average error of this oped performance counter options coarse-grained linear model is less than 10 percent for every benchmark—sufficient for with application profiling rather most scheduling optimizations. than power estimation in mind, sidebar describes different approaches to modeling power in embedded processors (EnergyBench) to evaluating the consumption in components, systems, and data centers. efficiency of data center cooling and power provisioning (Green Grid metrics). However, only JouleSort specifies ENERGY-EFFICIENCY METRICS a workload, a metric to compare two systems, and rules An ideal benchmark for energy efficiency would consist for running the benchmark. of a universally relevant workload, a metric that balances At the processor level, Ricardo Gonzalez and Mark power and performance in a universally appropriate way, Horowitz argued in 19963 that the energy-delay prod- and rules that provide impossible-to-circumvent, fair uct provides the appropriate metric for comparing two comparisons. Since this benchmark is impossible, several designs. They observed that a chip’s performance and different approaches have addressed pieces of the energy- power consumption are both related to the clock fre- efficiency evaluation problem, from chips to data centers. quency, with performance directly proportional to, and Table 1 summarizes these approaches. power consumption increasing quadratically with, Each metric in Table 1 addresses a particular energy- clock frequency. Comparing processors based on related problem, from minimizing power consumption energy, which is the product of execution time and 40 Computer Ismail Kadayif and colleagues proposed an interface work for Architectural-Level Power Analysis and Optimiza- based on “energy counters” that would virtualize the tions,” Proc. Int’l Symp. Computer Architecture (ISCA), ACM existing performance counters.7 Press, 2000, pp. 83-94. Processor performance counters can be used to esti- 2. W. Ye et al., “The Design and Use of SimplePower: A Cycle- mate processor and memory power consumption, but Accurate Energy Estimation Tool,” Proc. Design Automation do not take other parts of the system, such as I/O, into Conf. (DAC), IEEE CS Press, 2000, pp. 340-345. account. Some optimizations, such as data-center-level 3. S. Gurumurthi et al., “Using Complete Machine Simulation optimizations that turn off unused machines, must for Software Power Estimation: The SoftWatt Approach,” consider the full-system power. In this case, OS utiliza- Proc. High-Performance Computer Architecture Symp. (HPCA), tion metrics can be used to model the base system IEEE CS Press, 2002, pp. 141-150. components quickly, portably, and with reasonable 4. F. Bellosa, “The Benefits of Event-Driven Energy Accounting accuracy. Taliver Heath and colleagues8 and Dimitris in Power-Sensitive Systems,” Proc. SIGOPS European Work- Economou and colleagues9 build linear models based shop, ACM Press, 2000, pp. 37-42. on OS-reported utilization of each component. Both 5. G. Contreras and M. Martonosi, “Power Prediction for Intel approaches require an initial calibration phase, in XScale Processors Using Performance Monitoring Unit which developers connect the system to a power meter Events,” Proc. Int’l Symp. Low-Power Electronics and Design and run microbenchmarks to stress each component. (ISLPED), ACM Press, 2005, pp. 221-226. They then fit the utilization data to the power measure- 6. K. Skadron et al., “Temperature-Aware Microarchitecture: ments to construct a model. Modeling and Implementation,” ACM Trans. Architecture Figure A shows an example of one such model.9 and Code Optimization, vol. 1, no. 1, 2004, pp. 94-125. Parthasarathy Ranganathan and Phil Leech used a similar 7. I. Kadayif et al., “vEC: Virtual Energy Counters,” Proc. PASTE, approach to predict both power and performance by ACM Press, 2001, pp. 28-31. constructing lookup tables based on utilization.10 Finally, 8. T. Heath et al., “Energy Conservation in Heterogeneous researchers from Google found that an even simpler Server Clusters,” Proc. ACM SIGPLAN Symp. Principles and model, based solely on OS-reported processor utiliza- Practice of Parallel Programming (PpoPP), ACM Press, 2005, tion, proved sufficiently accurate to enable optimizations pp.