Exploiting Compiler-Generated Schedules for Energy Savings in High-Performance Processors Madhavi Valluri Lizy John Heather Hanson Laboratory for Computer Laboratory for Computer Computer Architecture and Architecture Architecture Technology Laboratory The University of Texas at The University of Texas at The University of Texas at Austin Austin Austin Austin, TX 78712 Austin, TX 78712 Austin, TX 78712 [email protected] [email protected] [email protected] ABSTRACT reorder buffer in addition to complex logic circuitry required This paper develops a technique that uniquely combines for the wake-up and select of instructions, the dynamic is- the advantages of static scheduling and dynamic schedul- sue hardware consumes a significant amount of the micro- ing to reduce the energy consumed in modern superscalar processor's energy budget. The energy consumption of the processors with out-of-order issue logic. In this Hybrid- dynamic issue hardware accounts for nearly 30% of the over- Scheduling paradigm, regions of the application containing all energy in existing processors [2][4][7], and is projected to large amounts of parallelism visible at compile-time com- grow even further with increasing issue widths and window pletely bypass the dynamic scheduling logic and execute sizes [20]. A recent study showing the power distribution es- in a low power static mode. Simulation studies using the timation for the Alpha 21464 microprocessor shows that the Wattch framework on several media and scientific bench- issue logic is responsible for nearly 46% of the total power marks demonstrate large improvements in overall energy dissipation on the chip [18]. consumption of 43% in kernels and 25% in full applications In processors with dynamic scheduling logic, the hardware with only a 2.8% performance degradation on average. searches for parallel instructions, irrespective of whether the compiler-generated schedule is perfect or not. The com- plex, power-hungry dynamic scheduling hardware is per- Categories and Subject Descriptors fectly justifiable in many applications with irregular control C.1 [Processor Architectures]: RISC/CISC, VLIW Ar- structures, where it is difficult for the compiler to derive chitectures compact schedules due to unpredictable branches and small basic blocks. However, for regular and well-structured pro- General Terms grams such as media and scientific applications, the compiler is easily able to generate efficient schedules for considerable Performance, Design portions of the code. The dependence analysis and schedul- ing performed by the hardware is completely redundant in Keywords these regions. Therefore for these regular regions, a large amount of energy is being expended repeatedly in the dy- Low Energy, Instruction-Level Parallelism, Dynamic Issue namic issue hardware for unnecessary work. Processors, Very Long Instruction Word Architectures We present a novel technique that combines the advan- tages of both compile-time static scheduling and run-time 1. INTRODUCTION dynamic scheduling to lower the energy consumption in a A large portion of the energy consumed in modern super- processor. In this Hybrid-Scheduling paradigm, regions of scalar processors such as the DEC Alpha 21264, Pentium code containing large amounts of parallelism that can be Pro, Pentium 4, HAL SPARC64, HP PA-8000 etc, can be identified and exploited at compile-time bypass the out-of- attributed to the dynamic scheduling hardware (or out-of- order issue logic and are issued and executed exactly in the order issue logic) responsible for identifying multiple instruc- order prescribed by the compiler. The processor runs in the tions to issue in parallel. Comprising of highly associative superscalar mode with dynamic scheduling until a special and multi-ported queues such as the instruction window and instruction that indicates the beginning of a statically sched- uled (S-Region) is detected, at which point, the out-of-order issue engine is shut down, and the processor switches to a VLIW-like static mode in which instruction packets sched- Permission to make digital or hard copies of all or part of this work for uled by the compiler are issued sequentially in consecutive personal or classroom use is granted without fee provided that copies are cycles with minimal hardware support. Energy is conserved not made or distributed for profit or commercial advantage and that copies primarily by reducing the work done in the out-of-order issue bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific logic. permission and/or a fee. The hybrid-scheduling scheme is particularly suited for ISLPED August 25–27, 2003, Seoul, Korea high-performance general-purpose systems which need to Copyright 2003 ACM 1-58113-682-X/03/0008 ...$5.00. The S-Buffer: All instructions following the \start-static" Program ALU instruction, i.e S-Region instructions are decoded and stored in the S-Buffer (shown in Figure 1). The instructions are ALU placed into the buffer before register renaming is performed. The S-Buffer, similar to a decoded instruction cache [8][12], Issue Reorder is a circular buffer that stores instructions in the decoded Window MUL Buffer Decode form. For instructions fetched from this buffer we need not DIV fully repeat the decoding step; only register read is per- formed. A detailed structure is shown in Figure 2. Each S-Buffer line holds one instruction packet. The packet size FP is fixed and can be determined at design time. The maxi- mum packet size is the number of function units available in the processor. Each S-Buffer line also contains a PC field where the program counter value of the first instruction in Exception Handling the group is stored, and two special bits (S-bit and B-bit) which are set if the line holds the first instruction of an S- S−Buffer Unit Region or if it holds a branch instruction respectively. The buffer also maintains a pointer to the next available entry Figure 1: The Hybrid-Scheduling Microarchitecture for filling and consecutive instruction packets are placed in consecutive lines of the S-Buffer. Instructions are placed un- cater to diverse application domains such as integer, me- til the \end-static" instruction is detected indicating that all dia and scientific applications. It is important to tune the the instructions belonging to the S-Region have been cap- system to the varying needs of the diverse applications. The tured in the buffer. hybrid-scheduling architecture thus allows us to use aggres- sive and power-hungry scheduling hardware for applications (a) S− Buffer (b) PC Table (c) Branch Table PC SB # V IB # Target SB # PC that warrant it, while facilitating low energy execution for PC S B Instr1 .................... Instr N structured applications that do not need such hardware. bit bit We evaluate the effectiveness of this scheme for several : : : media and scientific benchmarks using the Wattch 1.0 [2] : simulator. Our results reveal that we can achieve large : improvements in overall energy consumption amounting to 43% in kernels and 25% in full applications with minimal performance degradation. Figure 2: Structures used in the static mode The rest of the paper is organized as follows: The details of microarchitecture supporting the hybrid-scheduling scheme Switching to Static Mode: After the S-Region has been cap- are described in Section 2. The compiler support required tured within the S-Buffer, fetch from the instruction cache for this scheme is described in Section 3. Our experimental is stalled and the processor prepares to switch to the static framework and benchmarks used are explained in Section 4. mode of issue. S-Region instructions are scheduled assum- Section 5 presents our simulation results. Related work is ing that all the function units are available for use in the discussed in Section 6 and finally, we present concluding static mode and that all the register values are available remarks in Section 7. in the register file, implying that the S-Region instructions can begin execution only after the last instruction in the 2. THE HYBRID-SCHEDULING MICROAR- superscalar mode has completed. Therefore, static mode is- sue can begin immediately provided the superscalar pipeline CHITECTURE has drained while the S-Region was being captured. If not, The hybrid-scheduling microarchitecture is shown in Fig- we wait a few additional cycles for the superscalar pipeline ure 1. In this architecture, statically scheduled regions or to drain completely. Note that since we have to wait until S-Regions execute in a low power static mode. S-Region the superscalar pipeline is empty before we can start issue instructions are scheduled by the compiler into groups or in the static mode, there is a cycle-time overhead incurred packets of independent instructions that can be issued in in switching between the two modes. However, if we choose parallel. The compiler also annotates regions with special S-Regions such that they execute for long durations in the \start-static" and \end-static" instructions to indicate the static mode, the switching cost is amortized, leading to neg- beginning and the end of S-Regions. Once an S-region is de- ligible performance degradation. After the last instruction tected, instructions in the region can be issued to the func- in the superscalar mode has completed, the dynamic issue tion units without any dynamic dependence checks. logic is turned off and instructions begin issuing from the S-Buffer. Initial Program Execution: In this scheme, the program be- gins execution in the normal superscalar fashion, i.e. de- Instruction Execution in Static Mode: Instructions from the coded instructions are dispatched to the instruction window S-Buffer are issued to the functional units without any fur- where they wait for their operands, ready instructions are ther dependence analysis. One complete S-Buffer line is is- issued to the function units and finally instructions retire sued every cycle.