Scaling to the End of Silicon with EDGE Architectures

Home , Processor register, TRIPS architecture

COVER FEATURE Scaling to the End of Silicon with EDGE Architectures

The TRIPS architecture is the ﬁrst instantiation of an EDGE instruction set, a new, post-RISC class of instruction set architectures intended to match semiconductor technology evolution over the next decade, scaling to new levels of power efﬁciency and high performance.

Doug Burger nstruction set architectures have long lifetimes as well as simplified control logic, an entire RISC Stephen W. because introducing a new ISA is tremendously processor could fit on a single chip. By moving to a Keckler disruptive to all aspects of a computer system. register-register architecture, RISC ISAs supported However, slowly evolving ISAs eventually aggressive pipelining and, with careful compiler Kathryn S. I become a poor match to the rapidly changing scheduling, RISC processors attained high perfor- McKinley underlying fabrication technology. When that gap mance despite their reduction in complexity. Mike Dahlin eventually grows too large, the benefits gained by Since RISC architectures, including the x86 renormalizing the architecture to match the under- equivalent with µops, are explicitly designed to sup- Lizy K. John lying technology make the pain of switching ISAs port pipelining, the unprecedented acceleration of Calvin Lin well worthwhile. clock rates—40 percent per year for well over a Charles R. Microprocessor designs are on the verge of a decade—has permitted performance scaling for the Moore post-RISC era in which companies must introduce past 20 years with only small ISA changes. For new ISAs to address the challenges that modern example, Intel has scaled its x86 line from 33 MHz James Burrill CMOS technologies pose while also exploiting the in 1990 to 3.4 GHz today—more than a 100-fold Robert G. massive levels of integration now possible. To meet increase over 14 years. Approximately half of that McDonald these challenges, we have developed a new class of increase has come from designing deeper pipelines. ISAs, called Explicit Data Graph Execution However, recent studies show that pipeline scaling William (EDGE), that will match the characteristics of semi- is nearly exhausted,1 indicating that processor Yoder, and conductor technology over the next decade. design now requires innovations beyond pipeline- the TRIPS centric ISAs. Intel’s recent cancellation of its high- Team TIME FOR A NEW ARCHITECTURAL MODEL? frequency Pentium 4 successors is further evidence The University of The “Architecture Comparisons” sidebar pro- of this imminent shift. Texas at Austin vides a detailed view of how previous architectures Future architectures must support four major evolved to match the driving technology at the time emerging technology characteristics: they were defined. In the 1970s, memory was expensive, so CISC • Pipeline depth limits mean that architects must architectures minimized state with dense instruction rely on other fine-grained concurrency mecha- encoding, variable-length instructions, and small nisms to improve performance. numbers of registers. In the 1980s, the number of • The extreme acceleration of clock speeds has devices that could fit on a single chip replaced hastened power limits; in each market, future absolute transistor count as the key limiting resource. architectures will be constrained to obtain as With a reduced number of instructions and modes much performance as possible given a hard

Comparing EDGE with historic architectures illus- ture is that RAW uses compiler-determined issue trates that architectures are never designed in a vac- order (including the inter-ALU routers), whereas uum—they borrow frequently from previous archi- the TRIPS architecture uses dynamic issue, to tol- tectures and can have many similar attributes. erate variable latencies. However, since the RAW ISA supports direct communication of a producer • VLIW: A TRIPS block resembles a 3D VLIW instruction in one tile to a consuming instruction’s instruction, with instructions filling fixed slots in a ALU on another tile, it could be viewed as a stati- rigid structure. However, the execution semantics cally scheduled variant of an EDGE architecture, differ markedly. A VLIW instruction is statically whereas TRIPS is a dynamically scheduled EDGE scheduled—the compiler guarantees when it will ISA. execute in relation to all other instructions.1 All • Dataflow machines: Classic dataflow machines, instructions in a VLIW packet must be indepen- researched heavily at MIT in the 1970s and 1980s,4 dent. The TRIPS processor is a static placement, bear considerable resemblances to intrablock exe- dynamic issue (SPDI) architecture, whereas a VLIW cution in EDGE architectures. Dataflow machines machine is a static placement, static issue (SPSI) were originally targeted at functional programming architecture. The static issue model makes VLIW languages, a suitable match because they have architectures a poor match for highly communica- ample concurrency and have side-effect free, “write- tion-dominated future technologies. Intel’s family once” memory semantics. However, in the context of EPIC architectures is a VLIW variant with simi- of a contemporary imperative language, a dataflow lar limitations. architecture would need to store and process many • Superscalar: An out-of-order RISC or x86 super- unnecessary instructions because of complex con- scalar processor and an EDGE processor traverse trol-flow conditions. EDGE architectures use con- similar dataflow graphs. However, the superscalar trol flow between blocks and support conventional graph traversal involves following renamed point- memory semantics within and across blocks, per- ers in a centralized issue window, which the hard- mitting them to run traditional imperative lan- ware constructs—adding instructions individually— guages such as C or Java, while gaining many of the at great cost to power and scalability. Despite benefits of more traditional dataflow architectures. attempts at partitioned variants,2 the scheduling • Systolic processors: Systolic arrays are a special his- scope of superscalar hardware is too constrained torical class of multidimensional array processors5 to place instructions dynamically and effectively. In that continually process the inputs fed to them. In our scheduling taxonomy, a superscalar processor contrast, EDGE processors issue the set of instruc- is a dynamic placement, dynamic issue (DPDI) tions dynamically in mapped blocks, which makes machine. them considerably more general purpose. • CMP: Many researchers have remarked that a TRIPS-like architecture resembles a chip multi- References processor (CMP) with 16 lightweight processors. 1. J.A. Fisher et al., “Parallel Processing: A Smart Compiler In an EDGE architecture, the global control maps and a Dumb Machine,” Proc. 1984 SIGPLAN Symp. irregular blocks of code to all 16 ALUs at once, and Compiler Construction, ACM Press, 1984, pp. 37-47. the instructions execute at will based on dataflow 2. K.I. Farkas et al., “The Multicluster Architecture: Reduc- order. In a 16-tile CMP, a separate program counter ing Cycle Time Through Partitioning,” Proc. 30th Ann. exists at each tile, and tiles can communicate only IEEE/ACM Int’l Symp. Microarchitecture (MICRO-30), through memory. EDGE architectures are finer- IEEE CS Press, 1997, pp. 149-159. grained than both CMPs and proposed specula- 3. G.S. Sohi, S.E. Breach, and T.N. Vijaykumar, “Multiscalar tively threaded processors.3 Processors,” Proc. 22nd Int’l Symp. Computer Architec- • RAW processors: At first glance, the TRIPS ture (ISCA 95), IEEE CS Press, 1995, pp. 414-425. microarchitecture bears similarities to the RAW 4. Arvind, “Data Flow Languages and Architecture,” Proc. architecture. A RAW processor is effectively a 2D, 8th Int’l Symp. Computer Architecture (ISCA 81), IEEE tiled static machine, with shades of a highly parti- CS Press, 1981, p. 1. tioned VLIW architecture. The main difference 5. H.T. Kung, “Why Systolic Architectures?” Computer, Jan. between a RAW processor and the TRIPS architec- 1982, pp. 37-46.

July 2004 45 power ceiling; thus, they must support TRIPS: AN EDGE ARCHITECTURE power-efﬁcient performance. Just as MIPS was an early example of a RISC • Increasing resistive delays through global ISA, the TRIPS architecture is an instantiation of An EDGE ISA on-chip wires means that future ISAs must an EDGE ISA. While other implementations of provides a richer be amenable to on-chip communication- EDGE ISAs are certainly possible, the TRIPS archi- dominated execution. tecture couples compiler-driven placement of interface • Design and mask costs will make running instructions with hardware-determined issue order between the many application types across a single to obtain high performance with good power efﬁ- compiler and the design desirable; future ISAs should sup- ciency. At the University of Texas at Austin, we are microarchitecture. port polymorphism—the ability to use building both a prototype processor and compiler their execution and memory units in dif- implementing the TRIPS architecture, to address ferent ways and modes to run diverse the above four technology-driven challenges as applications. detailed below:

The ISA in an EDGE architecture supports •To increase concurrency, the TRIPS ISA in- one main characteristic: direct instruction com- cludes an array of concurrently executing munication. Direct instruction communication arithmetic logic units (ALUs) that provide both means that the hardware delivers a producer scalable issue width and scalable instruction instruction’s output directly as an input to a con- window size; for example, increasing the sumer instruction, rather than writing it back to a processor’s out-of-order issue width from 16 shared namespace, such as a register file. Using this to 32 is trivial. direct communication from producers to con- • To attain power-efficient high performance, the sumers, instructions execute in dataflow order, with architecture amortizes the overheads of sequen- each instruction firing when its inputs are available. tial, von Neumann semantics over large, 100- In an EDGE architecture, a producer with multi- plus instruction blocks. ple consumers would specify each of those con- • Since future architectures must be heavily par- sumers explicitly, rather than writing to a single titioned, the TRIPS architecture uses compile- register that multiple consumers read, as in a RISC time instruction placement to mitigate com- architecture. munication delays, which minimizes the phys- The advantages of EDGE architectures include ical distance that operands for dependent higher exposed concurrency and more power-effi- instruction chains must travel across the chip cient execution. An EDGE ISA provides a richer and thus minimizes execution delay. interface between the compiler and the microarchi- • Because its underlying dataflow execution tecture: The ISA directly expresses the dataflow model does not presuppose a given application graph that the compiler generates internally, instead computation pattern, TRIPS offers increased of requiring the hardware to rediscover data depen- flexibility. The architecture includes config- dences dynamically at runtime, an inefficient urable memory banks, which provide a gen- approach that out-of-order RISC and CISC archi- eral-purpose, highly programmable spatial tectures currently take. computing substrate. The underlying dataflow- Today’s out-of-order issue RISC and CISC designs like execution model, in which instructions fire require many inefficient and power-hungry struc- when their operands arrive, is fundamental to tures, such as per-instruction register renaming, asso- computation, efficiently supporting vectors, ciative issue window searches, complex dynamic threads, dependence chains, or other compu- schedulers, high-bandwidth branch predictors, large tation patterns as long as the compiler can multiported register files, and complex bypass net- spatially map the pattern to the underlying exe- works. Because an EDGE architecture conveys the cution substrate. compile-time dependence graph through the ISA, the hardware does not need to rebuild that graph at run- To support conventional languages such as C, time, eliminating the need for most of those power- C++, or Fortran, the TRIPS architecture uses hungry structures. In addition, direct instruction block-atomic execution. In this model, the com- communication eliminates the majority of a con- piler groups instructions into blocks of instruc- ventional processor’s register writes, replacing them tions, each of which is fetched, executed, and with more energy-efficient delivery directly from pro- committed atomically, similar to the conventional ducing to consuming instructions. notion of transactions: A block may either be com-

46 Computer (a) TRIPS execution node (b) TRIPS processor core (c) TRIPS prototype chip 125 MHz 125 MHz DDR 250 MHz DDR (2:1)

Input ports 20 120 148 148 120 RRRR G I Clk/Tst S CW CN S Operand buffers N N N N N E E E E D I CPU 0 N 0 8 16 24 A0 500 MHz N 1 9 17 23 A1 E E E E D I N 2 10 18 24 FP N 3 11 19 25 integer H5 N 4 12 20 26 H6 E E E E D I N 5 13 21 27 H7 Secondary CPU 1 N 6 14 22 28 Router cache interface 500 MHz 64 instruction N 7 15 23 31 N buffers E E E E D I N N N N N P Output ports J Interrupts S CS CE S

Global control: D-cache banks: 8 4 120 148 148 120 40 Protocols: fill, flush, commit 16KB 2-way, 1-port, cache-line interleaved banks GMI G D 250 MHz Contains I-cache tags, TLB, 8 MSHRs, LSQ, dependence pred. per bank Total signal pin block header state, r/w instructions Supports load speculation and distributed commit count: ~1,144 125 MHz (2:1) 125 MHz branch predictor I-cache banks: DDR DDR Register banks: 16KB 2-way, 1-port L1 instruction cache banks Chip-to-chip: I 32 registers per bank x 4 threads I Each bank delivers four insts/cycle Protocol: OCN extension R C 64 static rename registers per bank Banks are slaves to global control unit tag store 64b data path each direction Dynamically forwards interblock values Memory: 4 channels: N/S/E/W Execution nodes: DDR SDRAM, PC2100 DiMMs likely 2 GB/s each direction on each channel S Single-issue ALU tile 4 channels w/ page interleave Control processor interface: E Slave side of generic memory interface Full-integer and floating-point units (no FDIV) Synopsis memory controller MacroCell P Buffers 64 instructions (8 insts x 8 blocks) per tile 2 GB/s each channel Source interrupts to get attention Runs like asynchronous memory Includes CP command handler JTAG: Protocol: IEEE 1149 J 4 channels w/ page interleave Includes scan intercept TAP controller Used for test and early low-level debug

Figure 1. TRIPS prototype microarchitecture. (a) The prototype chip contains two processing cores, each of which is a 16-wide out-of-order issue processor that can support up to 1,024 instructions in flight. Each chip also contains 2 Mbytes of integrated L2 cache, organized as 32 banks connected with a lightweight routing network, as well as external interfaces. (b) The processor core is composed of 16 execution nodes connected by a lightweight network. The compiler builds 128-instruction blocks that are organized into groups of eight instructions per execution node. (c) Each execution node contains a fully functional ALU, 64 instruction buffers, and a router connecting to the lightweight inter-ALU network. mitted entirely or rolled back; a fraction of a block The equivalent TRIPS instruction specifies only the may not be committed. Each of these TRIPS blocks targets of the add, not the source operands: is a hyperblock2 that contains up to 128 instructions, which the compiler maps to an array of exe- ADD T1, T2 cution units. The TRIPS microarchitecture behaves like a conventional processor with sequential where T1 and T2 are the physical locations semantics at the block level, with each block (assigned by the compiler) of two instructions behaving as a “megainstruction.” Inside executing dependent on the result of the add, which would blocks, however, the hardware uses a fine-grained have read the result in R1 in the RISC example dataflow model with direct instruction communi- above. Each instruction thus sends its values to the cation to execute the instructions quickly and effi- consumers, and instructions fire as soon as all of ciently. their operands arrive. TRIPS instructions do not encode their source The compiler statically determines the locations operands, as in a RISC or CISC architecture. of all instructions, setting the consumer target loca- Instead, they produce values and specify where the tion fields in each instruction appropriately. architecture must route them in the ALU array. For Dynamically, the processor microarchitecture fires example, a RISC ADD instruction adds the values each instruction as soon as it is ready. When fetch- in R2 and R3, and places the result in R1: ing and mapping a block, the processor fetches the instructions in parallel and loads them into the ADD R1, R2, R3 instruction buffers at each ALU in the array. This

July 2004 47 block mapping and execution model control logic writes all register outputs and stores eliminates the need to go through any to the register tiles and memory, then it removes TRIPS can achieve fully shared structures, including the the block from the array. Each block emits exactly power-efficient register file, for any instruction unless one branch that produces the location of the sub- out-of-order instructions are communicating across sequent block. execution across distinct blocks. The only exceptions To expose more instruction-level parallelism, the are loads and stores, which must TRIPS microarchitecture supports up to eight an extremely access a bank of the data cache and blocks executing concurrently. When the G-tile large instruction memory ordering hardware. maps a block onto the array, it also predicts the window. To demonstrate the potential of next block, fetching and mapping it as well. In EDGE instruction sets, we are build- steady state, up to eight blocks can operate con- ing a full prototype of the TRIPS currently on the TRIPS processor. The renaming architecture in silicon, a compiler that logic at the register file bank forwards register val- produces TRIPS binaries, and a limited runtime ues that one block produces directly to consumers system. in another block to improve performance further. Figure 1a is a diagram of the prototype chip, With this execution model, TRIPS can achieve which will be manufactured in 2005 in a 130-nm power-efficient out-of-order execution across an ASIC process and is expected to run at 500 MHz. extremely large instruction window because it elim- The chip contains 2 Mbytes of integrated L2 cache, inates many of the power-hungry structures found organized as 32 banks connected with a lightweight in traditional RISC implementations. The architec- routing network. Each of the chip’s two processing ture replaces the associative issue window with cores is a 16-wide out-of-order issue processor that architecturally visible instruction buffers con- can support up to 1,024 instructions in flight, mak- structed from small RAMs. The partitioned register ing TRIPS the first kiloinstruction processor to be file requires fewer ports because the processor never specified or built. writes temporary values produced and consumed As Figure 1b shows, each processing core is com- within a block to the register file. As a result, the posed of a 4 × 4 array of execution nodes connected TRIPS processor reduces register file accesses and by a lightweight network. The nodes are not rename table lookups by 70 percent, on average. processors; they are ALUs associated with buffers The microarchitecture replaces the unscalable for holding instructions. There are four register file broadcast bypass network in superscalar processors banks along the top, and four instruction and data with a point-to-point routing network. Although cache banks along the right-hand side of the core, routing networks can be slower than broadcasting, as well as four ports into the L2 cache network. the average distance that operands must travel The compiler builds 128-instruction blocks, orga- along the network is short because the compiler nized into groups of eight instructions per node at places dependent instructions on the same node or each of the 16 execution nodes. on nearby nodes. The instruction cache can provide To fetch a block of instructions, the global con- 16 instructions per cycle and only needs to predict trol tile (“G” in Figure 1b) accesses its branch pre- a branch once every eight cycles, reducing the nec- dictor, obtains the predicted block address, and essary predictor bandwidth. The TRIPS ISA thus accesses the I-cache tags in the G-tile. If the block’s amortizes the overheads of out-of-order execution address hits in the I-cache, the G-tile broadcasts the over a 128-instruction block (and with eight blocks, block address to the I-cache banks. over a 1,024-instruction window), rather than Each bank then streams the block instructions incurring them on every single instruction. for its respective row into the execution array and into the instruction buffers at each node, shown in BLOCK COMPILATION Figure 1c. Since branch predictions need occur only Figure 2 shows an example of how the TRIPS once every eight cycles, the TRIPS architecture is ISA encodes a small, simple compiled block. We effectively unconstrained by predictor or I-cache assume that the C code snippet in Figure 2a allo- bandwidth limitations. cates the input integer y to a register. Figure 2b is Each block specifies register and memory inputs the same example in MIPS-like assembly code. The and outputs. Register read instructions inject the variable x is saved in register 2. block inputs into the appropriate nodes in the exe- The TRIPS compiler constructs the dataflow cution array. Instructions in the block then execute graph shown in Figure 2c. Two read instructions in dataflow order; when the block completes, the obtain the register values and forward them to their

48 Computer Figure 2. TRIPS code read r3 [y] example. (a) In the (a) C code snippet (b) RISC assembly (c) Dataflow graph C code snippet, // y, z in registers // R0 contains 0 muli #2 the compiler // R1 contains y allocates the input x = y*2; // R4 contains z x x integer y to a of (x > 7){ // R3 contains 7 y + = 7; tgti #7 read r4 [z] register. (b) In the z = 5; muli R2, R1, 2 // x = y * 2 MIPS-like assembly } ble R2, R3, L1 // if (x > 7) code, the variable x x + = y; addi R1, R1, #7 // y += 7 mov F addi #7 T mov F movi #5 T addi R4, R0, #5 // z = 5 is saved in register y // x, z are live registers L1: add R5, R2, R1 // x += y y+7 5. (c) The compiler converts the original write r4 [z] add branch to a test instruction and (d) TRIPS instruction placement write r5 [x] uses the result to read r4 read r3 predicate the write r4 write r5 control-dependent instructions, which add muli #2 (e) TRIPS instruction block (2 x 2 x 2) appear as dotted addi #7 tgti #7 Block header lines in the dataflow read r4, [1,0,0] read r3, [0,1,1] [1,0,1] movi #5 w0: write r4 w1: write r5 graph. (d) Each × mov mov Instruction block node in the 2 2 execution node add w1 muli #2 [0,0,1] [0,1,0] Corresponding instruction positions: addi #7 [0,0,1] tgti #7 [1,0,0] [0,0,0] [1,1,0] [1,1,1] array holds up to [0,0,1] [0,1,1] two buffered NOP movi #5 w0 [0,0,0] [0,1,0] mov [0,0,1] mov w0 instructions. (e) The [1,0,1] [1,1,1] compiler generates [1,0,0] [1,1,0] in target form, which correspond to the map of instruction consuming instructions mapped on the execution source operands—they contain only the physical locations at the array. The compiler converts the original branch to locations of their dependent consumers. For exam- bottom of part (d). a test instruction and uses the result to predicate ple, when the control logic maps the block, the read the control-dependent instructions, shown as dot- instruction pulls the value out of register R4 and ted lines. By converting branches to predicates, the forwards it to the instruction located in the slot 0 compiler creates larger regions with no control flow of ALU node [1,1]. As soon as that instruction changes for more effective spatial scheduling. receives the result of the test condition and the reg- At runtime, the instructions fire in any order sub- ister value, it forwards the value back to register ject to dataflow constraints. The compiled code write instruction 0, which then places it back into eventually forwards the block’s outputs to the two the register file (R4). If the predicate has a value of write instructions and to any other block waiting for false, the instruction doesn’t fire. those values. Figure 2d shows the dataflow graph While this block requires a few extra overhead (DFG) with the block of instructions scheduled onto instructions compared to a RISC ISA, it performs a 2 × 2 execution node array, each with an ALU, fewer register file accesses—two reads and two with up to two instructions buffered per node. writes, as opposed to six reads and five writes. Since the critical path is muli→tgti→addi→add, Larger blocks typically have fewer extra overhead the compiler assigns the first two instructions to the instructions, more instruction-level parallelism, and same ALU so that they can execute with no inter- a larger ratio of register file access savings. node routing delays. No implicit ordering governs the execution of instructions other than the COMPILING FOR TRIPS dataflow arcs shown in Figure 2c, so there are no Architectures work best when the subdivision of “program order” limitations to instruction issue. labor between the compiler and the microarchitec- When both writes arrive at the register file, the con- ture matches the strengths and capabilities of each. trol logic deallocates the block and replaces it with For future technologies, current execution models another. strike the wrong balance: RISC relies too little on Figure 2e shows the actual code that a TRIPS the compiler, while VLIW relies on it too much. compiler would generate, which readers can decode RISC ISAs require the hardware to discover using the map of instruction locations at the bottom instruction-level parallelism and data dependences of Figure 2d. Instructions do not contain their dynamically. While the compiler could convey the

July 2004 49 Front end High-level transformations Hyperblock formation Code generation Scheduler • parsing • loop unrolling • if-conversion • register allocation • inserts moves • generate CFG • loop peeling, flattening • predication • block validation, cutting • places instructions • inlining • back-end optimization • generates assembly

(a) Control-flow graph (b) Unrolled CFG (c) Hyperblocks (d) Generated code (e) Scheduled code

Figure 3. EDGE optimizations in the Scale compiler. (a) A front end parses the code and constructs a control-ﬂow graph (CFG). (b) The CFG after the compiler has unrolled it three times. (c) All of the basic blocks from the unrolled if-converted loop inside a single hyperblock. (d) After inserting the predicates, the compiler generates the code and allocates registers. (e) The scheduler attempts to place independent instructions on different ALUs to increase concurrency and dependent instructions near one another to minimize routing distances and communication delays.

dependences, the ISA cannot express them, forcing To demonstrate the tractability of the compiler out-of-order superscalar architectures to waste analyses needed for EDGE architectures, we retar- energy reconstructing that information at runtime. geted the Scale research compiler3 to generate opti- VLIW architectures, conversely, put too much of mized TRIPS code. Scale is a compilation frame- a load on the compiler. They require the compiler to work written in Java that was originally designed resolve all latencies at compile time to fill instruc- for extensibility and high performance with RISC tion issue slots with independent instructions. Since architectures such as Alpha and Sparc. unanticipated runtime latencies cause the machine Scale provides classic scalar optimizations and to stall, the compiler’s ability to find independent analysis such as constant propagation, loop invari- instructions within its scheduling window deter- ant code motion, dependence analysis, and higher- mines overall performance. Since branch directions, level transformations such as inlining, loop un- memory aliasing, and cache misses are unknown at rolling, and interchange. To generate high-quality compile time, the compiler cannot generate sched- TRIPS binaries, we added transformations to gen- ules that best exploit the available parallelism in the erate large predicated hyperblocks,2 a new back face of variable latency instructions such as loads. end to generate unscheduled TRIPS code, and a For current and future technologies, EDGE archi- scheduler that maps instructions to ALUs and gen- tectures and their ISAs provide a proper division erates scheduled TRIPS assembly in which every between the compiler and architecture, matching instruction is assigned a location on the execution their responsibilities to their intrinsic capabilities, array. and making the job of each simpler and more effi- Figure 3 shows the EDGE-specific transforma- cient. Rather than packing together independent tions on the code fragment in Figure 4. Scale uses instructions like a VLIW machine, which is difficult a front end to parse the code and construct the to scale to wider issue, an EDGE compiler simply abstract syntax tree and control-flow graph (CFG), expresses the data dependences through the ISA. The shown in Figure 3a. To form large initial regions, hardware’s execution model handles dynamic events the compiler unrolls loops and inlines functions. like variable memory latencies, conditional branches, Figure 3b shows the CFG after the compiler has and the issue order of instructions, without needing unrolled the inner do-while loop three times. The to reconstruct any compile time information. compiler must take four additional TRIPS-specific An EDGE compiler has two new responsibilities steps to generate correct TRIPS binaries. in addition to those of a classic optimizing RISC com- Large hyperblock formation. The compiler exposes piler. The first is forming large blocks (hyperblocks in parallelism by creating large predicated hyper- the TRIPS architecture) that have no internal control blocks, which it can fetch en masse to fill the issue flow, thus permitting them to be scheduled as a unit. window quickly. Hyperblocks are formally defined The second is the spatial scheduling of those blocks, as single-entry, multiple-exit regions of code. in which the compiler statically assigns instructions Because they have internal control-flow transfers, in a block to ALUs in the execution array, with the hyperblocks are ideal for mapping onto a spatial goal of reducing interinstruction communication dis- substrate. Branches can only jump out of a hyper- tances and exposing parallelism. block to the start of another hyperblock, never to

50 Computer Figure 4. Modified for (i = 0; i< loopcount; i++) { code fragment from code = 0x1234; gzip (SPECINT2000). len = (i % 15) + 1; res = 0; do { another point in the same hyperblock or into the res |= code & 1; middle of another. Large hyperblocks are desirable if (res & 0xfddd) res <<= 1; because they provide a larger region for scheduling code >>= 1, and expose more concurrency. } while (––len > 0); For simplicity, the TRIPS prototype ISA supports result += res; fixed 128-instruction hyperblocks, which it fetches } at runtime to provide eight instructions at each of the 16 execution nodes. A more complex implementation might permit variable-sized blocks, mally using the predicate flow graph, which it gen- which map varying numbers of instructions to exe- erates internally after leaving static single assign- cution nodes. ment (SSA) form. After inserting the predicates, the To grow hyperblocks, Scale uses inlining, aggres- compiler generates the code and allocates registers, sive unrolling (including unrolled while loops), loop as shown in Figure 3d, where the hyperblock’s peeling/flattening, and if-conversion (a form of dataflow graph is visible. predication). Figure 3c shows all the basic blocks Generating legal hyperblocks. Scale must confirm from the unrolled and if-converted loop, placed that hyperblocks adhere to their legal restrictions inside a single hyperblock. before scheduling them onto the execution array. At runtime, many hyperblocks will contain use- The TRIPS prototype places several restrictions on less instructions from CFG nodes included in the legal hyperblocks to simplify the hardware: A hyperblock but not on the dynamically taken path, hyperblock can have no more than 128 instruc- or from loop iterations unrolled past the end of a tions, 32 loads or stores along any predicated path, loop. To reduce the fraction of unneeded instruc- 32 register inputs to the block, and 32 register out- tions in hyperblocks, Scale uses edge profiling to puts from the block. These architectural restric- guide the high-level transformations by identifying tions simplify the hardware at the expense of some both loop counts and infrequently executed basic hyperblocks being less full to avoid violating those blocks. constraints. If the compiler discovers an illegal Predicated execution. In previous predication mod- hyperblock, it splits the block into multiple blocks, els, evaluating the predicate on every conditional allocating intrablock dataflow values and predi- instruction incurs no additional cost. The TRIPS cates to registers. architecture treats a predicate as a dataflow Physical placement. After the compiler generates operand that it explicitly routes to waiting predi- the code, optimizes it, and validates the legality of cated consumers. A naive implementation would the hyperblocks, it maps instructions to ALUs and route a predicate to every instruction in a predi- converts instructions to target form, inserting copy cated basic block, creating a wide fan-out problem. operations when a producer must route a value to Fortunately, given a dependence chain of instruc- many consumers. To schedule the code, the com- tions that execute on the same predicate, the compiler assigns instructions to the ALUs on the array, piler can choose to predicate only the first limiting each ALU to at most eight instructions instruction in the chain, so the chain does not fire, from the block. The scheduler, shown at a high or only the last instructions in the chain that write level in Figure 3e, attempts to balance between two registers or memory values. Predicating the first competing goals: instruction saves power when the predicate is false. Alternatively, predicating only the last, output-pro- • placing independent instructions on different ducing instructions in a dependence chain gener- ALUs to increase concurrency, thereby reduc- ally increases both power and performance by ing the probability of two instructions com- hiding the latency of the predicate computation. peting to issue on the same ALU in the same Since the control logic detects block termination cycle; and when it receives all outputs (stores, register writes, • placing instructions near one another to min- and a branch), in the TRIPS architecture a block imize routing distances and thus communica- must emit the same number of outputs no matter tion delays. which predicated path is taken. Consequently, the TRIPS ISA supports the notion of null stores and The scheduler additionally exploits its knowledge null register writes that execute whenever a predi- of the microarchitecture by placing instructions cated store or register write does not fire. The that use the registers near the register banks and by TRIPS compiler inserts the null assignments opti- placing critical load instructions near the data cache

July 2004 51 banks. The compiler uses a classic greedy less regular, with more complex control flow. technique to choose the order of instructions EDGE architectures are well suited to both regu- A single EDGE to place, with a few additional heuristics to lar and irregular DLP codes because the compiler architecture minimize routing distances and ALU con- can map the processing pipelines for multiple data can support tention. streams to groups of processing elements, using the three main Although the past two years of compiler efficient local communication and dataflow-driven parallelism development have been labor intensive, the activation for execution. fact that we could design and implement this Three additional mechanisms provide significant classes on the functionality in Scale with a small develop- further benefits to DLP codes. If the compiler can same hardware ment team demonstrates the balance in the fit a loop body into a single block, or across a small with the same ISA. architecture. The division of responsibilities number of blocks, the processor only needs to fetch between the hardware and compiler in the those instructions once from the I-cache. Sub- TRIPS architecture is well suited to the sequent iterations can reuse prior mappings for compiler’s inherent capabilities. Scale can highly power-efficient loop execution. presently compile C and Fortran benchmarks into In its memory system, the TRIPS prototype has fully executable TRIPS binaries. a lightweight network embedded in the cache to support high-bandwidth routing to each of its 32 SUPPORTING PARALLELISM L2 banks, and it supports dynamic bank configu- Our work with the TRIPS architecture has shown ration using a cache as an explicitly addressable that a single EDGE architecture can effectively sup- scratchpad. Selectively configuring such memories port the three main parallelism classes—instruc- as scratchpads enables explicit memory manage- tion, data, and thread—on the same hardware with ment of small on-chip RAMs, similar to signal pro- the same ISA. It may be possible to leverage this cessing chips such as those in the Texas Instruments inherent flexibility to merge previously distinct mar- C6x series. Our studies show that some DLP codes kets, such as signal processing and desktop com- can benefit directly from higher local memory puting, into a single family of architectures that has bandwidth, which developers can add to a TRIPS the same or similar instruction sets and execution system by augmenting it with high-bandwidth models. Area, power, and clock speeds would dif- memory access channels between the processing ferentiate implementations to target various core and explicitly managed memory. While we will power/performance points in distinct markets. not implement these channels in the prototype, To be fully general and to exploit many types of designers may include them in subsequent imple- parallelism, the TRIPS microarchitecture must sup- mentations. port the common types of graphs and communica- In prior work, we found that adding a small tion flows across instruction-parallel, data-parallel, number of additional “universal” hardware mech- and thread-parallel applications without requiring anisms allowed DLP codes to run effectively on a too much specialized hardware support. In previous TRIPS-like processor across many domains such work, we showed how the TRIPS architecture can as network processing and cryptography, signal harvest instruction-level parallelism from desktop- processing, and scientific and graphics workloads.5 style codes with complex dependence patterns and, With this additional hardware, a TRIPS-like with only minimal additional hardware support, EDGE processor would, across a set of nine highly can also support loop-driven data-level parallel varied DLP applications, outperform the best-in- codes and threaded parallelism.4 class specialized processor for four applications, come close in two, and fall short in three. Mapping data-level parallelism Data-level parallel (DLP) applications range Mapping thread-level parallelism from high-end, scientific vector code to graphic Most future high-end processors will contain processing to low-power, embedded signal pro- some form of multithreading support. Intel’s cessing code. These applications are characterized Pentium 4 and IBM’s Power 5 both support simul- by frequent, high-iteration loops, large amounts of taneous multithreading (SMT).6 parallel arithmetic operations, and regular, high- We believe that executing a single thread per bandwidth access to data sets. EDGE processing core will often be more desir- Even though many DLP codes are extremely reg- able than simultaneously executing multiple ular, some such workloads, particularly in the threads per core because the EDGE dataflow ISA graphics and embedded domains, are becoming exposes enough parallelism to effectively use the

52 Computer core’s resources. However, some markets—partic- jobs on the TRIPS prototype efficiently, the ularly servers—have copious amounts of thread- scheduler software should configure proces- level parallelism available, and an EDGE processor sors between single-threaded mode (for most Major ISA changes should exploit the thread-level parallelism for these ILP processes and DLP processes) and mul- are traumatic applications. tithreaded mode (for processes that expose for industry, The TRIPS prototype will support simultaneous parallelism via large numbers of threads). but solving current execution of up to four threads per processing core. Further, thread scheduler policies should challenges may The TRIPS multithreading model assigns a subset account for the differing demands of various of the in-flight instruction blocks to each thread. types of processes. require a major For example, while a single thread might have eight Overall, because the underlying EDGE architectural shift. blocks of 128 instructions per block in flight, substrate provides an abstraction that maps threads in a four-thread configuration will each well to a range of different parallelism mod- have two blocks of 128 instructions in flight. els, the architecture provides a good balance The processor’s control logic moves into and out between hardware and software complexity. Hard- of multithreading mode by writing to control reg- ware extensions to support ILP, DLP, and TLP are isters that allocate blocks to threads. In addition to modest, and mechanisms and policies for switching this control functionality, each processor needs a among modes of computation are tractable from a separate copy of an architectural register file for software perspective. each active thread, as well as some per-thread iden- tifiers augmenting cache tags and other state infor- TO CMP OR NOT TO CMP? mation. The semiconductor process community has done While the TRIPS prototype maps threads across its job all too well: Computer architects face daunt- blocks, thus permitting each thread to have access ing and interlocking challenges. Reductions in fea- to all ALUs in the execution array, different EDGE ture size have provided tremendous opportunities implementations might support other mappings. by increasing transistor counts, but these advances For example, the hardware could allocate one have introduced new problems of communication thread to each column or row of ALUs, thus giving delay and power consumption. We believe that them private access to a register or cache bank. finding the solution to these fundamental issues will However, these alternative mappings would require require a major architecture shift and that EDGE more hardware support than the block-based architectures are a good match for meeting these approach used in the TRIPS prototype. challenges. Despite the advantages that EDGE architectures Software challenges offer, major ISA changes are traumatic for industry, The hardware required to support TLP and DLP especially given the complexity that systems have applications effectively on an EDGE architecture accrued since the last major shift. However, since is but a small increment over mechanisms already then many institutions and companies have devel- present to exploit ILP, making EDGE architectures oped the technology to incorporate such a new a good match for exploiting broad classes of par- architecture under the hood. For example, Trans- allelism—with high-power efficiency—on a single meta’s code morphing software dynamically trans- design. Exploiting an EDGE architecture’s flexibil- lates x86 instructions into VLIW code for its pro- ity to run heterogeneous workloads comprising a cessors. Dynamic translation to an EDGE architec- mix of single-threaded, multithreaded, and DLP ture will likely be simpler than to VLIW, making programs will require modest additional support this technology a promising candidate for solving from libraries and operating systems. ISA backward compatibility. We are beginning In the TRIPS prototype, memory banks can be work on such an effort and have already built a sim- configured as caches or as explicitly addressable ple PowerPC-to-TRIPS static binary translator. scratchpads, which lack the process-ID tags that The competing approach for future systems is allow processes to share virtually addressed caches. explicitly parallel hardware backed by paralleliz- Therefore, to context switch-out/switch-in a ing software. IBM and Sun Microsystems are both process that has activated scratchpad memory, the moving to chip multiprocessor (CMP)7 and chip TRIPS runtime system must save and restore the multithreading models in which each chip contains contents of the scratchpad and reconfigure the many processing cores and thread slots that exploit memory banks to the appropriate mode of opera- explicitly parallelized threads. Other research tion. Additionally, to run a heterogeneous mix of efforts, such as Smart Memories8 and Imagine9 at

July 2004 53 Stanford and RAW10 at MIT, support DLP work- Reeber, Karthikeyan Sankaralingam, Simha loads with the copious explicit parallelism that soft- Sethumadhavan, Premkishore Sivakumar, and ware can obtain. This camp argues that future Aaron Smith. workloads will inevitably shift to be highly paral- This research is supported by the Defense lel and that programmers or compilers will be able Advanced Research Projects Agency under con- to map the parallelism in tomorrow’s applications tracts F33615-01-C-1892 and F33615-03-C-4106, onto a simple, explicitly parallel hardware sub- NSF infrastructure grant EIA-0303609, NSF strate. Although researchers have consistently made CAREER grants CCR-9985109 and CCR- this argument over the past 30 years, the general- 9984336, two IBM University Partnership awards, purpose market has instead, every time, voted in an IBM Shared University Research grant, as well favor of larger, more powerful uniprocessor cores. as grants from the Alfred P. Sloan Foundation and The difficulty of scaling out-of-order RISC cores, the Intel Research Council. coupled with IBM’s thread-rich server target market, have together driven the emergence of CMPs such as Power 4. References 1. M.S. Hrishikesh et al., “The Optimal Logic Depth per Pipeline Stage Is 6 to 8 fo4 Inverter Delays,” Proc. DGE architectures offer an opportunity to 29th Int’l Symp. Computer Architecture (ISCA 02), scale the single-processor model further, while IEEE CS Press, 2002, pp. 14-24. E still effectively exploiting DLP and TLP when 2. S.A. Mahlke et al., “Effective Compiler Support for the software can discover it. However, because of Predicated Execution Using the Hyperblock,” Proc. the difficulty of discovering and exploiting paral- 25th Ann. IEEE/ACM Int’l Symp. Microarchitecture lelism, we expect that software will make better use (MICRO-25), IEEE CS Press, 1992, pp. 45-54. of smaller numbers of more powerful processors. 3. R.A. Chowdhury et al., “The Limits of Alias Analy- For example, 64 simple processors are much less sis for Scalar Optimizations,” Proc. ACM SIGPLAN desirable than four processors, each of which are 2004 Conf. Compiler Construction, ACM Press, eight times more powerful than the simple proces- 2004, pp. 24-38. sors. EDGE architectures appear to offer a pro- 4. K. Sankaralingam et al., “Exploiting ILP, TLP, and gressively better solution as technology scales down DLP with the Polymorphous TRIPS Architecture,” to the end of silicon, with each generation provid- Proc. 30th Int’l Symp. Computer Architecture (ISCA ing a richer spatial substrate at the expense of 03), IEEE CS Press, 2003, pp. 422-433. increased global communication delays. EDGE 5. K. Sankaralingam et al., “Universal Mechanisms for ISAs may also be a good match for postsilicon Data-Parallel Architectures,” Proc. 36th Ann. devices, which will likely be communication-dom- IEEE/ACM Int’l Symp. Microarchitecture (MICRO- inated as well. 36), IEEE CS Press, 2003, pp. 303-314. Whether EDGE architectures prove to be a com- 6. D.M. Tullsen, S.J. Eggers, and H.M. Levy, “Simulta- pelling alternative will depend on their performance neous Multithreading: Maximizing On-Chip Paral- and power consumption relative to current high- lelism,” Proc. 22nd Int’l Symp. Computer end devices. The prototype TRIPS processor and Architecture (ISCA 95), IEEE CS Press, 1995, pp. compiler will help to determine whether this is the 392-403. case. We expect to have reliable simulation results 7. K. Olukotun et al., “The Case for a Single-Chip Mul- using optimized compiled code in mid-2004, tape- tiprocessor,” Proc. 6th Int’l Conf. Architectural Sup- out in the beginning of 2005, and full chips run- port for Programming Languages and Operating ning in the lab by the end of 2005. ■ Systems (ASPLOS 94), ACM Press, 1994, pp. 2-11. 8. K. Mai et al., “Smart Memories: A Modular Recon- figurable Architecture,” Proc. 27th Int’l Symp. Com- Acknowledgments puter Architecture (ISCA 00), IEEE CS Press, 2000, We thank the following student members of the pp. 161-171. TRIPS team for their contributions as coauthors of 9. S. Rixner et al., “A Bandwidth-Efficient Architecture this article: Xia Chen, Rajagopalan Desikan, for Media Processing,” Proc. 31st Ann. IEEE/ACM Saurabh Drolia, Jon Gibson, Madhu Saravana Sibi Int’l Symp. Microarchitecture (MICRO-31), IEEE CS Govindan, Paul Gratz, Heather Hanson, Changkyu Press, 1998, pp. 3-13. Kim, Sundeep Kumar Kushwaha, Haiming Liu, 10. E. Waingold et al., “Baring It All to Software: RAW Ramadass Nagarajan, Nitya Ranganathan, Eric Machines,” Computer, Sept. 1997, pp. 86-93.

54 Computer Doug Burger, Stephen W. Keckler, Kathryn S. Charles R. Moore, a senior research fellow at the McKinley, and Mike Dahlin are associate profes- University of Texas at Austin from 2002 through sors in the Department of Computer Sciences at the 2004, is currently a Senior Fellow at Advanced Micro University of Texas at Austin. They are members Devices. Contact him at [email protected]. of the ACM and senior members of the IEEE. Con- tact them at {dburger, skeckler, mckinley, dahlin} James Burrill is a research fellow in the Computer @cs.utexas.edu. Science Department at the University of Massa- chusetts at Amherst. Contact him at burrill@cs. Lizy K. John is an associate professor in the Depart- umass.edu. ment of Electrical and Computer Engineering at the University of Texas at Austin. She is a member Robert G. McDonald is the chief engineer for the of the ACM and a senior member of the IEEE. TRIPS prototype chip at the University of Texas at Contact her at [email protected]. Austin. Contact him at [email protected].

Calvin Lin is an associate professor in the Depart- William Yoder is a research programmer at the ment of Computer Sciences at the University of University of Texas at Austin and a member of the Texas at Austin and is a member of the ACM. Con- ACM and the IEEE. Contact him at byoder@cs. tact him at [email protected]. utexas.edu.

SET wireless networks gigabit Ethernet INDUSTRY enhanced parallel ports STANDARDS FireWire 802.11 token rings

IEEE Computer Society members work together to deﬁne standards like IEEE 802, 1003, 1394, 1284, and many more.

HELP SHAPE FUTURE TECHNOLOGIES ■ JOIN AN IEEE COMPUTER SOCIETY STANDARDS WORKING GROUP AT www.computer.org/standards/

July 2004 55