THE VECTOR-THREAD ARCHITECTURE THE VECTOR-THREAD (VT) ARCHITECTURE SUPPORTS A SEAMLESS INTERMINGLING OF VECTOR AND MULTITHREADED COMPUTATION TO FLEXIBLY AND COMPACTLY ENCODE APPLICATION PARALLELISM AND LOCALITY. VT PROCESSORS EXPLOIT THIS ENCODING TO PROVIDE HIGH PERFORMANCE WITH LOW POWER AND SMALL AREA. As embedded computing applica- purpose processors are flexible, they are often tions become more sophisticated, the demand too large, too slow, or burn too much power for high-performance, low-power information for embedded computing. In particular, mod- processing grows. Custom circuits provide the ern superscalars expend considerable hardware optimal solution for any processing task, but resources to dynamically extract limited par- rising mask and development costs limit appli- allelism from sequential encodings of unstruc- cation-specific chips to devices that will be sold tured applications. All-purpose processors Ronny Krashinsky in very high volumes. Even then, hardwired have a different goal. Embedded applications circuits are unsuitable for designs that must often contain abundant structured paral- Christopher Batten rapidly adapt to changing requirements. Pro- lelism, where dependencies can be determined grammable domain-specific processors are a statically. The challenge is to develop an Mark Hampton more flexible alternative, and they have evolved instruction set architecture that flexibly to exploit particular forms of parallelism com- encodes parallel dependency graphs and Steve Gerding mon to certain classes of embedded applica- improves the performance-efficiency of tions. Examples include digital signal processor implementations. Brian Pharris processors, media processors, network proces- sors, and field-programmable gate arrays. Full Vector-thread architectural paradigm Jared Casper systems, however, often require a heteroge- The vector-thread (VT) architectural para- neous mix of these cores to be competitive on digm describes a class of architectures that Krste Asanovi´c complex workloads with a variety of process- unify the vector and multithreaded execution ing tasks. The resulting devices have multiple models. VT architectures compactly encode Massachusetts Institute of instruction sets with different parallel execu- large amounts of structured parallelism in a tion and synchronization models, making form that lets simple microarchitectures attain Technology them difficult to program. They are also inef- high performance at low power by avoiding ficient when the application workload causes complex control and datapath structures and load imbalance across the heterogeneous cores. by reducing activity on long wires. Ideally, a single all-purpose programmable architecture would efficiently exploit the dif- Abstract model ferent types of parallelism and locality present A VT programming model combines vec- in embedded applications. Although general- tor and multithreaded computation. A con- 84 Published by the IEEE Computer Society 0272-1732/04/$20.00 2004 IEEE Control vector-fetch processor cross-VP pop VP0 VP0VP1 VP0VP2 VP0VPN thread- vector- cross-VP load fetch push Regs Regs Regs Regs vector- store cross-VP ALUs ALUs ALUs ALUs queue Memory Figure 1. Abstract model of a vector-thread architecture. A control processor interacts with a vector of virtual processors (VPs). Control Control processor VP0 VP1 VP2 VPN processor VP0 VP1 VP2 VPN vector-load ld ld ld ld vector-load ld ld ld ld vector-load ld ld ld ld vector-fetch vector-fetch ld ld ld ld Atomic + + + + == == == == instruction block (AIB) >> >> >> >> br br br br vector-store st st st st ld ld ld thread- (a) == fetch == == Control br br br processor VP0 VP1 VP2 VPN ld vector-load ld ld ld ld == vector-fetch + + + + br cross-VP queue >> >> >> >> vector-store st st st st vector-store st st st st (b) (c) Figure 2. Mapping code to the vector-thread architecture: vectorizable loops (a), loops with cross-iteration dependencies (b), and loops with internal control flow (c). ventional control processor interacts with a loop iteration, and the control processor strip- vector of virtual processors (VPs), as Figure 1 mines the execution and factors out common shows. A virtual processor contains a set of bookkeeping overhead. To execute data-par- registers and can execute groups of RISC-like allel code, the control processor uses vector- instructions packaged into atomic instruction fetch commands to broadcast AIBs to all VPs, blocks (AIBs). VPs have no automatic pro- as Figure 2a shows. The control processor can gram counter or implicit instruction fetch also use vector-load and vector-store com- mechanism; rather, all instruction blocks must mands to efficiently move vectors of data be explicitly requested by either the control between memory and the VP registers. To processor or the VP itself. allow efficient mapping of loop-carried depen- Although we can map applications to VT dencies to VT, VPs are connected in a uni- in several ways, VT is especially well suited to directional ring topology, with pairs of sending executing loops. Each VP executes a single and receiving instructions transferring data NOVEMBER–DECEMBER 2004 85 MICRO TOP PICKS Command Control processor Lane 0 Lane 1 Lane 2 Lane 3 Command management unit Command management unit Command management unit Command management unit cmd-Q thread-fetch cmd-Q thread-fetch cmd-Q thread-fetch cmd-Q thread-fetch addr. AIB AIB AIB AIB tags tags tags tags miss Execution cluster Execution cluster Execution cluster Execution cluster VP12 VP13 VP14 VP15 Execute VP8 Execute VP9 Execute VP10 Execute VP11 cross-VP directive directive directive directive queue VP4 VP5 VP6 VP7 VP0 VP1 VP2 VP3 AIB fill unit VP VP VP VP Vector AIB AIB AIB AIB ALU ALU ALU ALU memory cache cache cache cache unit Level-1 cache Figure 3. Physical model of a VT machine. The implementation shown has four parallel lanes in the vector-thread unit (VTU), and VPs are striped across the lane array. directly between them, as Figure 2b illustrates. Physical model In contrast to software pipelining on VLIW A VT machine contains a conventional architectures, the compiler or programmer control processor and a vector-thread unit need only schedule code for one loop iteration (VTU) that executes the VP code, as Figure 3 mapped to one VP, and the hardware dynam- illustrates. To exploit the parallelism exposed ically schedules the cross-VP data transfers to by the VT abstract model, the VTU contains resolve when the data becomes available. a parallel array of processing lanes. Each lane To execute loop iterations with condition- contains physical registers, which hold the als or even inner loops, each VP directs its own state of VPs mapped to the lane, and func- control flow using thread fetches to fetch its tional units, which are time-multiplexed AIBs, as Figure 2c shows. By allowing soft- across the VPs. ware to freely intermingle vector and thread Unlike traditional vector machines, in a VT fetches, a VT architecture can combine the machine the lanes execute decoupled from best attributes of the vector and multithread- each other, and each lane has a small AIB ed execution paradigms. The control proces- cache. A lane’s command management unit sor can issue a vector-fetch command to (CMU) buffers commands from the control launch a vector of VP threads, each of which processor in a queue (cmd-Q) and holds continues to execute as long as the VP issues pending thread-fetch addresses for the lane’s thread fetches. Thread fetches break the rigid VPs. The CMU chooses a vector-fetch or control flow of traditional vector machines, thread-fetch command to process, and looks letting the VPs follow independent control up its address in the AIB cache tags. After pro- paths. By letting VPs conditionally branch, cessing an AIB cache hit or miss refill, the VT provides more efficient execution of large CMU generates an execute directive contain- conditionals than traditional vector masking. ing an index to the AIB cache. For a vector- Thread fetches also let outer-loop parallelism fetch command, the execute directive map naturally to VT, as VPs can execute indicates that all VPs should execute the AIB; inner-loop iterations that have little or no for a thread-fetch command, it identifies a sin- available parallelism. In addition to executing gle VP to execute the AIB. The CMU sends loop iterations, VPs can serve as free-running execute directives to a queue, and it can over- threads, operating independently from the lap the AIB cache refill for new fetch com- control processor and retrieving tasks from a mands with the execution of previous ones. shared work queue. To process an execute directive, the execu- 86 IEEE MICRO tion cluster reads VP instructions one by one use a conventional cache to amplify memory from the AIB cache and executes them for the bandwidth and tolerate long memory laten- appropriate VP. When processing an execute cies, and avoid complicating the software directive from a vector-fetch command, the interface with a hierarchical vector (or stream) execution cluster executes all of the instruc- register file.5,6 tions in the AIB for one VP before moving on To evaluate Scale’s performance and flexi- to the next. bility, we mapped a diverse selection of embedded benchmarks (including examples Scale VT processor from cryptography and image, audio, and net- The Scale architecture, an instantiation of work processing) from the Embedded Micro- the VT paradigm, aims to provide high per- processor Benchmark Consortium (EEMBC) formance at low power for a wide range of and other suites. The results in Tables 1 and embedded applications while using only a 2 are based on a detailed execution-driven small area. The planned Scale prototype simulator,1 and they represent a snapshot of includes a MIPS-RISC control processor, 32 our ongoing progress in optimizing the Kbytes of cache, and a four-lane vector-thread microarchitecture and benchmark mappings. unit that can execute 16 operations per cycle The simulator modeled a four-lane Scale con- and supports up to 128 simultaneously active figuration running at 400 MHz with a 64-bit virtual processor threads.