Comparability Graph Coloring for Optimizing Utilization of Software-Managed Stream Register Files for Stream Processors 1 Xuejun Yang School of Computer, National University of Defense Technology Li Wang School of Computer, National University of Defense Technology Jingling Xue School of Computer Science and Engineering, University of New South Wales and Qingbo Wu School of Computer, National University of Defense Technology The stream processors represent a promising alternative to traditional cache-based general-purpose processors in achieving high performance in stream applications (media and some scientific ap- plications). In a stream programming model for stream processors, an application is decomposed into a sequence of kernels operating on streams of data. During the execution of a kernel on a stream processor, all streams accessed must be communicated through a non-bypassing software- managed on-chip memory, the SRF (Stream Register File). Optimizing utilization of the scarce on-chip memory is crucial for good performance. The key insight is that the interference graphs (IGs) formed by the streams in stream applications tend to be comparability graphs or decompos- able into a set of comparability graphs. We present a compiler algorithm for finding optimal or near-optimal colorings, i.e., SRF allocations in stream IGs, by computing a maximum spanning forest of the sub-IG formed by long live ranges, if necessary. Our experimental results validate the optimality and near-optimality of our algorithm by comparing it with an ILP solver, and show that our algorithm yields improved SRF utilization over the First-Fit bin-packing algorithm, the best in the literature. 1. INTRODUCTION Hardware-managed cache has traditionally been used to bridge the ever-widening performance gap between processor and memory. Despite this great success, some deficiencies with cache are well-known. First, their complex hardware logic incurs 1Extension of Conference Paper. This journal paper has extended our earlier PPoPP'09 [Yang et al. 2009] in four directions: |A general algorithm that works for any arbitrary stream IG is presented while our earlier algorithm is limited to stream IGs decomposable into comparability graphs and a forest. |More benchmarks and more experimental evaluation are included. |A counter example is presented showing First-Fit may occasionally achieve better colorings than our algorithm. |All results are now rigorously proved. 2 · high overhead in power consumption and area. Second, their simple application- independent management strategy does not benefit from some data access charac- teristics in many applications. For example, media applications and some scientific applications exhibit producer-consumer locality with little global data reuse, which are hardly fully exploited by hardware-managed cache. Finally, their uncertain access latencies make it difficult to guarantee real-time performance. In contrast to cache, software-managed on-chip memory has advantages in area, cost, and access speed, etc [Banakar et al. 2002]. It is thus widely adopted in embedded systems (known as scratchpad memory or SPM for short), stream ar- chitectures (known as stream register file, local memory or streaming memory), and GPUs (known as shared memory in NVIDIA's new generation GPUs under its CUDA programming model). In the case of supercomputers, software-managed on-chip memory is also frequently used, especially in their accelerators. Exam- ples include Merrimac [Dally et al. 2003], Cyclops64 [Cuvillo et al. 2005], Grape- DR [Makino et al. 2007], Roadrunner [Koch 2006], and TianHe-1A (world's fastest supercomputer in TOP500 list released in November 2010). The (programmable) stream processors, such as Imagine [Owens et al. 2002], Raw [Taylor et al. 2002], Cell [Williams et al. 2006], Merrimac [Dally et al. 2003] and GPUs, represent a promising alternative in achieving high performance in media applications. In addition, stream processing is also well suited for some scientific applications [Dally et al. 2003; Williams et al. 2006; Yang et al. 2007]. In [Yang et al. 2007], we introduced the design and fabrication of FT64, the first 64-bit stream processor for scientific computing. Like Imagine [Owens et al. 2002], Cell [Williams et al. 2006] and Merrimac [Dally et al. 2003], FT64, as shown in Figure 1, can be easily mapped to the stream virtual machine architecture described in [Labonte et al. 2004]. Such stream processor executes applications that have been mapped to the stream programming model: a program is decomposed into a sequence of computation-intensive kernels that operate on streams of data elements. Kernels are compiled to VLIW microprograms to be executed on clusters of ALUs, one at a time. Streams are stored in a software-managed on-chip memory, called SRF (Stream Register File). The stream programming models, Brook [Buck et al. 2004], CUDA, StreamC/Ke- rnelC [Das et al. 2006] and StreamIt [Thies et al. 2001], which facilitate locality exploitation and bandwidth optimization, have been proven to be useful for pro- gramming stream architectures [Das et al. 2006; Yang et al. 2007; Kudlur and Mahlke 2008]. Some other research results also demonstrate their usefulness for general-purpose architectures [Gummaraju and Rosenblum 2005; Leverich et al. 2007; Gummaraju et al. 2008]. Research into advanced compiler technology for stream languages and architec- tures is still at its infancy. Among several challenges posed by stream processing for compilation [Das et al. 2006], a careful allocation of the scarce on-chip SRF becomes imperative. SRF, the nexus of a stream processor, is introduced to cap- ture the widespread producer-consumer locality in media applications to reduce expensive off-chip memory traffic. Unlike conventional register files, however, SRF is non-bypassing, namely, the input and output streams of a kernel must be all stored in the SRF when a kernel is being executed. If the data set of a kernel is · 3 (a) Chip layout. (b) Top and bottom views. (c) Basic Modules. Fig. 1: FT64 stream processor. too large to fit into the SRF, strip mining can be applied to segment some large streams into smaller strips so that the kernel can then be called to operate on one strip at a time. Alternatively, some streams can be double-buffered [Das et al. 2006] or spilled [Wang et al. 2008] until the data set of every kernel does not exceed the SRF capacity. Therefore, optimizing utilization of SRF is crucial for good perfor- mance. Presently, SRF utilization is predominantly optimized by applying First-Fit bin-packing heuristics [Das et al. 2006], which can be sub-optimal for some large applications. In this paper, we present a new compiler algorithm for optimizing utilization of SRF for stream applications. The central machinery is the traditional interference graph (IG) representation except that an IG here is a weighted (undirected) graph formed by the streams operated on by a sequence of kernels. The key discovery is that the IGs in many media applications are comparability graphs, enabling the compiler to obtain optimal colorings in polynomial time. This has motivated us to develop a new algorithm for optimizing utilization of SRF when allocating the streams in stream IGs to the SRF by comparability graph coloring. This paper makes the following main contributions: |We show that stream IGs tend to be comparability graphs, which can thus be optimally colored. |We propose to optimize utilization of SRF by comparability graph coloring and present a compiler algorithm for coloring arbitrary stream IGs through graph decompositions and maximal spanning forest computation, if necessary. |We show by experiments that our algorithm can find optimal and near-optimal colorings efficiently for well-structured media and scientific applications that are amenable to stream processing, by comparing with both an ILP-based approach and a First-Fit based approach, thereby outperforming First-Fit heuristics. The rest of this paper is organized as follows. For background information, Section 2 introduces the stream programming model and some graph theory results 4 · to provide a basis for understanding our approach. Section 3 describes the SRF management problem we solve. Section 4 casts it as a comparability graph coloring problem and presents our algorithm for solving this new formulation. Section 5 evaluates our approach. Section 6 discusses related work. Section 7 concludes the paper. 2. BACKGROUND 2.1 Stream Programming Model The stream programming model employed in FT64 is StreamC/KernelC, which is also used in the Imagine processor [Das et al. 2006]. The central idea behind stream processing is to divide an application into kernels and streams to expose its inherent locality and parallelism. As a result, an application is split into two programs, a stream program running on the host and a kernel program on the stream processor. The stream program specifies the flow of streams between kernels and initiates the execution of kernels. The kernel program executes these kernels, one at a time. 1 ddot(stream<double> u, stream<double> v, 2 UC<double> alpha) 1 double umat[M*N], vmat[M*N]; 3 { 2 double xmat[M*N]; 4 double u_tmp, v_tmp; 3 double ymat[M*N], zmat[M*N]; 5 double alpha_tmp; 4 UC<double> alpha; 5 stream<double> u(N), v(N); 6 alpha_tmp = 0; 6 stream<double> x(N); v 7 for (int i = 0; i < N; i++) { 7 stream<double> y(N), z(N); u 8 u>>u_tmp; 9 v>>v_tmp; 8 dataInit('umatix.dat', umat); ddot 10 alpha_tmp += u_tmp*v_tmp; 9 dataInit('vmatrix.dat', vmat); 11 } 10 dataInit('xmatrix.dat',