Sound, Fine-Grained Traversal Fusion for Heterogeneous Trees∗ Extended version
Laith Sakka, Kirshanthan Ryan R. Newton Milind Kulkarni Sundararajah Computer Science Electrical and Computer Engineering Electrical and Computer Engineering Indiana University Purdue University Purdue University [email protected] [email protected] {lsakka,ksundar}@purdue.edu Abstract writing these applications in the most performant manner— Applications in many domains are based on a series of traver- where many AST traversals must be fused into a single tra- sals of tree structures, and fusing these traversals together to versal to reduce the overhead of traversing and manipulating reduce the total number of passes over the tree is a common, potentially-large programs [21]. important optimization technique. In applications such as In an attempt to balance these competing concerns, there compilers and render trees, these trees are heterogeneous: has been prior work on compiler and software-engineering different nodes of the tree have different types. Unfortu- techniques for writing simple, fine-grained tree traversal nately, prior work for fusing traversals falls short in different passes that are then automatically fused into coarse-grained ways: they do not handle heterogeneity; they require using passes for performance reasons [19–21, 23–25]. In the world domain-specific languages to express an application; they of functional programs, deforestation techniques rewrite data rely on the programmer to aver that fusing traversals is safe, structure operations to avoid materializing intermediate without any soundness guarantee; or they can only perform data structures, either through syntactic rewrites [5, 29] or coarse-grain fusion, leading to missed fusion opportunities. through the use of special combinators that promote fu- This paper addresses these shortcomings to build a frame- sion [7]. For web browsers, render-tree passes can be ex- work for fusing traversals of heterogeneous trees that is pressed in high-level, attribute grammar–like languages [19] automatic, sound, and fine-grained. We show across several and then passed to a compiler that generates fused passes [20]. case studies that our approach is able to allow programmers For solving differential and integral equations, computations to write simple, intuitive traversals, and then automatically on spatial functions can be expressed using high-level nu- fuse them to substantially improve performance. merical operators [24] that are then fused together into com- bined kd-tree passes by domain-specific compilers23 [ , 24]. Keywords Fusion, Tree traversals In compilers, AST-rewriting passes can be restructured us- ing special miniphase operations that are then combined (as directed by the programmer) into larger AST phases that 1 Introduction perform multiple rewrites at once [21]. Many applications are built around traversals of tree struc- Previous approaches rely on programmers using special- tures: from compilers, where abstract syntax trees represent purpose languages or programming styles to express tree the syntactic structure a program and traversals of those traversals, limiting generality, TreeFuser, by Sakka et al., ASTs are used to analyze and rewrite code; to web browsers offers an alternative [25]. Programmers write generic tree arXiv:1904.07061v1 [cs.PL] 11 Apr 2019 and layout engines, where render trees express the structure traversals in an imperative language—a subset of C—with no of documents and traversals of those trees determine the restrictions on how trees are traversed (unlike Rajbhandari location and appearance of elements on web pages and docu- et al. who limit computations to binary trees traversed in ments; to solving integral and differential equation of multi- pre- or post-order [23, 24], or Petrashko et al. who require dimensional spatial functions where kd-trees are used to rep- very specific traversal structures in miniphases [21]). Tree- resent piecewise functions and operations on those functions Fuser analyzes the dependence structure of the general tree are implemented as tree traversals. There is a fundamental traversals to perform call-specific partial fusion, which al- tension between writing these applications in the most er- lows parts of traversals to be fused (unlike other prior work), gonomic manner—where, for example, a compiler is written and integrate code motion which implicitly restructures the as dozens of individual AST-rewriting passes [21, 27]—and order of traversals (e.g., transforming a post-order traversal into a pre-order traversal) to maximize fusion opportunities. TreeFuser hence represents the most general extant fusion ∗Extended version of “Sound Fine-Grained Traversal Fusion for Heteroge- framework for imperative tree traversals. neous Trees,” Sakka et al., PLDI 2019[26]. 1 Unfortunately, TreeFuser suffers from several key limita- Table 1. Grafter in comparison to prior work. Note that tions that prevent it from fulfilling the goal of letting pro- Grafter provides finer-grained fusion than TreeFuser. We grammers write idiomatic, simple tree traversals while rely- exclude syntactic rewrites of functional programs (e.g., [29]), ing on a compiler to automatically generate coarse-grained as their rewrites are not directly analogous to fusion. efficient traversals. First, TreeFuser’s dependence represen- tation requires that trees be homogeneous: each node in the Approach Hetero- Fine- General Depen- tree must be the same data type. This means that to sup- geneous grained express- dence trees fusion ivity analysis port trees, such as abstract syntax trees, that are naturally Stream fusion [7] ✓ ✗ ✗ NA heterogeneous, TreeFuser requires programmers to unify all Attribute grammars [20] ✓ ✗ ✗ ✓ the subtypes of a class hierarchy into a single type—e.g., a Miniphases [21] ✓ ✗ ✗ ✗ tagged union—distinguishing between them with condition- Rajbhandari et al. [23] ✗ ✗ ✗ ✗ ✗ ✓ ✓ ✓ als. Second, TreeFuser does not support mutual recursion— TreeFuser [25] Grafter ✓ ✓ ✓ ✓ traversals written as a set of functions, rather than a single one—requiring the use of many conditionals to handle dif- ferent behaviors. As a corollary of not supporting heteroge- 3. Grafter generalizes prior work’s call-specific par- neous trees or mutual recursion, TreeFuser does not support tial fusion to incorporate type-specific partial fusion. virtual functions, a key feature that, among other things, This allows traversals to be fused for some node types allows complex traversals to be decomposed into operations but not others, yielding more opportunities for fusion. on individual node types. These limitations require express- Moreover, by leveraging type-specific partial fusion ing traversals with unnatural code and produce spurious and dynamic dispatch, Grafter’s code generator pro- dependences, that can inhibit fusion. Finally, TreeFuser does duces simpler, more efficient fused traversal code than not support tree topology mutation. While fields within nodes prior approaches, resulting in not just fewer passes in a tree can be updated in TreeFuser traversals, the topology over the traversed tree, but fewer instructions total. of the tree must be read-only. This makes it unnatural to Table1 summarizes Grafter’s capabilities in relation to express some AST rewrites (by, e.g., changing a field in a prior work. At a high level, prior work either supports het- node to mark it as deleted, instead of simply removing the erogeneous trees but not fine-grained fusion or vice versa, node) and impossible to express others. while Grafter supports both, as well as the ability to write general traversals, analyze dependences and perform sound 1.1 Contributions fusion automatically. This paper presents a new fusion framework, Grafter, that We show across several case studies that Grafter is able addresses these limitations to support a more idiomatic style to deliver substantial performance benefits by fusing to- of writing tree traversals, getting closer to the goal of fusing gether simple, ergonomic implementations of tree traversals truly general tree traversals. The specific contributions this into complex, optimized, coarse-grained traversals. paper makes are: 1.2 Outline 1. Grafter provides support for heterogeneous types. The remainder of the paper is organized as follows. Section2 Rather than requiring that each node in the tree share provides an overview of our fusion framework Grafter. Sec- the same type (as in prior work on fusing general tion3 lays out the design of Grafter: the types of traversals traversals), Grafter allows recursive fields of a node it supports, how it represents dependences given the more in a tree to have any type, enabling the expression of complex traversals, how it performs type-directed fusion, complex heterogeneous tree structures such as ASTs and how it synthesizes the final fused traversal(s). Section4 and render trees. details the prototype implementation of Grafter. Section5 2. To further support heterogeneous tree types, Grafter evaluates Grafter across several case studies. Section6 supports mutual recursion and virtual functions, allow- discusses related work and Section7 concludes. ing children of nodes to be given static types that are resolved to specific subtypes at runtime, more closely 2 Grafter Overview matching the natural way that traversals of heteroge- Grafter adopts a strategy for fusing traversals where a neous trees are written. This support requires devel- programmer writes individual tree traversals in a standard, oping a new dependence representation that enables C++-like language (Section 3.1), as functions that traverse precise dependence tests in the presence of mutual a tree structure. The fields of each tree node can be het- recursion and dynamic dispatch. Grafter also adds erogeneous, and part of a complex class hierarchy, and the support in its dependence representation for accom- (mutually) recursive functions that perform a tree traversal modating insertion and deletion of nodes in the tree. can leverage dynamic dispatch when visiting children of a 2 f12 f12
root.f1(); root.f12(); root.f2(); child.f3() S1 f1() S1 f1(){ child.f3(); S2 this.x = …; // s1 } S2 f12(){
f2(){ f2() child.f3() this.x = …; // s1 … = this.x; // s2 … = this.x; // s2 child.f4(); child.f34(); child.f4() } child.f4() }
(a) Unfused code (b) Outline and inline step (c) Reorder step (d) Fused code
Figure 1. Fusing mutual traversals in Grafter. node. A full example of this style of program is shown in Fig- them into one call and both save a function invocation and ure2, but for illustration purposes, we use a simple example “visit” root.child only once, instead of twice. shown in Figure 1a, with two calls to the functions f1 and The key challenge to performing this fusion is that the f2. Each of those functions consists of a single statement (s1 reordering necessary to bring statements closer together and, and s2, respectively), and a traversing call on the root’s child, more importantly, to bring traversal calls closer together, is child (f3 and f4, respectively). These two function calls are not always safe. Grafter performs reordering for the state- not independent of each other: s1 in f1 updates this.x while ments using the dependence graph, trying to bring traversals s2 in f2 reads this.x. Grafter performs fusion on such tra- of the same child closer to each other without reordering versals to generate a new set of mutually-recursive functions any dependence edges (and hence violating dependences). that perform fewer traversals of the tree. This reordering is done by grouping the traversal calls that The fusion process starts with a sequence of traversals visit the same node together. Figure 1c shows the results of that are invoked on the same node of a tree, in this case root. such re-ordering. Note that it is clearly safe to outline the two calls in Figure 1a Note that in this example the reordering step changes into one call to function f12 that executes the two functions the traversal f1 from a post-order traversal to a pre-order back-to-back in their original order as shown in Figure 1b traversal (s1 is executed at parents before their children in (the two calls are outlined and then inlined). f12, while it is executed at the children before their parents Grafter then creates a dependence graph representation in the original function f 1). In other words this reordering of the new function f12. This dependence graph represents involves performing implicit code motion that can safely each call and statement of the traversals as a vertex, and (di- changes the schedule of the traversals for the purpose of rected) edges are placed between vertices if two statements achieving more fusion. when executing at a particular node in the tree can access the As calls are grouped together, Grafter is presented with same memory location (Grafter borrows this representa- new sequences of functions that this same fusion process tion from TreeFuser [25].) In Figure 1b, there is a dependence can be applied to. In our example, the two calls grouped in between s1 and s2. We also assume, for illustration purposes, the dashed box in Figure 1c are invoked on the same node that there is a dependence between s2 and the call child.f4(). of the tree (root.child), so Grafter can repeat this merg- Grafter finds dependences between calls and statements by ing process, creating a new merged function f34, building a considering the transitive closure of what calls may access. new dependence graph for the merged function, rearrang- Section 3.2 describes how Grafter analyzes accesses to find ing its statements, and so on. Each time this re-ordering is dependences and construct the dependence graph. performed, more and more operations from multiple logical f12 is now a new, single function that performs multi- traversals on the same node(s) of the tree are brought closer ple pieces of work on root, and invokes multiple traversal together, improving locality, and more and more function functions on root.child. To optimize the body of f12 its invocations from multiple logical traversals are collapsed, desirable to have s1, s2 executed closer to each other for lo- reducing invocation overhead and the total number of times cality benefits—if they access shared fields of root, then that the collection of traversals visit nodes of the tree. data is likely to remain in cache. Furthermore, if the calls f3 and f4 on child are back-to-back then we can further fuse
3 If Grafter encounters a sequence of functions that has 1 intCHAR_WIDTH; been fused before, of course, it can simply call the already- 2 fused implementation. Grafter bounds the amount of func- 3 class Element { 4 Element *Next; tions that can be fused together to ensure this process termi- 5 int Height = 0, Width = 0; nates. Section 3.3 describes in detail how Grafter performs 6 int MaxHeight = 0, TotalWidth = 0; its fusion, including how it handles virtual functions. 7 virtual void computeWidth(){}; Note that encountering cases where an already created 8 virtual void computeHeight(){}; fused function is being called again is the key for having 9 }; 10 significant performance improvement, since that means that 11 class TextBox: public Element { the locality and overhead enhancements will be achieved 12 String Text; recursively. In the limit, instead of 2 traversals visiting each 13 void computeWidth(){ node of the tree once each, we will have a single traversal 14 Next->computeWidth(); that visits each node only once—total fusion. But any amount 15 Width = Text.Length; 16 TotalWidth = Next->Width + Width; of collapsing still promotes locality and reduces node visits. 17 }; The end result of Grafter’s fusion process is a set of 18 void computeHeight(){ mutually recursive functions that together form a partially- 19 Next->computeHeight(); fused traversal. Crucially, these fused functions are analyzed 20 Height = Text.Length * (Width/CHAR_WIDTH) + on a per-type basis, meaning that fusion can occur partially— 1; 21 MaxHeight = Height; not all sequences of calls in a function need to be fused—and 22 if(Next->Height > Height) type-specifically—fusion can occur for some concrete instan- 23 MaxHeight = Next->Height; tiations of virtual functions, but not others. This process 24 }; leads to more fine-grained, precise fusion decisions than 25 }; prior work such as TreeFuser. 26 27 class Group: public Element { The following section describes the details of Grafter 28 Element *Content; fusion process in details. 29 BorderInfo Border; 30 void computeWidth(){ 31 Content->computeWidth(); 32 Next->computeWidth(); 3 Design 33 Width = Content->Width + Border.Size*2; This section describes the design of Grafter’s components 34 TotalWidth = Width + Next->Width; }; in detail. In particular, we first describe how Grafter ana- 35 36 void computeHeight(){ lyzes traversal functions to identify dependences, allowing it 37 Content->computeHeight(); to build the dependence graph representation used to drive 38 Next->computeHeight(); fusion (Section 3.2). Then we explain how Grafter uses the 39 Height = Content->MaxHeigh + Border.Size*2 dependence representation to synthesize new, fused func- 40 MaxHeight = Height; if(Next->Height > Height ) tions (Section 3.3). But first, we explain the language that 41 42 MaxHeight = Next->Heigh; Grafter uses to express its traversals. 43 }; 44 }; 45 46 class End: public Element { 3.1 Language 47 }; The language programmers use to write traversals in Grafter 48 is a subset of C++, allowing programmers to integrate fusible 49 int main(){ tree traversals in larger projects. 50 Element *ElementsList = ...; 51 ElementsList->computeWidth(); ++ A tree in Grafter is defined as an annotated C class, 52 ElementsList->computeHeight(); where instances of the class represent tree nodes. We call 53 } any such annotated class a tree type. Figure 3a shows the grammar for defining tree types. Children of a tree node are Figure 2. An example of a program written in Grafter. pointers to other tree types (not necessarily the same type 1 as, or a subtype of, the node itself). Tree nodes can also store We call these traversal methods . In order to support tree other (non-child) objects and primitive fields—we call these children with abstract types, a tree in Grafter can inherit data fields. fields and virtual traversal methods from other tree types Grafter traversals are written as member methods of tree 1For completeness, Grafter allows other methods to be defined for tree types, implicitly “visiting” the tree node they are invoked on, types, but if they are not explicitly annotated as traversal methods, Grafter and calling other traversal functions to continue the traversal. will not consider them for fusion. 4 and can specialize inherited virtual traversals by overriding s ∈
Building access automata for traversing calls Representing accesses of traversal calls is not as simple. For a (c) Step 3: Minimize automata. given call statement we want to construct a finite automaton Figure 5. Construction of tree writes automata for the that captures any possible access path that could arise during traversing statement content->computeWidth(). root is the the call relative to the tree node being traversed by caller— traversed node transition. including the fact that a call may invoke more traversals. When building the access automaton for a traversal call, Grafter first creates a call graph that includes all the possi- access paths in q to produce the access paths relative to this bly (transitively) reachable functions from that call. We first (the traversed node in f ). For example, when a function f note that any off-tree data accesses made by any of these invokes another function д on a child x of this, it invokes reachable functions are, inherently, not parameterized by x.g(); the receiver object of д is this.x. Thus, to incor- the receiver of the traversal calls—regardless of when and porate the effects of д in to the access paths of f , we can where the function gets called, those access paths will be the prefix the access paths of д with this.x. If д in turn calls same. Thus, we can simply union those automata together h through child y, then we can prefix the access paths of h for the functions in the call graph to capture those accesses. with this.x.y and add them to the access paths of f . The situation is more complicated for on-tree accesses, To account for on-tree accesses, Grafter takes the call as those are parameterized by the receiver of the call, this graph for the traversal call, and labels each edge with the tra- (i.e., the node that is being traversed). To find the accessed versed field that is the receiver for that call. Figure 5a shows locations relative to this, we need to find two things: the the call graph that is generated for Content->computeWidth() functions that are reachable during the call (to know which from our running example. Paths in this graph thus corre- statements are executed), and the tree nodes that those func- spond to possible sequences of child-node traversals to reach tions are invoked on, relative to this. each function in the graph. For each function, Grafter then Consider building the access paths for some function f . attaches all of the statement automata (see the previous sec- For each function q reachable from f , the sequence of chil- tion) for that function to the corresponding node in the call dren traversed to get to the invocation of q gives an access graph. This has the effect of treating the regular language path for the node that q is invoked on, relative to the receiver from the statement automata as the suffix attached to the of f . This access path can be pre-pended to the statements prefix that designates the receiver object. Figure 5b shows 7 Function buildExtendedOnTreeAutomata (CallStmt) : and calls (or calls and calls) can be achieved by intersecting Automata.addRootNodeTransition(0, 1) the automata and testing for emptiness. appendCall(CallStmt, 1) 3.3 Fusing Traversals Function appendCall (CallStmt, State) : for Arg : CallStmt.Arguments() do Overview At a high level, Grafter performs fusion by appendExprAccesses(Arg, 0); repeatedly invoking the following steps: end 1. Find a sequence L of consecutive traversal functions for T : CallStmt.VisitedNode.PossibleTypes() do invoked on the same tree node n. F = getActualCalledFunction(T, CallStmt) 2. Outline these traversal functions into a new function if ! FunctionToState.found(F) then FunctionToState[F] = NewState = Automata.addState() fL that is called on n. If such an fL has already been for Stmt : F.body() do created in an earlier iteration of this process, simply if isCallStmt(Stmt) then call the existing fL. appendCall(Stmt, NewState); 3. Inline each of the individual traversal functions in fL else appendStmtAccesses(Stmt, NewState); to expose the work done on the node n. end 4. Reorder the statements in fL to bring statements that end access the same fields closer together and to create end new sequences of traversal functions invoked on the Automata.addTransition(CallStmt.VisitedNode, same tree node (typically, some child node of n). FunctionToState[F]); 5. Repeat the process for these newly created sequences end Algorithm 1: Construction of tree access automata for traversing state- of calls. ments. These steps constitute the fusion algorithm of Grafter. In particular, every time a sequence L is encountered more the resulting automaton, and Figure 5c shows the reduced than once, and hence an existing fL can be reused, Grafter version. has exploited an opportunity for fusion. We now explain The pseudo code in Figure1 illustrates the construction this fusion process in more detail, and also sketch a proof of process in detail starting from a traversing statement. First, correctness. the algorithm adds the traversed-node transition. After that, Details Fusion starts with a sequence of traversal func- for each possible called function that corresponds to each tions that are invoked at the same tree node (e.g., the root). possible dynamic type of the called child, a state that repre- Grafter searches for such candidates in the compiled pro- sents all the accesses with in that function is created, and a gram and initiates the fusion process for each of them. For transition on the traversed child is added. Accesses within a example, the sequence ElementsList->computeWidth(); function are the union of the accesses of all the statements in followed by ElementsList->computeHeight() in Figure2 the body of the function. Hence, for the function’s traversing line 51. statements, the process will be called recursively. To guar- Because a given function may be virtual, Grafter first antee the termination of such process, accesses of a unique computes all possible sequences of concrete functions that function do not need to have more than one corresponding may be invoked as a result of a sequence of function calls. state in the automata. If a function is already in the automa- For each type T that the sequence of calls can be invoked on, ton, then a transition to the existing state is added. Since Grafter constructs a sequence L of concrete calls. In our ex- there is only a finite number of function definitions the pro- ample, there are three, depending on whether ElementList cess is guaranteed to terminate. points to a TextBox, Group or End. Note that the constructed automata handle the possibility For each function sequence L a fused function with label f of non-statically-bounded trees; whenever we encounter a L is created (if one has not already been generated). If the label function we already have created a state for, we add a "back f does not already exist, then its corresponding function edge" in the automaton to the state that corresponds to that L needs to be generated. A dependence graph G is constructed function. Unbounded recursion is hence represented by loops L for the statements in the traversals in L. In other words, the in the automaton. fused function is essentially a function containing the inlined Finding dependences between statements and calls The statements from each call in the sequence L, in order. Note access automata constructed for calls are no different than that a sequence L may contain the same static function more those constructed for statements. By using the call graph than one time (i.e., the same function can be invoked on a to construct access paths for receiver objects of functions, given node of a tree more than once). In this case, GL contains all the access paths generated by the final access automata statements from multiple copies of that function, and the are rooted at the same receiver object as the automata for statements from the two copies are treated as coming from statements. Hence, finding dependences between statements different traversal functions. 8 Once GL is constructed, the statements (nodes) in the are invoked on root.left, after outlining and inlining, the statement can be reordered as long as no dependences are statements within fL are relative to the formal parameter n violated (as long as a pair of dependent statements are not of fL, and will be exactly the same as if fL were produced reordered). Grafter thus reorders the statements to try and from the same sequence of functions invoked on a different group invocations on the same node together. It then gener- tree node, such as root.right. In other words, two identical ates the fused function code, as explained in the next section. sequences of traversal functions, L and L′ that are invoked on This newly generated function has grouped traversals invo- different tree nodes will yield identical functions fL and fL′ cations on the same node together (and these invocations after outlining and inlining. Because the dependence graph may have come from different functions than the original for these functions are identical, any reordering Grafter sequence L), creating new sequences of functions. Grafter does to create a fused function can be applied to both fL and then process these new sequences of functions to generate fL′ . It is obvious, then, that, upon encountering the same more fused functions. Whenever Grafter encounters a se- sequence of traversal functions L, even if those functions are quence of functions it has seen before, it does not need to invoked on different nodes, Grafter can reuse an existing generate a new function, but instead inserts a call to the synthesized function. already generated function. Crucially, if this new sequence is However, there remains one gap: if, while fusing a se- the same as for a function currently being generated, Grafter quence of functions L to generate fL, Grafter encounters just inserts a recursive call to that function. the same sequence of invocations L that is reachable (tran- The end result is a set of mutually recursive fused func- sitively) from fL, Grafter will substitute a call to fL. In tions, each for a different set of traversals that are executed the simplest case, if fL contains L, then a new invocation together at some point. Furthermore each of those functions to fL will be inserted into the body of fL. Hence, in these is fused independently of the others. This process introduces situations, Grafter is changing the behavior of fL while type-specific-partial-fusion, since for an invocation of a tra- using it to replace L. This process feels circular. However, a versals on a super type, the set of the called functions that straightforward inductive argument on the depth of the call corresponds to each dynamic type are fused independently, stack (in other words, the number of recursive invocations and hence some of them actually might be fusible while of fL before reaching the end of the tree or some base case) others are not. shows that this new invocation of (the rewritten) fL behaves Note that Grafter only generates new functions for se- the same as the original sequence L. This argument mirrors quences it has not seen before. Because Grafter limits the the proof for the soundness of TreeFuser [25, Section 7]. number of functions that can be fused together (see Sec- tion4), the number of sequences of functions is finite, and 3.4 Traversal Code Generation hence fusion is guaranteed to terminate. As described in the previous section, to fuse a sequence of functions L, Grafter generates a graph G with statements Proof sketch of soundness The argument for the sound- and groups of calls that represents the fused function fL. The ness of Grafter’s fusion procedure is straightforward. First, body of the function fL is generated from the graph in a way we note that the outlining and inlining steps (steps 2 and similar to TreeFuser [25], yet it incorporates several changes 3) in the fusion process are trivially safe because they do to account for mutual recursions and virtual calls. not reorder any computations, and hence cannot break any The generated function fL is a global function that takes dependences. Step 4 could potentially break dependences, a pointer to the traversed node as the first parameter (this but because Grafter performs a dependence analysis, it can function will be called from a virtual function stub placed ensure that statements are only reordered if dependences in the tree classes). Since the functions in L can be defined are preserved. Hence, this step is also clearly sound. in different classes, those traversals might be operating on The tricky step in Grafter’s fusion algorithm is the step different types, however since they are all invoked on the where it gains the advantage of fusion: if a sequence of calls same child, there must be a super type that encloses all of to a particular sequence of traversal functions matches a se- them. This type is used for the traversed node parameter in quence that Grafter has already generated a fused function the generated function fL. A lattice for the types traversed for, Grafter immediately replaces the original sequence of in the functions in L is created to find such type. Line 3in calls with a call to the fused function rather than generating figure6 shows the result of fusing the two functions that another new function. This is only safe if the already-fused computes the width and the height for TextBox element in function will do the same thing as the original function se- the program shown in figure2. quence. The traversed node parameter is followed by the traver- To see that this is safe, we note that the process of outlin- sal’s parameters, and an integer that represents the set of ing followed by inlining means that the code of the fused active traversals, active_flags. This parameter can be seen function fL is not dependent on the node fL is invoked on— as a vector of flags where each bit determines whether aspe- in other words, if the original sequence of traversal functions cific traversal function is active or truncated at anypoint 9 1 type while the type of the traversed node is a base type) and 2 ... correctness (a derived class can shadow a base class variable 3 void _fuse__F3F4(TextBox *_r, int active_flags) { of the same name). To handle this, an alias of the traversed 4 TextBox *_r_f0 = (TextBox *)(_r); 5 TextBox *_r_f1 = (TextBox *)(_r); node parameter is created with the desired type for each 6 if (active_flags & 0b11)/*call*/{ traversal function using casting (lines 4 and 5 in figure6) 7 unsigned int call_flags = 0; and those aliases are used by the statements whenever a tree 8 call_flags <<= 1; access happens (lines 15-19). 9 call_flags |= (0b01 & (active_flags >> 1)); A topological order of the nodes in the graph G is then 10 call_flags <<= 1; 11 call_flags |= (0b01 & (active_flags >> 0)); obtained that represents the order of the statement in body 12 _r_f0->Next->__stub1(call_flags); of the fused function. Each node in the topological order 13 } (which corresponds to a top level statement in one of the 14 if (active_flags & 0b1) { traversals) is then written to the function in order. A simple 15 _r_f0->Width = _r_f0->Text.Length; statement is only executed if the traversal that it belongs to is 16 _r_f0->TotalWidth = _r_f0->Next->Width + _r_f0 -> Width ; not terminated. Return statements terminate a traversal and 17 } updates the corresponding active flags as in TreeFuser [25]. 18 if (active_flags & 0b10) { Appropriate flags should be passed when a fused call isin- 19 _r_f1->Height = _r_f1->Text.Length * (_r_f1-> voked within a traversal. The call_flags variable, defined Width / CHAR_WIDTH) + 1; at line 7 in Figure6, holds the active flags that are passed 20 _r_f1->MaxHeight = _r_f1->Height; 21 if (_r_f1->Next->Height > _r_f1->Height) { to the next traversing call. Lines 7–11 fill in the appropriate 22 _r_f1->MaxHeight = _r_f1->Next->Height; flags from the active_flags in the call_flags based on 23 } which traversals the outlined calls belong to. 24 } 25 }; 3.5 Limitations of Grafter 26 void TextBox::__stub1(int active_flags) { 27 _fuse__F3F4(this, active_flags); Limitations of Grafter’s dependence analyses, fusion pro- 28 } cedure, and language mean that it cannot exploit all possible 29 void Group::__stub1(int active_flags) { fusion opportunities for all possible traversal implementa- 30 _fuse__F5F6(this, active_flags); tions. Here, we group the limitations of Grafter into three 31 } categories. 32 void End::__stub1(int active_flags) { 33 _fuse__F1F2(this, active_flags); First, Grafter’s language and implementation have been 34 } limited in some ways merely to simplify the dependence 35 int main() { analysis. For example, Grafter does not support pointers 36 Group *ElementsList; other than to nodes of the tree, but relaxing this simply 37 //ElementsList->computeWidth(); requires enriching the access analysis with standard alias 38 //ElementsList->computeHeight(); 39 ElementsList->__stub1(0b11); analysis techniques. The dependence analysis can similarly 40 } be extended to support loops within traversal functions (that do not themselves invoke additional traversal functions). In Figure 6. Output of Grafter. these scenarios, Grafter’s basic fusion principles need not change. Finally, adding a shape analysis to Grafter could during the execution of the fused function. Those flags are allow it to avoid annotating data structures to establish that needed because the fused traversals can have different ter- they are trees. mination conditions. Second, some extensions to Grafter would require ex- To call the the generated function fL, Grafter replaces tending the machinery of code generation to handle them. the original sequence of calls with a call to a newly created For example, supporting conditional traversal invocation virtual function that acts as a stub and calls the corresponding can be done through syntactic manipulation (pushing the fL for each possible traversed type T . Lines 26-34 in Figure6 condition into an unconditionally-invoked traversal function shows an example of such stub which is called at line 39. that immediately returns if the condition is false), but this The stub’s arguments are the arguments of the fused original introduces instruction overhead. Managing conditional calls, call expression in addition to an integer that represents the traversal functions invoked within loops, or return values active traversals. The stub’s arguments are passed to the new from traversal functions will require some new strategies for fused function as well as the traversed node (this). generating fused traversal code, but likely would not require Statements of different traversals in the fused function substantial changes to the rest of the fusion machinery. should see the traversed node type the same way they see Finally, some extensions to Grafter may require devising it in the original function that they came from. This is is new theories of fusion: new principles for how a fused tra- needed for accessibility (a field might be defined in a derived versal actually operates. In this category are extensions like 10 functions that operate over multiple trees (e.g.,, traversals Experimental platform . Since rendering is a common that zip together two trees), or traversals that perform more task performed on mobile phones, yet the memory on such sophisticated tree mutation such as rotations. devices is relatively small, we evaluated the render tree tra- versals on a smartphone with Qualcomm Snapdragon 425 4 Implementation SoC. The main platform, which is used for the other case studies, is a dual 12-core, Intel Xeon 2.7 GHz Core with 32 Grafter is implemented as a Clang tool that performs source KB of L1 cache, 256 KB of L2 cache, and 20 MB of L3 cache. to source transformation for input programs 5. Grafter uses All cache lines are of size 64 B. L1, L2 caches are 8-way Clang’s internal AST to analyze the annotated traversals and associative and L3 cache is 20-way associative. We have tree structure, and performs checks to validate that the anno- used single-threaded execution throughout our evaluation. tated traversals satisfies Grafter’s restrictions. Any traver- Clang++ was used for compilation with "-O2" optimization sal that does not adhere to Grafter’s language is excluded level for all case studies. from being fused. Grafter uses OpenFST library [1] to con- For each experiment, we measure four quantities: struct the automata that represent accesses of statements and perform operations on these automata. 1. The number of node visits. This measures the num- Different criteria can be used to perform grouping ofcall ber of times any traversal function is called on any nodes in the dependence graph during fusion. Grafter uses node in the tree. This provides a performance-agnostic a greedy approach for grouping: it selects an arbitrary un- measure of fusion effectiveness: the more fusion is per- grouped call node, and tries to maximize the size of the group formed, the fewer functions are called per tree node. by accumulating other un-grouped call nodes. The process 2. The number of instructions executed. Synthesizing continues until there is no more grouping left. This criteria fused functions requires additional work to keep track is sufficient to show significant improvements (Section5), of when various fused traversals truncate, additional thus we did not investigate any other approach for grouping. stub virtual functions, etc. On the other hand, fusion As mentioned earlier, in order to control the fusion pro- reduces call and memory instructions. Measuring in- cess Grafter must limit the number of functions that can structions executed provides an estimate of Grafter’s be fused together. It may seem odd that these cutoffs are overall instruction overhead. needed at all, since there are only a finite number of function 3. The number of cache misses. One benefit of fusion we definitions in the program. However, if a traversal function expect to see is improved locality and reduced memory calls multiple functions on the same child node (say, two), accesses. Cache misses provide a good proxy for this. and each of those functions call two functions on the same 4. Overall runtime. Fusion improves locality, but poten- child node, then it is clear that at each level of the tree, there tially at the cost of increased instruction overhead (as are more active traversal functions than at the previous level. reported by Sakka et al. [25]). The final effectiveness Because each step of Grafter’s fusion process essentially of fusion is hence determined by runtime. descends through one level of the tree to expose more fu- sion opportunities, we will systematically uncover more and 5.1 Case Study 1: Render Tree more functions to fuse together. Hence the need for a cutoff. Tree traversal is an integral part of document rendering. A Grafter limits fusion in two ways: by limiting the length of render tree that represents the organizational structure of a sequence of functions to fuse, and by limiting the number the document is built and traversed a number of times to of times any one static function can appear in a group. compute visual attributes of elements of the document. We implemented a render tree for a document that consists of 5 Evaluation pages composed using nested horizontal and vertical con- We evaluate Grafter through four case-studies from differ- tainers with leaf elements (TextBox, Image, etc.). ent domains, demonstrating its ability to express traversals Figure7 shows the structure and the class hierarchy of over heterogeneous tree structures without compromising the render tree. The tree nodes are of 17 different types; efficiency, to perform fusion efficiently, and to significantly boxes in the figure represent types, arrows represent the enhance the performance of traversals, even when process- fields and point to type of the child node. For instancea ing small trees. The four case-studies are: HorizontalContainer contains a list of elements that can • Fusing multiple traversals over a render tree. be accessed through the field Elements. Boxes with dashed • Fusing multiple AST optimization passes. borders are of super-types of the boxes to the right of them. • Fusing multiple operations on piecewise functions. For instance, an Element can be a TextBox, a Image or a • Fusing two fast multipole method traversals. VerticalContainer. Figure8 shows an example of a page that can be represented using this class hierarchy. Five rendering passes are implemented and listed in Ta- 5Grafter is available at https://bitbucket.org/plcl/grafter_pldi2019/. ble2. These passes are dependent on each other. For example, 11 Table 2. Render-tree and AST passes. Render-tree traversals AST traversals Resolve flex widths De-sugar increment Resolve relative Widths De-sugar decrement Set font style Constant propagation Compute height Replace variable references Compute positions Constant folding Remove unused branches
into a single type, using conditionals to determine which code path to take. Again, we evaluate a baseline that uses the TreeFuser language to implement the passes, and a version that performs as much fusion as possible with TreeFuser. For this experiment we created documents of various sizes by replicating the page shown in Figure8. Figure 9a evaluates the Grafter-fused implementation of the render-tree case study, normalized to the unfused Grafter baseline, while Figure 9b evaluates the TreeFuser-fused implementation, Figure 7. Class hierarchy of the render tree. normalized to its unfused baseline6. Both systems show the expected results of fusion: reduced node visits after fusion, and reduced cache misses. Neverthe- less, on both metrics, Grafter does better than TreeFuser: due to its finer-grained representation and fusion, it can more aggressively fuse traversals, resulting in 60% fewer node visits than the baseline, compared to 40% fewer node visits for TreeFuser. Note that because both the Grafter and TreeFuser implementations do the same work, the baselines have exactly the same absolute number of node visits. This increased fusion is reflected in cache misses: TreeFuser’s fusion reduces cache misses by 40% while Grafter reduces misses by 80%. We also see the advantage of Grafter’s type-specific fu- sion approach: because fused functions are on a per-type basis, Grafter is able to leverage dynamic dispatch to spe- cialize traversals. As a result, it exhibits virtually no instruc- tion overhead relative to the baseline. TreeFuser, in contrast, has a 30–40% instruction overhead. All told, these effects mean that TreeFuser cannot achieve performance improvements for the inputs we investigate: the improved memory system behavior cannot outweigh the instruction overhead. In contrast, Grafter sees substantial Figure 8. The layout of the rendered page. performance improvements: 20% even for the smallest input (a single page), and 60% for inputs of 1000 pages or more. computing the height of an element depends on computing And note that this is despite the fact that Grafter’s baseline the width and font style. In Grafter, passes are implemented is already substantially faster than TreeFuser’s, as seen in as fine-grained, stand-alone functions for each type. the baseline runtimes shown in Figures 9a and 9b. In the first experiment we compared the effectiveness of Grafter and TreeFuser [25], prior work that can also per- 6For small input sizes, each experiment is run in a loop to achieve a reason- form (partial) fusion for general recursion. We produced a able overall runtime, then divided through by the number of loop iterations baseline that performs no fusion, as well as a version that to find the per-run metrics. 95% confidence intervals for render tree experi- ments are within ±5% for all measurements except for cache misses of trees uses Grafter’s full fusion capabilities. We also implemented smaller than 50 pages in figure9. For those the intervals expands as the tree the same passes in TreeFuser [25]. To accommodate the limi- size decreases down to ±60%. Note that for such small trees, the absolute tations of TreeFuser, that implementation collapses the types number of misses is quite small. 12 (a) Grafter-fused implementation normalized to unfused Grafter (b) TreeFuser-fused implementation normalized to unfused Tree- implementation. Fuser implementation. (1M page baseline does not complete.) Figure 9. Performance comparison of render-tree passes written in Grafter and TreeFuser for different tree sizes. Number in parentheses is runtime of the baseline. Number of pages on the x axis and normalized measurement on the y axis.
Table 3. Performance of fused traversals normalized to un- Table2 shows the six different AST traversal passes we fused ones for different render tree configurations. implemented in Grafter: two de-sugaring passes for in- crement and decrement operations, and three optimization runtime cache misses node visits tree size description passes. The constant propagation pass is written as two tra- 0.17 (L2) 105 simple Doc1 0.22 0.14 (L3) 0.38 90MByte pages versals and works as follows: the constant propagation tra- Doc2 0.65 0.28 (L2) 0.4 4MByte 1 dense page versal looks for constant assignments and for each of them it 0.24 (L2) 150 pages of initiates a traversal to replace variable references with con- Doc3 0.47 0.18 (L3) 0.4 58MByte different sizes stants. The AST passes depends on each other; de-sugaring must happen before the optimization passes, and removing unused branches depends on constant folding and propa- Programmability-wise, the overall logical lines of code gation since they may produce constant branch conditions. (LLOC) 7 for the body of the traversals is the same for both Furthermore, those passes mutate the tree (e.g., to de-sugar TreeFuser and Grafter. However, those LLOC are distributed an expression, one part of the AST is deleted and another among 55 different simple functions in Grafter while Tree- part is constructed). Fuser requires one function per traversal, with complex con- We wrote a function that consists of different statement ditionals to disambiguate types. types, and expressed it as an AST. This function was repli- We also evaluated Grafter’s render tree traversal on mul- cated in order to obtain bigger trees for the evaluation. Figure tiple different documents configurations (input trees), with 11 shows the performance of the fused AST traversals with different properties. The results are summarized in Table3: respect to the un-fused ones 9. Grafter reduces L2 cache Grafter achieves speedups between 1.5× and 4.5×8. misses by 75% and once the tree is big enough, it reduces L3 misses by 70% as well. The fused traversals have instruc- 5.2 Case Study 2: AST Traversals tion overhead (between 15% and 4%), but that overhead is Abstract Syntax Tree (AST) representation of programs is overcome by the reduction in cache misses. The fused tra- common in modern compiler frontend. Various validation versals are 1.25× to 2.5× faster than the un-fused traversals and optimization passes are performed on AST representa- depending on the tree size. tion. We implemented AST passes for a simple imperative The instruction overhead is caused by different AST passes language that has assignments, if statements, functions, and having different truncation conditions, unlike the render allows certain syntactic sugars. Figure 10 shows the language tree traversals, which all completely traverse the tree and constructs and the class hierarchy of the AST that represents get truncated at the same time. For instance, the “replace programs in the language. Node types in the AST belong variable references” pass gets truncated once the reference to different hierarchy levels and passes are implemented as is reassigned. Because this truncation is dynamic, parame- virtual functions defined at the top level type. ters of the truncated traversals will kept being passed, and
7We use Fenton’s definition of LLOC for C++: the number of instructions with the semantic delimiter [11]. 995% confidence intervals for the AST experiments are within ±5% for all 8The results in Table3 were collected on the main platform, rather than the measurements, except for the point 103 in Figure 11, where confidence mobile platform. intervals are within ±33% for L3 cache misses and ±15% for runtime. 13 Table 5. Traversals over piecewise functions.
Function Description scale(c) f (x) = c f (x) add(c) f (x) = f (x) + c square() f (x) = f (x)· f (x) differentiate() f (x) = f (1)(x) addRange(c, a, b) f (x) = f (x) + c(u(a) − u(b)) f (x) = x f (x)·(u(a) − u(b)) multXRange(a, b) +f (x)·(1 − u(a) + u(b)) addXRange(a, b) f (x) = f (x) + x(u(a) − u(b)) ∫ b ( ) integrate(a, b) a f x project(x0) f (x0) Figure 10. AST class hierarchy with 20 different types.
5.3 Case Study 3: Piecewise Functions MADNESS (Multiresolution, Adaptive Numerical Environment for Scientific Simulation) [13] uses kd-trees to compactly rep- resent piecewise functions over a multi-dimensional domain. The inner nodes of the tree divide the domain of the function into different sub-domains, while leaf nodes store the coef- ficients of a polynomial that estimates the function within the node’s sub-domain. MADNESS uses this representation to solve differential and integral equations: mathematical operations on these functions (e.g.,, differentiation, scaling, sqaring) can be captured as traversals of the tree. In this case study, we implemented kd-trees for single variable functions, and different traversals to perform com- Figure 11. Performance measurements for fused AST tra- putations on these functions. Table5 shows these traversals. versals normalized to the unfused ones for different tree Some of these traversals requires structural changes to the sizes. Number of functions on the x axis, and normalized tree. For example, the multXRange(a, b) traversal multiplies measurements on the y axis. the function in the range [a,b] by x where x is the variable representing the domain. A leaf node that has a non-empty intersection interval of its domain and the interval [a,b], needs to be split into multiple nodes if its domain does not Table 4. Performance of fused traversals normalized to un- lie completely within the interval [a,b]. fused ones for different AST configurations Unlike the previous case studies, the schedule of traver-
Runtime Cache misses Node visits Tree size Description sals in this case-study depends on the constructed equation Prog1 0.71 0.26 (L2) 0.76 8MByte Small functions and differs from one another. Hence, manual fusion ofsuch Prog2 0.87 0.58 (L2) 0.92 6MByte One large function 0.48 (L2) traversals is not practical, because it needs to be done for Prog3 0.31 0.26 (L3) 0.93 31MByte Long live ranges each equation separately based on the different operations in the equation. We constructed three different equations that use different schedules of traversals, shown in Table6. We evaluated the performance of the fused traversals for the truncation flags will continue to be checked, until all each of these equations on a balanced kd-tree constructed − traversals truncate, increasing overhead. by uniformly partitioning the interval [105,10 5]. Table4 shows the performance of the fused traversals Figure 12 shows the performance of fused traversals corre- for different AST inputs with respect to the unfused ones. sponding to the first equation in Table6 for different depths Prog1 consists of a large number of normal-sized functions. of kd-tree, normalized to the unfused ones. 10 The fused Since all the traversals are fusible across the function list, traversals reduce node visits by 83%, and we see a 90% re- Prog1 has the highest reduction in node visits, 34%. On the duction in L2 cache misses. Overall, the fused traversals are other hand Prog2 consists of only one large function, and faster, with a runtime improvement ranging from 15% for hence has less reduction in node visits, 8%. Prog3 consists of small trees, to 66% for large ones. Table6 summarizes the multiple functions with long live ranges. The tree of Prog3 1095% confidence intervals for kd-tree experiments are within ±1% for all is the largest and thus reductions in L3 cache misses are reported measurements except for L3 cache misses of trees of depth 16, achieved along with a 70% reduction in the runtime. reported in Figure 12, where intervals are ±16%. 14 1.2 1 1 0.8 0.92 0.8 0.91 0.78 0.6 0.6 0.4 0.4 0.2 0.2
0 0 4 8 12 16 20 24 28 1.0E+05 1.0E+06 1.0E+07 1.0E+08 Runtime L2 Cache Misses L3 Cache Misses Instructions Node Visits runtime L2 misses L3 misses instructions Performance measurements for fused FMM tra- Figure 12. Performance measurements for fused kd-tree tra- Figure 13. versals normalized to the unfused ones for different tree sizes. versals normalized to the unfused ones for different number Depth of the tree on the x axis, and normalized measurement of points. Number of points on the x axis and normalized on the y axis. measurement on the y axis.
Performance improvement of different fused tra- Table 6. Specialized or domain-specific programs In these sys- versals on piecewise functions. tems, the programmer aids the tool by expressing their tree traversals in a structured form. For example, in miniphases Equation Runtime Cache misses Node visits 4 (2) 2 Í3 i [21], the programmer writes an AST transformation as a col- x (f (x)) + i=0 x 0.66 0.09(L2) 0.51(L3) 0.17 (5) f (x)|x=0 0.49 0.20(L2) 0.51(L3) 0.20 lection of “hooks” or callbacks that observe individual syntax 105 ∫ 3( ( ) . )2 · ( ) nodes and act on them, while abstracting the recursion over −105 x f x + 5 u 0 0.88 0.33(L2) 0.65(L3) 0.33 the tree so that it is handled by the underlying framework. performance of the corresponding fused traversals for each The user manually groups these miniphases into tree traver- equation normalized to the unfused ones when performed sals, but there is no guarantee of equivalence between fused on a balanced kd-tree of depth 20. and unfused executions—that soundness burden is on the programmer. This style of decomposing passes is often ap- 5.4 Case Study 4: Fast Multipole Method proximated in many compiler implementations—strategies The fast multipole method (FMM) is a numerical technique for it are part of compiler writer folklore. It also appears used in evaluating pairwise interactions between large num- explicitly in software frameworks such as the a nanopass ber of points distributed in a space (e.g., Long-ranged forces framework [27]. in the n-body problem, computation of gravitational poten- Another specialized way to express tree traversals is by us- tial, and computational electromagnetic problems) [12]. ing attribute grammars to specify tree transducers—automata In this case study, we reimplement the FMM benchmark that traverse trees and create output [6, 9]. Using this special- from TreeFuser in Grafter, which is based on the imple- ized representation of programs, subsequent authors were mentation from the Treelogy benchmark suite [14]. Figure 13 able to improve efficiency of compositions of tree transduc- shows the performance of fused traversals for different input ers [10, 18]. However, to our knowledge none of these ap- sizes. Grafter was able to fully fuse the two passes and yield proaches handle as general a class of programs and fusion a performance improvement up to 22% over the unfused opportunities as Grafter: for example, we are not aware of version.11 a tree transducer solution for partial fusion. Enabling fusion based on a high-level representation of 6 Related Work traversals has likewise been explored in the HPC domain: The overall goal of this work is to enable programmers to for instance the MADNESS simulation framework uses this achieve efficient tree traversals. Thus it is obliquely related approach for fusing kd-tree passes Rajbhandari et al.[23, 24]. to prior work on that improves the performance of a single This abstraction supports a more restrictive notion of fusion tree traversal by adjusting memory layout or encoding [3, 4, than Grafter. All traversals must visit the same children and 17, 28]. The most directly related work, however, optimizes must visit them in either a pre-order or post-order; partial series of traversals, typically by fusing together multiple fusion is not possible. Moreover, the framework relies on passes. This work can be sub-divided based on the source programmer-provided dependence information. representation it targets: Functional Programs As discussed in the introduction, 1195% confidence intervals for the FMM experiments are within ±1% for all the prior research explores fusing general recursive func- measurements. tional programs and eliminating intermediate data structures 15 (deforestation) [29]. In practice, functional programming lan- Optimizer. In Proceedings of the 29th ACM SIGPLAN Conference on Pro- guages have settled on using libraries of combinators with gramming Language Design and Implementation (PLDI ’08). ACM, New known fusion transformations (like map and fold). For ex- York, NY, USA, 101–113. https://doi.org/10.1145/1375581.1375595 [3] TM Chilimbi, MD Hill, and JR Larus. 1999. Cache-conscious structure ample, in the Haskell ecosystem, many everyday libraries layout. ACM SIGPLAN Notices (1999). http://dl.acm.org/citation.cfm? use the stream fusion approach [7] and variations on it. This id=301633 combinator style works well for collections (arrays, lists, etc) [4] Trishul M. Chilimbi and James R. Larus. 1999. Using generational but is sharply more limited than general recursive functions garbage collection to implement cache-conscious data placement. , on trees. One example of this fact is that compiler writers 37–48 pages. https://doi.org/10.1145/301589.286865 [5] Wei-ngan Chin, Zhenjiang Hu, and Masato Takeichi. 1999. A Modular cannot use such frameworks for fusing AST traversals. Derivation Strategy via Fusion and Tupling. (12 1999). [6] H. Comon, M. Dauchet, R. Gilleron, C. Löding, F. Jacquemard, D. Lugiez, Imperative Programs The most closely related works to S. Tison, and M. Tommasi. 2007. Tree Automata Techniques and Ap- Grafter are systems that target imperative or object-oriented plications. Available on: http://www.grappa.univ-lille3.fr/tata. release source programs. Aside from TreeFuser (which we have al- October, 12th 2007. ready described) most work on fusion in imperative pro- [7] Duncan Coutts, Roman Leshchinskiy, and Don Stewart. 2007. Stream grams focuses on loop fusion, merging the bodies of adjacent Fusion: From Lists to Streams to Nothing at All. In Proceedings of the 12th ACM SIGPLAN International Conference on Functional Pro- loops. Typically this is applied to programs operating on ar- gramming (ICFP ’07). ACM, New York, NY, USA, 315–326. https: rays and matrices [8, 15, 22]. The most well known work in //doi.org/10.1145/1291151.1291199 this area targets the further restricted class of programs with [8] Alain Darte. 1999. On the Complexity of Loop Fusion. In Proceedings affine indexing expressions [2]. These approaches do not of the 1999 International Conference on Parallel Architectures and Com- generalize to recursive programs operating over irregular pilation Techniques (PACT ’99). IEEE Computer Society, Washington, DC, USA, 149–. http://dl.acm.org/citation.cfm?id=520793.825721 data such as the tree traversals we explore in this paper. [9] John Doner. 1970. Tree Acceptors and Some of Their Applications. J. Comput. Syst. Sci. 4, 5 (Oct. 1970), 406–451. https://doi.org/10.1016/ 7 Conclusions S0022-0000(70)80041-1 [10] Joost Engelfriet and Sebastian Maneth. 2002. Output String Languages This paper introduced Grafter, a framework for performing of Compositions of Deterministic Macro Tree Transducers. J. Comput. fine-grained fusion of generic tree traversals. In comparison Syst. Sci. 64, 2 (March 2002), 350–395. https://doi.org/10.1006/jcss. with prior work, Grafter is either more general (in its abil- 2001.1816 ity to handle general recursion and heterogeneity), more [11] Norman E. Fenton. 1991. Software Metrics: A Rigorous Approach. Chap- man & Hall, Ltd., London, UK, UK. effective (in its ability to perform more aggressive fusion), [12] L. Greengard and V. Rokhlin. 1987. A Fast Algorithm for Particle sound (i.e., without relying on programmer assumptions of Simulations. J. Comput. Phys. 73, 2 (Dec. 1987), 325–348. https: fusion safety), or some combination of all three. We showed //doi.org/10.1016/0021-9991(87)90140-9 that Grafter is able to effectively fuse together traversals [13] R. Harrison, G. Beylkin, F. Bischoff, J. Calvin, G. Fann, J. Fosso- from two domains that rely on repeated tree traversals: ren- Tande, D. Galindo, J. Hammond, R. Hartman-Baker, J. Hill, J. Jia, J. Kottmann, M. Yvonne Ou, J. Pei, L. Ratcliff, M. Reuter, A. Richie- dering passes for document layout, and AST traversals in Halford, N. Romero, H. Sekino, W. Shelton, B. Sundahl, W. Thorn- compilers. Not only can Grafter perform more aggressive ton, E. Valeev, ÃĄ. VÃązquez-Mayagoitia, N. Vence, T. Yanai, and fusion than prior work, it also delivers substantially better Y. Yokoi. 2016. MADNESS: A Multiresolution, Adaptive Numerical performance. Grafter allows programmers to write simple, Environment for Scientific Simulation. SIAM Journal on Scientific Com- ergonomic tree traversals while relying on automation to puting 38, 5 (2016), S123–S142. https://doi.org/10.1137/15M1026171 arXiv:https://doi.org/10.1137/15M1026171 produce high-performance fused implementations. [14] N. Hegde, J. Liu, K. Sundararajah, and M. Kulkarni. 2017. Treelogy: A benchmark suite for tree traversals. In 2017 IEEE International Sympo- Acknowledgments sium on Performance Analysis of Systems and Software (ISPASS). 227– 238. https://doi.org/10.1109/ISPASS.2017.7975294 This work was supported in part by National Science Foun- [15] Ken Kennedy and Kathryn S. McKinley. 1994. Maximizing Loop Paral- dation awards CCF-1725672 and CCF-1725679, and by De- lelism and Improving Data Locality via Loop Fusion and Distribution. partment of Energy award DE-SC0010295. We would like to In Proceedings of the 6th International Workshop on Languages and thank our shepherd, Ben Titzer, for his help with the revi- Compilers for Parallel Computing. Springer-Verlag, London, UK, UK, sions of this paper, as well as the anonymous reviewers for 301–320. http://dl.acm.org/citation.cfm?id=645671.665526 [16] J. R. Larus and P. N. Hilfinger. 1988. Detecting Conflicts Between their suggestions and comments. Structure Accesses. In Proceedings of the ACM SIGPLAN 1988 Conference on Programming Language Design and Implementation (PLDI ’88). ACM, References New York, NY, USA, 24–31. https://doi.org/10.1145/53990.53993 [17] Chris Lattner and Vikram Adve. 2005. Automatic pool allocation: [1] Cyril Allauzen, Michael Riley, Johan Schalkwyk, Wojciech Skut, and improving performance by controlling data structure layout in the Mehryar Mohri. 2007. OpenFst: A General and Efficient Weighted heap. ACM SIGPLAN Notices 40 (2005), 129–142. https://doi.org/10. Finite-State Transducer Library. In Implementation and Application of 1145/1065010.1065027 Automata. Springer Berlin Heidelberg, Berlin, Heidelberg, 11–23. [18] Andreas Maletti. 2008. Compositions of Extended Top-down Tree [2] Uday Bondhugula, Albert Hartono, J. Ramanujam, and P. Sadayap- Transducers. Inf. Comput. 206, 9-10 (Sept. 2008), 1187–1196. https: pan. 2008. A Practical Automatic Polyhedral Parallelizer and Locality 16 //doi.org/10.1016/j.ic.2008.03.019 New York, NY, USA, 152–162. https://doi.org/10.1145/2892208.2892228 [19] Leo A. Meyerovich and Rastislav Bodik. 2010. Fast and Parallel Web- [25] Laith Sakka, Kirshanthan Sundararajah, and Milind Kulkarni. 2017. page Layout. In Proceedings of the 19th International Conference on TreeFuser: A Framework for Analyzing and Fusing General Recursive World Wide Web (WWW ’10). ACM, New York, NY, USA, 711–720. Tree Traversals. Proc. ACM Program. Lang. 1, OOPSLA, Article 76 (Oct. https://doi.org/10.1145/1772690.1772763 2017), 30 pages. https://doi.org/10.1145/3133900 [20] Leo A. Meyerovich, Matthew E. Torok, Eric Atkinson, and Rastislav [26] Laith Sakka, Kirshanthan Sundararajah, Ryan R. Newton, and Milind Bodik. 2013. Parallel Schedule Synthesis for Attribute Grammars. In Kulkarni. 2019. Sound, Fine-Grained Traversal Fusion for Heteroge- Proceedings of the 18th ACM SIGPLAN Symposium on Principles and neous Trees. In Proceedings of the 40th ACM SIGPLAN Conference on Practice of Parallel Programming (PPoPP ’13). ACM, New York, NY, Programming Language Design and Implementation (PLDI 2019). ACM, USA, 187–196. https://doi.org/10.1145/2442516.2442535 New York, NY, USA, 16. https://doi.org/10.1145/3314221.3314626 [21] Dmitry Petrashko, Ondřej Lhoták, and Martin Odersky. 2017. [27] Dipanwita Sarkar. 2008. Nanopass Compiler Infrastructure. Ph.D. Disser- Miniphases: Compilation Using Modular and Efficient Tree Trans- tation. Indianapolis, IN, USA. Advisor(s) Dybvig, R. Kent. AAI3337263. formations. In Proceedings of the 38th ACM SIGPLAN Conference on [28] D. N. Truong, F. Bodin, and A. Seznec. 1998. Improving Cache Behav- Programming Language Design and Implementation (PLDI 2017). ACM, ior of Dynamically Allocated Data Structures. In Proceedings of the New York, NY, USA, 201–216. https://doi.org/10.1145/3062341.3062346 1998 International Conference on Parallel Architectures and Compilation [22] Apan Qasem and Ken Kennedy. 2006. Profitable Loop Fusion and Techniques (PACT ’98). IEEE Computer Society, Washington, DC, USA, Tiling Using Model-driven Empirical Search. In Proceedings of the 20th 322–. http://portal.acm.org/citation.cfm?id=522344.825680 Annual International Conference on Supercomputing (ICS ’06). ACM, [29] Philip Wadler. 1990. Deforestation: transforming programs to eliminate New York, NY, USA, 249–258. https://doi.org/10.1145/1183401.1183437 trees. Theoretical Computer Science 73, 2 (1990), 231 – 248. https: [23] Samyam Rajbhandari, Jinsung Kim, Sriram Krishnamoorthy, Louis- //doi.org/10.1016/0304-3975(90)90147-A Noel Pouchet, Fabrice Rastello, Robert J. Harrison, and P. Sadayap- [30] Yusheng Weijiang, Shruthi Balakrishna, Jianqiao Liu, and Milind pan. 2016. A Domain-specific Compiler for a Parallel Multiresolution Kulkarni. 2015. Tree Dependence Analysis. In Proceedings of the Adaptive Numerical Simulation Environment. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design International Conference for High Performance Computing, Networking, and Implementation (PLDI ’15). ACM, New York, NY, USA, 314–325. Storage and Analysis (SC ’16). IEEE Press, Piscataway, NJ, USA, Article https://doi.org/10.1145/2737924.2737972 40, 12 pages. http://dl.acm.org/citation.cfm?id=3014904.3014958 [31] Ben Wiedermann and William R. Cook. 2007. Extracting Queries [24] Samyam Rajbhandari, Jinsung Kim, Sriram Krishnamoorthy, Louis- by Static Analysis of Transparent Persistence. In Proceedings of the Noël Pouchet, Fabrice Rastello, Robert J. Harrison, and P. Sadayappan. 34th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Pro- 2016. On Fusing Recursive Traversals of K-d Trees. In Proceedings of the gramming Languages (POPL ’07). ACM, New York, NY, USA, 199–210. 25th International Conference on Compiler Construction (CC 2016). ACM, https://doi.org/10.1145/1190216.1190248
17