Static Loop Parallelization Decision Using Template Metaprogramming Alexis Pereda, David R.C
Total Page:16
File Type:pdf, Size:1020Kb
Static Loop Parallelization Decision Using Template Metaprogramming Alexis Pereda, David R.C. Hill, Claude Mazel, Bruno Bachelet To cite this version: Alexis Pereda, David R.C. Hill, Claude Mazel, Bruno Bachelet. Static Loop Parallelization Decision Using Template Metaprogramming. IEEE International Conference on High Performance Computing and Simulation (HPCS), Jul 2018, Orléans, France. pp.1015-1021, 10.1109/HPCS.2018.00159. hal- 01848372 HAL Id: hal-01848372 https://hal.archives-ouvertes.fr/hal-01848372 Submitted on 29 Dec 2020 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Static loop parallelization decision using template metaprogramming Alexis Pereda, David R.C. Hill, Claude Mazel, Bruno Bachelet Université Clermont Auvergne, CNRS, LIMOS Clermont-Ferrand, France {alexis.pereda, david.hill, claude.mazel, bruno.bachelet}@uca.fr Abstract (if parallelized, does the loop run faster?). The data dependency information can be processed by This article proposes to use C++ template metaprogram- some code analyser that must be executed before the com- ming techniques to decide at compile-time which parts of a pilation (optionally requiring annotations to make the pro- code sequence in a loop can be parallelized. The approach cessing easier) by the way of meta-compilation or compiler focuses on characterizing the way a variable is accessed in plugin. The former is rather complex to implement and a loop (reading or writing), first to decide how the loop implies redoing nearly the compiler work. With a compiler should be split to enable the analysis for parallelization on plugin, users will be tied to the compilers for which it has each part, and then to decide if the iterations inside each been designed. loop are independent so that they can be run in parallel. Our research is focused on identifying whether program The conditions that enable the parallelization of a loop are segments are parallelizable or not with C++ Template first explained to justify the proposed decision algorithm Metaprogramming (TMP) independently from the under- exposed. Then, a C++ library-based solution is presented lying C++ compiler. TMP is part of the C++ compilation that uses expression templates to get the relevant informa- process, thus it is standard and is available independently tion necessary for the parallelization decision of a loop, and of the compiler, and it permits code re-writing in a syntax- metaprograms to decide whether to parallelize the loop checked way by allowing us to intervene in the compilation and generate a parallel code. process (without being a plugin). The solution presented Keywords – code parallelization; metaprogramming; in this article uses Expression Templates (ET) [16] to re- C++ expression templates; parallelization decision trieve relevant information for parallelization from the code sequence of a loop, and common TMP techniques [1] to de- sign metaprograms to analyse this information to possibly 1 Introduction split the loop, to decide the parallelization and to produce the parallel code. This solution, that operates mainly at Today’s computers are capable of executing multiple in- compile-time, attempts to avoid, as possible, any execution structions in parallel, and developers must adapt their code time overhead. to get benefits from that. There are now many tools to Section 2 introduces related work. In section 3, we make this task easier, allowing developers to indicate which present the conditions required to detect parallelizable pieces of their code will be executed in parallel. Among loops and to run them without altering the results. From these tools, there are MPI, POSIX Threads, OpenMP, In- these conditions, we present a two-step decision algorithm tel Threading Building Blocks (TBB). With such tools, that first identifies the parts of the code sequence inside developers still have to decide whether the code can actu- the loop that are independent, in order to possibly split ally be parallelized or not. This decision depends mainly the loop, and then, on each part of the loop, detects if the on which data is accessed and how it is accessed (read- iterations are independent so the part can be parallelized. ing and/or writing), which would require a dependence Then, section 4 shows how we can implement this detec- analysis. However, a wrong decision from the developer tion at language level with C++ TMP, based on ET for will lead to incorrect code, meaning a parallel code that information retrieval and metaprograms for analysis and produces a result different from its sequential counterpart. parallel code generation. Examples are also presented to In this article, we focus on detecting if a loop can be paral- demonstrate the automatic parallelization of loops with lelized (if parallelized, does the loop produce the expected our library-based solution. Section 5 presents performance result?), but we do not consider the performance aspect results of our proposed design. 1 2 Related work 1 /* a, b, c are arrays */ 2 for(int i=0; i< n;++i) { Automatic parallelization tools are studied since many 3 a[i]= a[i]* b[i]; years [12, 8]. They usually work as meta-compilers [20, 2], 4 c[i]= c[i+1]+ d[i]; producing a parallel source code from a sequential source 5 b[i]= b[i]+1; code, or as compilers [6], producing directly an executable. 6 d[i]= c[i]; More recently, we can find complete infrastructure dedi- 7 } cated to automatic parallelization [15] and also advanced Listing 1: Loop example graph and operation research tools enabling a better paral- lelization [7]. If we focus on recent C++ oriented solutions, 1 /* a, b, c are arrays */ there are dynamic and static propositions. 2 for(int i=0; i< n;++i) { A tool proposed by Bauer et al., Legion [4], helps devel- 3 a[i]= a[i]* b[i]; oping parallel programs by defining logical regions. Those 4 b[i]= b[i]+1; regions make it possible for Legion to do run-time depen- 5 } dence analysis. Then dynamic task scheduling is done, 6 for(int i=0; i< n;++i) { considering all acquired information about data dependen- 7 c[i]= c[i+1]+ d[i]; cies. However, we think running all dependence analysis 8 d[i]= c[i]; during run-time generates an overhead that could possibly 9 } be reduced by doing some of the work during compile-time. Barve’s sequential to parallel C++ code converter [3] Listing 2: Example of loop splitting is an example of metaprogramming used to enable paral- lelism from a sequential code. It runs dependence analysis, based of Banerjee’s test [19], and generates a suitable par- Now, the second loop still cannot be run in parallel, allelized version of the sequential input code. This solution whereas the first loop can. This section will explain first demonstrates the possibility to process data dependencies how to decide whenever a program segment can be split, statically. It requires a first pass on a sequential C++ code in order to separate a loop in several loops that will be before compiling it. analysed independently for parallelization. This section MetaPASS [10] is a library allowing implicit task-based will then present how we can determine if a loop can be parallelism from a sequential C++ code. Its dependence run in parallel. properties are acquired statically using C++ TMP. It then relies on a run-time dependence analyser, making that tool 3.1 Conditions for permutation both static and dynamic. Our solution makes use of C++ TMP for the full depen- Three conditions are sufficient to allow the permutation of dence analysis and parallel code generation. This way, we two program segments %0 and %1, with ,8 the set of the can aim to very little execution time overhead, like it has output variables of %8 and '8 the set of the input variables already been achieved in similar contexts [13]. of %8. These are the conditions given by Bernstein [5]. ,1 \ '0 = ; (1) 3 Conditions for parallel loops '1 \ ,0 = ; (2) ,0 \ ,1 = ; (3) In this section, we will see what conditions allow us to execute a loop in parallel, meaning running iterations of A program segment % = f퐼1, ..., 퐼=g can be composed of the loop in parallel, without changing the final result of one or more instructions 퐼1 to 퐼=. Algorithm 1 presents a the code. As parallelizing a loop induces performing the process to identify independent instructions of %, and to iterations in an undefined order, we focus here on producing group dependent ones. This algorithm relies on Bernstein’s a parallel code only if its correctness can be preserved. The conditions. We assume that function BernsteinTest(G, performance aspects are not discussed here. H) returns true if the conditions are met for the program In the listing 1, as it is, the iterations of the program segments G and H. segment at lines 3 to 6 cannot be executed in parallel To identify the instructions of % that cannot be per- because of line 4. This instruction requires each iteration muted, a set 퐺 of independent program segments can be of the loop to be executed in the correct order. The progressively built as follows. Starting with an empty set instruction at line 4 accesses the same array element of 2 퐺, each instruction 퐼: , considered as a program segment as in the instruction at line 6, first for writing and then for %0 = f퐼: g, is tested for Bernstein’s conditions with each reading, so we should not separate them.