Parallel Functional Programming with Skeletons the ocamlpl exp eriment ? Marco Danelutto Rob erto Di Cosmo Xavier Leroy Susanna Pelagatti ? Dipartimento di Informatica LIENSDMI INRIA Ro cquencourt Universita di Pisa Ecole Normale Superieure Domaine de Voluceau PISA ITALY PARIS FRANCE ROCQUENCOURT FRANCE parallel the evaluation of all the parameters of a strict func Abstract tion call Both the p ossibility of automatically exploiting Writing parallel programs is not easy and debugging them this kind of implicit parallelism through parallel graph re is usually a nightmare To cop e with these diculties a duction or other compiler techniques and structured approach to parallel programs using skeletons the p ossibility of providing the user with ways to annotate and template based compiler techniques has b een developed somehow functional programs in order to drive parallelism over the past years by several researchers including the PL exploitation have b een explored with dierent results group in Pisa After Coles work on skeletons a new research track has This approach is based on the use of a set of primitive b een initiated concerning parallel functional programming forms that are just functionals implemented via templates with skeletons As an example Bratvold showed how skele exploiting the underlying parallelism so it is natural to ask tons can b e lo oked for and exploited within an ML dialect whether marrying a real functional language like Ocaml with and Darlingtons group at the Imp erial College in Lon the PL skeletons can b e the basis of a p owerful parallel pro don started considering plain functional skeletons and gramming environment We show that this is the case our came up with the denition of a skeleton functional co ordi prototype written entirely in Ocaml using a limited form of nation language closure passing allows a very simple and clean programming Some of the authors of this pap er developed at the Uni style shows real sp eedup over a network of workstations versity of Pisa a skeleton parallel programming language and as an added fundamental b onus allows logical debug pl pl is actually an imp erative programming lan ging of parallel programs in a sequential framework without guage although the skeleton framework used is completely changing the user co de functional The pl compiler uses an original template Key words skeletons functional languages closures based parallelism exploitation technique achieving very go o d parallelism p erformance as well as programmability and p ortability fea tures The authors started lo oking at the p ossibility of exp orting the templatebased skeleton parallelism exploita Introduction tion techniques to the functional world In the meanwhile the Ob jective Caml functional pro Functional programming languages have greatly improved gramming language has b een developed at INRIA Ro cquen since their app earance in the sixties The p erformance court in France Ob jective Caml ocaml in the se achieved in functional program execution is now compara quel is a functional language of the ML family It ble to the p erformance achieved using classical imp erative supp orts b oth functions as rstclass values and full imp er programming languages In addition mo dern func ative features in particular arrays mo diable inplace This tional programming languages have advanced features like combination of features makes it well adapted to skeleton strong typing and advanced mo dule systems that improve based programming higherorder functions functions tak b oth programmer pro ductivity and co de quality and main ing userprovided functions as arguments can b e suitably tainability used to mo delimplement skeletons while the imp erative But what ab out parallel functional programming Since features can b e exploited to provide a parallel implementa the very b eginning it has b een claimed that functional pro tion of skeletons Other useful features of ocaml include a gramming languages are implicitly parallel mainly due to p owerful mo dule system allowing several implementations the p ossibility of using eager evaluation strategies for all the of the skeletons to b e substituted for one another without strict functions app earing in a program Eager evaluation changing the user co de and a builtin marshaler allowing strategies in conjunction with the referential transparency transmission of arbitrary data structures over byte streams prop erty sp orted by pure functional languages allow func based on the same structural information used by the ocaml tional languages compilers or interpreters to schedule in garbage collector 0 The ocamlpl pro ject has b een partially funded by the The goal of the authors at the b eginning of the ocamlpl Galileo bilateral FranceItaly pro ject n Further in pro ject was to assess the merits and the feasibility of the formation on the ocamlpl Ocaml and pl pro jects can integration of the pl language inside ocaml This gave b e found at the URLs httpwwwdiunipiitmarcod ocamlplocamlplhtml httppauillacinriafrocaml and ocamlpl a programming environment that allows to write httpwwwdiunipiitsusannaplhtml parallel programs in ocaml according to the skeleton mo del supp orted by the parallel language pl It provides seamless pip eline stages integration of parallel programming and functional program In ocamlpl we allow skeletons to b e arbitrarily com ming and advanced features like sequential logical debugging p osed and we provide the user with b oth stream parallel of parallel programs and strong typing useful b oth in teach and data parallel skeletons data parallelism b eing the one ing parallel programming and in building fullscale applica coming from p erforming in parallel computations relative to tions In addition we wanted the skeleton implementation a single input data item to run on widely available computer systems The skeleton set of ocamlpl contains most of the skele During the implementation of the system it turned out tons app earing in pl although some of the ocamlpl skele that we could get more than that In our implementation the tons are actually simplied versions of those app earing in user co de containing the skeleton expressions can b e linked pl here we will denote by f the function computed by with dierent mo dules in order to get either a parallel exe skeleton F on a single data item of its input data stream cutable running on a network of workstation or a sequen Farm skeleton The farm skeleton written farm computes tial executable running on a single workstation Therefore in parallel a function f over dierent data items app ear users are enabled to p erform logical program debugging us ing in its input stream From a functional viewp oint ing the sequential version of the program and then to move given a stream of data items x x farmF k to the parallel version by just changing a linker option 1 n computes f x f x F b eing the skeleton pa The high level of abstraction provided by functional pro 1 n rameter Parallelism is exploited by having k inde gramming coupled with the ability to send closures over an p endent parallel pro cesses computing f on dierent IO channel provided the key to an elementary and robust items of the input stream runtime system that consists of a very limited number of lines of co de Pip eline skeleton The pip eline skeleton denoted by the inx op erator p erforms in parallel the computa Skeletons tions relative to dierent stages of a function over dif ferent data items of the input stream Functionally The skeleton parallel programming mo del supp orts struc F F F computes f f f x over 1 2 N n 2 1 i tured parallel programming Using this mo del all the data items x b elonging to the input stream i the parallel structureb ehavior of an application has to b e Parallelism is exploited by having n indep endent par expressed by using skeletons picked up from of a set of prede allel pro cesses Each pro cess computes a function f i ned ones p ossibly in a nested way Each skeleton mo dels a over the data items pro duced by pro cess computing typical pattern of parallel computation and it is parametric f and delivers results to pro cess computing f i1 i+1 in the computation p erformed in parallel Skeletons can b e understo o d as second order functionals that mo del the par Map skeleton The map skeleton written mapvector allel computation coming out from the application of a given computes in parallel a function over all the data items parallelism exploitation pattern to the parameter functions of a vector generating a new vector Therefore for As an example a common skeleton is the pipeline each vector X app earing in the input data stream one Pip eline is a stream parallel skeleton The paral mapvector F n computes the function f over all the lelism exploited comes from p erforming in parallel compu items of the vector using n dierent parallel pro cesses tations relative to dierent input data sets app earing on computing f over distinct vector items the input data stream In particular the pip eline skele Reduce skeleton The reduce skeleton written ton mo dels the parallelism coming from the parallel com reducevector folds a binary asso ciative func putation of dierent stages of a function onto dierent tion over all the data items of a vector Therefore input data items app earing on the input data stream reducevector F n computes x f x f f x out of 1 2 n In other words pipeline f f computes the func 1 n the vector x x for each one of the vectors 1 n