Experience with Parallel Programming Using Code Templates
Total Page:16
File Type:pdf, Size:1020Kb
Experience with Parallel Programming Using Code Templates 1 2 2 Ajit Singh Jonathan Schaeer Duane Szafron asinghetudeuwaterlo oca jonathancsualb ertaca duanecsualb ertaca 1 2 University of Waterlo o University of Alb erta Dept of Electrical and Computer Eng Dept of Computing Science Waterlo o Ontario Edmonton Alb erta Canada NL G Canada TG H Abstract For almost a decade we have b een working at developing and using templatebased mo dels for parallel computing Templatebased mo dels separate the sp ecication of the parallel structuring asp ects from the applicati on co de that is to b e parallelized A user provides the applicatio n co de and sp ecies the parallel structure of the application using highlevel icons called templates The parallel programming system then generates the co de necessary for paralleliz in g the application The goal here is to provide a mechanism for quick and reliable development of coarsegrain parallel applicatio ns that employ frequently o ccurring parallel structures Our initial templatebased system FrameWorks was p ositively received but had a numb er of shortcomings The Enterprise parallel programming environment evolved out of this work Now after several years of exp erience with the system its shortcomings are b ecoming evident Controlled exp eriments have b een conducted to assess the usability of our system in comparison with other systems This pap er outlines our exp eriences in developing and using these systems A list of desirable characteristics of templatebased mo dels is given FrameWorks and Enterprise systems are discussed in the context of these characteristics and the results of our usability exp eriments Many of our observations are relevant to other parallel programming systems even though they may b e based on dierent assumptions Although templatebase mo dels have the p otential for simplifying the complexities of parallel programming they have yet to realize these exp ectations for highp erformance applications Exp erience with Parallel Programming Using Co de Templates Intro duction Along with the growing interest in parallel and distributed computing there has b een a corresp onding in crease in the development of mo dels to ols and systems for parallel programming Consequently practitioners in the area are now faced with a somewhat dicult challenge how to select parallel programming to ols that will b e appropriate for their applications There is no easy answer The decision is a function of many parameters including some that are sp ecic to the user and their computing environment These include the typ e of parallelism available in the application for example ne or coarsegrained data parallel or not pip eline or masterslave the target architectures for example shared or distributed memory language constraints and p erformance exp ectations Other parameters are sp ecic to the to ol and its capabilities including its feature set p ortability fault tolerance and usability ease of use exibility expressive p ower As is evident from the formation of user groups such as the Parallel To ols Consortium there is a concern in the community ab out the lack of p ostdevelopment analysis and evaluation for various to ols and technologies that are b eing prop osed Typically researchers envision a new to ol or technology develop it and dep ending on their initial exp eriences rep ort it in the literature With few exceptions longterm exp eriences with parallel programming systems and their relationships with similar systems are hardly ever rep orted Many dierent approaches have b een taken towards the development of parallel programming mo dels A new parallel programming language is one approach for example a pro cedural language such as Orca or a functional language like Sisal However practical considerations such as legacy co de and the demand for FortranCbased languages often make this an impractical choice Alternatives that allow the programmer to take advantage of the existing co de and exp ertise in common sequential languages have found much wider acceptance One such approach includes providing libraries for parallelization PVM P and MPI are examples Another approach is to extend an existing sequential language with compiler directives High Performance Fortran or keywords for example Mentat and PAMS A relatively new alternative has b egun to emerge that allows a programmer to b enet from the existing co de and knowledge of a sequential program while minimizing the mo dications that are required for parallelization The programmer provides a sp ecication of the parallel structuring asp ects of the application in the form of co de annotations One interesting approach to co de annotation is to recognize that there are commonly o ccurring parallel techniques A parallel programming to ol can supp ort these techniques by providing algorithmic skeletons or templates that capture the parallel b ehavior The user provides the sequential application co de and selects the templates required to parallelize the application such as in PIE Exp erience with Parallel Programming Using Co de Templates and HeNCE The system then generates the necessary parallel co de Templatebased mo dels separate the sp ecication of the parallel structuring asp ects such as synchronization communication and pro cesspro cess or mapping from the application co de that is to b e parallelized A template implements commonly o ccurring parallel interactions in an applicationindep endent manner The goal here is to provide an easy approach for the initial development and restructuring of coarsegrain parallel applications that relies on commonly used parallelization techniques This pap er discusses our longterm exp eriences with two templatebased parallel programming systems for coarsegrained parallelism Our research b egan in when we used templates to exp eriment with dierent parallel structures for a computer animation application We quickly realized that the approach was more general and could b e used to build a larger class of parallel applications Building on this success the FrameWorks parallel programming to ol was develop ed Our initial exp erience with FrameWorks was encouraging However for a numb er of reasons describ ed later in this pap er it was not p ossible to evolve the system b eyond a certain p oint Consequently an entirely new pro ject called Enterprise was initiated Enterprise is a templatebased parallel programming environment which oers a much wider range of related to ols for parallel program design co ding debugging and p erformance tuning It has b een publicly available since httpwebcsualb ertacaenter Several other parallel programming systems have relied on techniques that are similar to the approach used by us for example Many of our results and exp eriences with FrameWorks and Enterprise are applicable to such systems as well as other highlevel parallel programming systems Before we delve into details it is useful to clarify a couple of p oints regarding our use of the term template In the past techniques based on the use of applicationindep endent common parallel structures have often b een describ ed under dierent names such as algorithmic skeletons mo del programs based on parallel programming paradigms and parallel program archetyp es In addition to us some other researchers also have recognized or used the term template to refer to such techniques Although the underlying details of these techniques vary signicantly they all have the common goal of sp ecifying commonly o ccurring parallel structures in the form of applicationindep endent and reusable co de For the last ten years we have used the term templatebased to refer to this technique At this p oint it should also b e p ointed out that our use of the term template here is quite distinct from C templates Our usage is restricted to the context of parallel programming where it is used to denote a prepackaged set of applicationindep endent characteristics This has no intended relationship with the C templates Exp erience with Parallel Programming Using Co de Templates which are used to build generic classes in sequential programs In this pap er we lo ok at templatebased parallel programming mo dels from two viewp oints First as the designers we can address the diculties in the design and implementation of these to ols Second we have had considerable interaction with users developing templatebased parallel applications Controlled exp eriments which compared Enterprise with a numb er of to ols including PVM give insights ab out the strengths and weaknesses of the templatebased approach The result is that although templatebased mo dels have tremendous p otential for bridging the gap b etween sequential and parallel co de there still remain a numb er of shortcomings that must b e addressed b efore the technology will b e widely used Section describ es the templatebased approach Section explains the distinctions b etween this ap proach and other highlevel techniques for parallel programming Section outlines the ob jectives for an ideal templatebased parallel programming to ol and discusses their signicance Section briey describ es FrameWorks and its shortcomings These problems with FrameWorks led to the design of Enterprise as describ ed in Section Section describ es our exp eriences with templatebased mo dels and the lessons learned Extending the template mo del to other asp ects of parallel programming is discussed in Section Section describ es the requirements for future templatebased to ols