Evaluation of UPC Programmability Using Classroom Studies Carlos Teijeiro, Guillermo L. Damian´ A. Mallon,´ Andres´ Brian Wibecan Taboada, Juan Tourino,˜ Basilio B. Gomez,´ J. Carlos Mourino˜ Fraguela, Ramon´ Doallo Computer Architecture Group Galicia Supercomputing Center Hewlett-Packard University of A Coruna,˜ Spain Santiago de Compostela, Spain Nashua (NH), USA fcteijeiro,taboada,juan,basilio,[email protected] fdalvarez,agomez,[email protected] [email protected] Abstract—The study of a language in terms of programmability the participants are a heterogeneous group of research staff is a very interesting issue in parallel programming. Traditional at the Galicia Supercomputing Center (CESGA) [2]. These approaches in this field have studied different methods, such two groups present special features that are interesting for as the number of Lines of Code or the analysis of programs, in order to prove the benefits of using a paradigm compared the analysis, specially in terms of programmer profile. The to another. Nevertheless, these methods usually focus only on students at UDC are a quite homogeneous group, with minor code analysis, without giving much importance to the conditions variations in their academic curricula, but the staff at CESGA of the development process and even to the learning stage, or present clearly different profiles, as they have different degrees the benefits and disadvantages of the language reported by from different universities. the programmers. In this paper we present a methodology to accomplish a programmability study with UPC (Unified Parallel This paper is organized as follows. First, some general C) through the use of classroom studies with a group of novice remarks on programmability in High Performance Computing UPC programmers. This work will show the design of these (HPC) are made, and the related work on UPC in this field sessions and the analysis of the results obtained (code analysis is presented. Then, the design of the activities is explained, and survey responses). Thus, it is possible to characterize the and the codes and software used in the sessions are presented. current benefits and disadvantages of UPC, as well as to report some desirable features that could be included in this language Afterwards, detailed information about the most relevant re- standard. sults is given. Finally, some conclusions are extracted from this study. I. INTRODUCTION In order to measure the benefits of using UPC, a PGAS II. PROGRAMMABILITY IN HPC (Partitioned Global Address Space) extension to C, for parallel In the last years, several works on programmability and programming, a specific methodology based on classroom productivity for HPC have been developed. The most impor- studies has been used. This work presents the results of two tant results on this area are related to the High Productivity classroom studies, which consist of four-hour sessions with a Computer Systems (HPCS) project [3], funded by DARPA, group of UPC-unexperienced programmers organized in differ- which led to the proposal of three relevant languages that ent stages. First, the participants fill out a form to characterize focus on programmability (X10, Chapel and Fortress) [4]. their profile. Then, a seminar explaining the basic constructs These languages have been designed specifically to improve of UPC (using a slideshow and some practical examples) programmability and productivity in code development, but is given to the programmers. Afterwards they are asked to they are not mature enough yet. Some other languages, such parallelize several sequential codes in order to test the acquired as the PGAS-based UPC, Co-Array Fortran or Titanium, have skills. Finally data about their impressions on UPC and some been designed as extensions of well-known programming lan- detected benefits and disadvantages are obtained. The main guages (C, Fortran and Java, respectively) to provide parallel advantages of this approach are (1) the time control of the programming by means of the addition of syntactic constructs. development stage, (2) the use of unexperienced programmers Many studies related with this subject are usually devoted in the studied language but with some general background to general considerations and requirements for a parallel lan- knowledge on different programming paradigms, and (3) the guage in terms of productivity [5], as well as comments about inclusion of their opinions as a complement to the analysis of benefits and disadvantages of the most popular approaches the developed codes, which gives some guidelines to identify (MPI and OpenMP). Moreover, there are also some works desirable features in a parallel programming language. This on programmability in HPC devoted to the proposal and information is very useful for future UPC programmability analysis of different metrics [6], [7] and the design of specific improvements. In the first session, the participants are final benchmarks [8]. An interesting conclusion from some of these year students of the B.S. in Computer Science Engineering [1] studies is that a language is considered to be good in terms at University of A Coruna˜ (UDC), whereas in the second one of programmability if it contains expressive constructs which allow a more compact and simple coding, hence making low- evaluation. These restricted classroom studies allow to control level complexity transparent to the user. However, program- the duration and keep track of the whole development process, ming languages are difficult to compare in terms of real as well as to ensure that all participants have the same initial programmability, because programmability-oriented languages and basic knowledge of UPC (presented at the beginning of are quite novel and code developers are used to working with the session). Control on the information flow is performed by the most popular approaches. Therefore, the success of these using logging tools. new languages for HPC seems to be bound only to their The two sessions organized at UDC and CESGA have productivity enhancements, whereas extensions of traditional followed the same overall structure. First, every participant languages can benefit from a more broad acceptance. was asked to fill out an initial questionnaire about his academic Additionally, a good way to prove if a language provides profile and parallel programming background, as well as his good programmability is to make a survey on a group of interest on this area. The questions have been adapted to the programmers. The benefits and disadvantages reported by participants of each study. various code developers, especially when they have different After the initial test, the participants attended a seminar skills, can give valuable information about the programming on UPC. The contents of this talk were taken from slides language and also help guess if it could become popular among used in UPC seminars at UC Berkeley [18], and included all the parallel programming community. basic concepts and constructs needed to understand the PGAS paradigm (shared address space vs. private address space) and III. RELATED WORK develop UPC codes (upc forall construct, barrier synchroniza- Until now, most studies on UPC have focused on perfor- tions, pointers, distribution of shared arrays and raw memory mance, and there are still very few specific works on UPC copies). During this talk, the students were asked to test some related to programmability. The most relevant previous works sample codes (Hello World, Pi Computation using the Monte deal with constructs and algorithms focused on performance Carlo Approach and Matrix-Vector Multiplication). The UPC increase [9], [10] compared to other approaches measuring seminar, including the explanations and the execution of the programming effort in terms of Lines Of Code (LOC) [11]. test codes, lasted about 1 hour and 30 minutes. Classroom programmability sessions are used as a good Then, at the development stage, the participants were asked reference in order to measure productivity. Some tests with to develop three parallel codes in UPC from their sequential homogeneous groups of students have also been carried out, versions, implemented in C, that were given to them. The obtaining general conclusions for different languages and overall coding time was 2 hours. The three proposed codes paradigms [12], [13], [14]. UPC has also been considered for are: 6 a programmability study [15] in a comparison with MPI that • A simple Stencil operation on a 10 -element vector, includes an analysis of statistical significance of the results. analogous to one of the example codes included in the However, the development times were not taken into account Berkeley UPC distribution. The vector has 106 elements and the experimental conditions of the study were undefined. and the operation is performed 100 times (it uses an In these studies, the typical measures that are used to eval- external 100-iteration loop). uate programmability are Source LOC (SLOC) and speedup, • The Buffon-Laplace Needle problem, a Monte Carlo directly measured from the codes developed during a pro- simulation that gives an accurate approximation of Pi grammability session. Here, special applications to manage log based on the probability that a needle of length l that and report complete information about the work performed by is dropped in a grid of equally spaced parallel lines will each participant in the study are generally used [16]. Among touch at least one line. The number of predefined trials these tools it is worth mentioning UMDInst [17], which is 107. consists of a set of wrappers that create XML logs including • The Computation of the Minimum Distance among dif- the most relevant actions performed at code development ferent nodes in a graph (a version of the Floyd-Warshall (edition, compilation and execution), also saving snapshots algorithm [19]). This code was implemented by some of of the codes. These features help to give a more accurate the students at UDC as an MPI project during a previous measure of the development time and cost associated to the course on parallel programming. The size of the source parallelization of a code.