SANDIA REPORT SAND2007-XXXX Unlimited Release Printed Workshop on Programming Languages for High Performance Computing (HPCWPL) Final Report Prepared by Sandia National Laboratories Albuquerque, New Mexico 87185 and Livermore, California 94550 Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy’s National Nuclear Security Administration under Contract DE-AC04-94-AL85000. Approved for public release; further dissemination unlimited. Issued by Sandia National Laboratories, operated for the United States Department of Energy by Sandia Corporation. NOTICE: This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government, nor any agency thereof, nor any of their employees, nor any of their contractors, subcontractors, or their employees, make any warranty, express or implied, or assume any legal liability or responsibility for the accuracy, completeness, or usefulness of any infor- mation, apparatus, product, or process disclosed, or represent that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recom- mendation, or favoring by the United States Government, any agency thereof, or any of their contractors or subcontractors. The views and opinions expressed herein do not necessarily state or reflect those of the United States Government, any agency thereof, or any of their contractors. Printed in the United States of America. This report has been reproduced directly from the best available copy. Available to DOE and DOE contractors from U.S. Department of Energy Office of Scientific and Technical Information P.O. Box 62 Oak Ridge, TN 37831 Telephone: (865) 576-8401 Facsimile: (865) 576-5728 E-Mail: [email protected] Online ordering: http://www.osti.gov/bridge Available to the public from U.S. Department of Commerce National Technical Information Service 5285 Port Royal Rd Springfield, VA 22161 Telephone: (800) 553-6847 Facsimile: (703) 605-6900 E-Mail: [email protected] Online ordering: http://www.ntis.gov/help/ordermethods.asp?loc=7-4-0#online NT O E F E TM N R E A R P G E Y •D • U N A I C T I E R D E S M T A ATES OF 2 SAND2007-XXXX Unlimited Release Printed Workshop on Programming Languages for High Performance Computing (HPCWPL) Final Report Richard C. Murphy HPCWPL Technical Chair Abstract This report summarizes the deliberations and conclusions of the Workshop on Pro- gramming Languages for High Performance Computing (HPCWPL) held at the San- dia CSRI facility in Albuquerque, NM on December 12-13, 2006. 3 Acknowledgment This report would be impossible without the efforts of a large number of people. Each of the participants presenters provided invaluable expert input into our effort to understand the state of applications, architecture, and programming models. We are very thankful for their active participation. The workshop’s steering and program committees were invaluable in shaping its con- tent and direction: Ron Brightwell, Bill Carlson, David Chase, Bill Gropp, Bruce Hendrickson, Mike Heroux, Fred Johnson, Peter Kogge, Bob Lucas, Bob Numrich, Marc Snir, Thomas Sterling, Jeffrey Vetter, Christoph von Praun, Zhaofang Wen, and Hans Zima. We are thankful to the chairs for providing direction for each of their sessions: Pe- ter Kogge served as the workshop chair, Thomas Sterling chaired the architecture session, Jeffrey Vetter chaired the applications session, Marc Snir chaired the HPCS Languages Panel, which served as the basis for many of the recommendations given in this report, Bill Gropp chaired the programming models session, and Bob Lucas chaired final conclusions session. Each of the deputy chairs from Sandia also provided invaluable input into the final report: Keith Underwood, Mike Heroux, Arun Rodrigues, and Ron Brightwell. Richard C. Murphy, Technical Chair Neil D. Pundit, Sandia Host 4 Executive Summary This report describes the critical need for additional research in programming lan- guages for high performance computing to enable new applications and the architec- tures to support those applications. It is the result of a synthesis of the contents of the Workshop on Programming Languages for High Performance Computing (HPCWPL) held at Sandia National Laboratories’ Computer Science Research Institute on De- cember 12-13, 2006. This report makes the following observations: 1. Today’s High Performance Computing (HPC) applications consist of large, slowly evolving software frameworks. These frameworks are typically decades old and have already survived large transitions in computer architecture (e.g., Vector to MPP). 2. Emerging HPC applications are very different from their traditional scientific computing counterparts. As a result, the “lessons” from scientific computing do not address the problems of these applications. 3. Programmers within the DOE are typically experts. Consequently, performance is more important than rapid prototyping. 4. HPC workloads share little in common with commercial workloads. Thus, very little reuse can be made of commercial programming models. For example, Transactional Memory is touted as the correct mechanism for pro- gramming multicore architectures, but it is inappropriate for HPC workloads. 5. The memory wall dominates computer architecture and is increas- ingly problematic. Even in the commodity space, the transition to multi- core architectures will tremendously increase the demands on memory systems. Memory latency dominates the performance of individual processor cores, while memory bandwidth dominates the number of cores that can be put on a die. Both are problems. 6. While supercomputer scaling was previously dominated by intercon- nect performance future machine scaling will be dominated by mem- ory. Because current programming models focus on interconnection networks rather than memory, this could significantly limit the ability to scale applica- tions. 7. Both conventional and emerging architectures require additional con- currency to be easily specified. MPI is virtually certain not to be able to provide all of the required concurrency. 5 8. Emerging heterogeneous architectures (such as Cell) pose a significant workload partitioning problem. The job of performing this partitioning is currently the programmer’s, and there is little hope for automation on the horizon. As a result, the following recommendations are proffered: I. We should pursue an evolutionary rather than revolutionary approach to HPC languages. There are both historical precedents (in the form of MPI) and practical requirements (in the form of a multi-billion dollar investment in existing code) that dictate this path. These languages must be compatible with existing tools and legacy code. II. The evolutionary path should address application requirements first and machine architecture requirements second (but should address both). New programming models (and architectures) have already been adopted to solve problems that cannot be addressed using the conventional MPI/MPP model. There are common features among radically different problems that cannot be solved today that should be the focus of new programming language extensions. These features include requirements for fine-grained parallelism, synchronization, and data access. III. An evolutionary language should introduce a minimal set of abstrac- tions, which should match HPC’s unique requirements. MPI succeeded by introducing a very small number of key routines that mapped well onto ap- plications and architectures. In the commercial space, Transactional Memory introduces small extensions to introduce nondeterminism into programs (and find parallelism). However, HPC applications typically require deterministic parallelism. IV. Languages need to support large numbers of efficient, light-weight, locality aware threads. This serves to increase concurrency, address the core problem of the memory wall, and begin to address the cache coherency problem. V. We must identify problems that cannot be solved today, and de- velop programming languages (and architectures) that solve those problems. This supports recommendations I-IV by enabling the creation of a significant user-base to support the early development of these languages, and focusing language development on satisfying a specific need. We are currently at a unique point in the evolution of HPC. Application and un- derlying technology requirements dictate programming models to support new ways of solving humanity’s problems. Today, architectures and applications are evolving faster than the programming models needed to support them. These recommenda- tions are intended as a first cut strategy for addressing those needs and enabling the solution to new, larger-scale problems. 6 Table 1. HPCWPL Participants Name Affiliation Name Affiliation Berry, Jon SNL Lucas, Bob ISI Brightwell, Ron SNL Maccabe, Barney UNM Chamberlain, Brad Cray Mehrotra, Piyush NASA Chase, David Sun Murphy, Richard SNL Chtchelkanova, Almadena NSF Numrich, Bob U. of Minnesota DeBenedictis, Erik SNL Olukotun, Kunle Stanford DeRose, Luiz Cray Perumalla, Kalyan ORNL Doerfler, Doug SNL Poole, Stephen ORNL Gao, Guang U. of Delaware Pundit, Neil SNL Gokhale, Maya LANL Rodrigues, Arun SNL Gropp, Bill ANL Snir, Marc UIUC Hall, Mary USC/ISI Sterling, Thomas LSU Hecht-Nielsen, Robert UCSD Underwood, Keith SNL Hemmert,