ORNL/TM-2007/238 Scientific Application Requirements for Leadership Computing at the Exascale December 2007 Prepared by Computing Requirements Team National Center for Computational Sciences DOCUMENT AVAILABILITY Reports produced after January 1, 1996, are generally available free via the U.S. Department of Energy (DOE) Information Bridge. Web site http://www.osti.gov/bridge Reports produced before January 1, 1996, may be purchased by members of the public from the following source. National Technical Information Service 5285 Port Royal Road Springfield, VA 22161 Telephone 703-605-6000 (1-800-553-6847) TDD 703-487-4639 Fax 703-605-6900 E-mail [email protected] Web site http://www.ntis.gov/support/ordernowabout.htm Reports are available to DOE employees, DOE contractors, Energy Technology Data Exchange (ETDE) representatives, and International Nuclear Information System (INIS) representatives from the following source. Office of Scientific and Technical Information P.O. Box 62 Oak Ridge, TN 37831 Telephone 865-576-8401 Fax 865-576-5728 E-mail [email protected] Web site http://www.osti.gov/contact.html This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States government nor any agency thereof, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof. ORNL/TM-2007/238 SCIENTIFIC APPLICATION REQUIREMENTS FOR LEADERSHIP COMPUTING AT THE EXASCALE Computing Requirements Team Sean Ahern Sadaf Alam Mark Fahey Rebecca Hartman-Baker Richard Barrett Ricky Kendall Douglas Kothe O.E. Messer Richard Mills Ramanan Sankaran Arnold Tharrington James B. White III Date Published: December 2007 Prepared by OAK RIDGE NATIONAL LABORATORY Oak Ridge, Tennessee 37831-6283 managed by UT-BATTELLE, LLC for the U.S. DEPARTMENT OF ENERGY under contract DE-AC05-00OR22725 CONTENTS Page LIST OF TABLES ........................................................................................................................................ v EXECUTIVE SUMMARY .......................................................................................................................... 1 1. INTRODUCTION AND SCIENCE DRIVERS..................................................................................... 2 2. MODEL AND ALGORITHM REQUIREMENTS ............................................................................... 5 3. SOFTWARE REQUIREMENTS ........................................................................................................... 7 4. SYSTEM REQUIREMENTS ................................................................................................................ 8 4.1 SYSTEM HARDWARE ................................................................................................................ 8 4.2 SYSTEM SOFTWARE ................................................................................................................ 12 5. DATA ANALYTICS REQUIREMENTS ........................................................................................... 12 6. ACCELERATING DEVELOPMENT AND READINESS ................................................................ 16 6.1 AUTOMATED DIAGNOSTICS ................................................................................................. 16 6.2 HARDWARE LATENCY ........................................................................................................... 16 6.3 HIERARCHICAL ALGORITHMS ............................................................................................. 16 6.4 PARALLEL PROGRAMMING MODELS ................................................................................. 16 6.5 SOLVER TECHNOLOGY AND INNOVATIVE SOLUTION TECHNIQUES ........................ 17 6.6 ACCELERATED TIME INTEGRATION ................................................................................... 18 6.7 MODEL COUPLING ................................................................................................................... 18 6.8 MAINTAINING CURRENT LIBRARIES .................................................................................. 18 7. SUMMARY ......................................................................................................................................... 19 ACKNOWLEDGMENTS .......................................................................................................................... 19 REFERENCES ........................................................................................................................................... 19 APPENDIX: QUESTIONNAIRE ON SCIENTIFIC APPLICATION REQUIREMENTS FOR LEADERSHIP COMPUTING IN THE NEXT DECADE ......................................................................... 21 iii LIST OF TABLES Table Page 1. Select science drivers for leadership computing at the exascale ............................................................ 4 2. Algorithms expected to play a key role within select scientific applications at the exascale, characterized according to a seven dwarfs classification ....................................................................... 6 3. Science application behavioral and algorithmic drivers for leadership system attributes ...................... 9 4. Relative prioritization of twelve leadership system attributes for relevant science domains ............... 10 5. Typical development characteristics and runtime requirements of a single simulation for selected application codes on the NCCS LCF systems circa June 2006 .............................................. 11 v EXECUTIVE SUMMARY The Department of Energy’s Leadership Computing Facility, located at Oak Ridge National Laboratory’s National Center for Computational Sciences, recently polled scientific teams that had large allocations at the center in 2007, asking them to identify computational science requirements for future exascale systems (capable of an exaflop, or 1018 floating point operations per second). These requirements are necessarily speculative, since an exascale system will not be realized until the 2015–2020 timeframe, and are expressed where possible relative to a recent petascale requirements analysis of similar science applications [1]. Our initial findings, which beg further data collection, validation, and analysis, did in fact align with many of our expectations and existing petascale requirements, yet they also contained some surprises, complete with new challenges and opportunities. First and foremost, the breadth and depth of science prospects and benefits on an exascale computing system are striking. Without a doubt, they justify a large investment, even with its inherent risks. The possibilities for return on investment (by any measure) are too large to let us ignore this opportunity. The software opportunities and challenges are enormous. In fact, as one notable computational scientist put it, the scale of questions being asked at the exascale is tremendous and the hardware has gotten way ahead of the software. We are in grave danger of failing because of a software crisis unless concerted investments and coordinating activities are undertaken to reduce and close this hardware- software gap over the next decade. Key to success will be a rigorous requirement for natural mapping of algorithms to hardware in a way that complements (rather than competes with) compilers and runtime systems. The level of abstraction must be raised, and more attention must be paid to functionalities and capabilities that incorporate intent into data structures, are aware of memory hierarchy, possess fault tolerance, exploit asynchronism, and are power-consumption aware. On the other hand, we must also provide application scientists with the ability to develop software without having to become experts in the computer science components. Numerical algorithms are scattered broadly across science domains, with no one particular algorithm being ubiquitous and no one algorithm going unused. Structured grids and dense linear algebra continue to dominate, but other algorithm categories will become more common. A significant increase is projected for Monte Carlo algorithms, unstructured grids, sparse linear algebra, and particle methods, and a relative decrease foreseen in fast Fourier transforms. These projections reflect the expectation of much higher architecture concurrency and the resulting need for very high scalability. The new algorithm categories that application scientists expect to be increasingly important in the next decade include adaptive mesh refinement, implicit nonlinear systems, data assimilation, agent-based methods, parameter continuation, and optimization. The attributes of leadership computing systems expected to increase most in priority over the next decade are (in order of importance) interconnect bandwidth, memory bandwidth, mean time to interrupt, memory latency, and interconnect