Sustainable Software for Computational Chemistry and Materials Modeling Daniel Crawford (Virginia Tech), Robert Harrison (U. Tennessee), Anna I. Krylov (U. Southern California), Theresa Windus (Iowa State), Emily Carter (Princeton U.), Edmund Chow (Georgia Tech), Erik Deumens (U. Florida), Mark Gordon (Iowa State U.), Martin Head-Gordon (U. C. Berkeley), Todd Martinez (Stanford U.), David McDowell (Georgia Tech), Vijay Pande (Stanford U.), Manish Parashar (Rutgers U.), Ram Ramanujam (LSU), Beverly Sanders∗ (U. Florida), Bernhard Schlegel (Wayne State U.), David Sherrill (Georgia Tech), Lyudmila Slipchenko (Purdue U.), Masha Sosonkina (Iowa State U.), Edward Valeev (Virginia Tech), Ross Walker (San Diego Supercomputing Center) ∗Corresponding author: [email protected]fl.edu Abstract—Computational chemistry underpins a broad array sustainability and extension of the software on ever-changing of vital scientific applications and is one of the pillars of high-performance computing hardware. For example, while computational science. However, the field also faces some of quantum chemistry has produced some of the most accurate the most daunting software challenges — perhaps greater than any field of computational science. This position paper describes and reliable computational models of molecular properties, some of the main challenges inhibiting the development of such highly accurate models have yet to take full advantage sustainable and reusable software in computational chemistry of modern high-performance computing architectures. Code and materials modeling. The authors are participants in the complexity also hinders the education of the next generation recently established Scientific Software Innovation Institute for of computational chemists. Many incoming chemistry graduate Computational Chemistry and Materials Modeling (S2I2C2M2). students have little programming experience, and they fre- quently lack advanced math skills. This often makes it difficult I. INTRODUCTION for them to get up to speed in theory and code development. Computational chemistry underpins a broad array of vital The authors of this position paper are participants in the scientific applications such as designing more efficient com- recently established Scientific Software Innovation Institute bustion systems, drug design and development, understanding for Computational Chemistry and Materials (S2I2C2M2 ), biological systems, semiconductor design, water sustainability, one of the goals of which is to overcome these obstacles advanced materials, and CO2 sequestration. It is one of the of both algorithms and culture and change the fundamental pillars of computational science, and its impact reaches well nature of computational chemistry software development. The beyond chemistry into biomolecular and polymer physics, team includes computational chemists, computer scientists, materials science, and condensed-matter physics. Over its applied mathematicians, and computer engineers to attack the long history, which stretches back more than half a century, fundamental problems of software complexity and education. computational chemistry has achieved a much-deserved status A year-long conceptualization phase is expected to be funded as a full partner with experiment in scientific discovery, by NSF. yielding simulations of such high accuracy that its predic- tions of a variety of molecular properties may be considered II. MAJOR CHALLENGES “computational experiments,” often with greater reliability Computational chemistry faces a broad array of software than laboratory measurements for many chemical properties. challenges. Advanced molecular models exhibit astonishing However, computational chemistry also faces some of the most levels of complexity that can easily eclipse the computing daunting software challenges — perhaps greater than any field resources of even the world’s most powerful supercomputers. of computational science. Faithful and robust molecular dynamics simulations of physi- The history of computational chemistry endows the field cally realistic, large-scale chemical problems in heterogeneous not only with great experience, but also with a legacy of catalysis, drug design, or carbon capture will involve exabytes diverse and complex code stacks. Many molecular dynam- of output data that must be quickly mined and visualized. ics and quantum chemistry programs involve hundreds of The high-order polynomial scaling of the current generation of thousands to even a million lines of hand-written code in hyper-accurate electronic structure models required for the de- a variety of languages, including Fortran-77, Fortran-90, C, sign of advanced materials such as future organic photovoltaic and C++. While this complexity has arisen naturally from cells can demand ultrafast computing and communication re- the intricacy of the problems these programs were designed sources, such as the exascale parallel architectures anticipated to solve, it also presents a crucial obstacle to the long-term in the coming decade. The level of accuracy and scaling needed for computational chemistry to face these scientific of which are relatively small — places a heavy burden on Grand Challenges requires innovative algorithms, software their shoulders. A community-oriented approach to the training development techniques, and a new approach to education that of new students in computational science has the potential to are far beyond the current state of the art. yield great rewards for both students and software. A. Current State of Computational Chemistry III. OVERCOMING THE BARRIERS The history of computational chemistry software develop- In this section, we discuss the highest priority issues for ment reaches back to the earliest days of computing, and the overcoming the barriers to sustainable software in computa- legacies of many modern molecular dynamics and quantum tional chemistry and materials modeling. chemistry packages span decades. These programs1, which include both open-source and commercial licenses, involve A. Portable Parallel Infrastructure many hundreds of thousands to even a million lines of hand- Software developers will need to navigate the next decade of written code in languages such as Fortran-77, Fortran-90, C, rapid change in computer technology that impacts everything and C++ (some using several such languages). Such diverse spanning from laptops in the classroom, through college clus- code stacks require vast numbers of human hours and teams ters, to extreme scale supercomputers. The relevant technology of many programmers to develop and maintain. Their effort trends to consider are: has resulted in an array of new software technology, including both domain-specific code such as multi-dimensional integral • Massive concurrency within a single socket evidenced by engines and general-purpose algorithms such as the Davidson multi-core processors (e.g., currently 128 x86 thread units method for computing eigenvalues of large matrices (some or more on single chip) with increasingly long vector with ranks in the tens of billions). units (e.g., 512 bytes on Intel MIC), and many-core light The Gordian complexity of computational chemistry pro- weight processors (e.g., currently over 10k+ on a chip). grams is a natural reflection of the intricacy of the prob- • Massive numbers of sockets in the largest supercomputers lems they are designed to solve. Modern high-accuracy ab implying that bleeding-edge science must coordinate over 9 initio quantum chemical simulations, for example, involve vast 10 threads possibly with applications needing for the numbers of wave function parameters that must be computed first time to tolerate faults in the system. and stored, with computational scalings of O(N 7) (or even • Complex memory hierarchies including limited coher- higher), where N represents the size of the molecular sys- ence, partitioned address spaces, and software managed tem (e.g., numbers of atoms, electrons, or basis functions). memories. As a result, much of the computational chemistry software • Memory and communication bandwidth and capacity (not development in the last 30 years has focused on shared- FLOP/s) being the primary constraints on performance, and distributed-memory parallel architectures, with varying cost, and energy consumption. levels of success. Programs such as NWChem, MADNESS, Current programming models in computational chemistry and MPQC were originally designed for parallel computing are mostly designed for machines of the past and have not systems, while most others have been adapted or extended over kept pace with the underlying technology, and more recent time to try take advantage of new hardware as it has become models such as CUDA are suitable only for a small subset available to the developers. A few development groups have of applications with appropriate massive, uniform parallelism. intentionally made extensive software modifications to take Powerful new concepts for asynchronous distributed computa- advantage of high performance computing, with GAMESS, tion, such as found in the DARPA HPCS languages (Chapel, MPQC, ACESIII, and Molpro serving as prime examples. X10, Fortress), are often poorly understood by the campus In addition, the majority of the software development in computational community and, moreover, are only part of the computational chemistry has been carried out by graduate required solutions. Work is needed to frame and answer key students and postdoctoral associates, some of whom plan questions concerning sustainable programming