Spack Tutorial on AWS The most recent version of these slides can be found at: July 28-29, 2020 https://spack-tutorial.readthedocs.io LLNL-PRES-806064 This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under contract DE-AC52-07NA27344. Lawrence Livermore National Security, LLC Tutorial Materials Download the latest version of slides and handouts at: spack-tutorial.readthedocs.io For more: § Spack website: spack.io § Spack GitHub repository: github.com/spack/spack § Spack Reference Documentation: spack.readthedocs.io Q&A on Slack: spackpm.herokuapp.com Tutorial materials: spack-tutorial.readthedocs.io 2 LLNL-PRES-806064 Software complexity in HPC is growing glm suite-sparse yaml-cpp metis cmake ncurses parmetis pkgconf nalu hwloc libxml2 xz bzip2 openssl boost trilinos superlu openblas netlib-scalapack mumps openmpi zlib netcdf hdf5 matio m4 libsigsegv parallel-netcdf Nalu: Generalized Unstructured Massively Parallel Low Mach Flow Q&A on Slack: spackpm.herokuapp.com Tutorial materials: spack-tutorial.readthedocs.io 3 LLNL-PRES-806064 Software complexity in HPC is growing adol-c automake autoconf perl glm suite-sparse yaml-cpp metis cmake ncurses gmp libtool parmetis pkgconf m4 libsigsegv nalu hwloc libxml2 xz bzip2 openssl p4est pkgconf boost hwloc libxml2 trilinos superlu openblas xz netlib-scalapack netcdf-cxx netcdf mumps openmpi zlib netcdf hdf5 nanoflann matio m4 libsigsegv parallel-netcdf matio hdf5 Nalu: Generalized Unstructured Massively Parallel Low Mach Flow sundials glm netlib-scalapack mumps openmpi trilinos hypre openblas gdbm superlu-dist parmetis sqlite readline zlib petsc python dealii slepc ncurses arpack-ng openssl metis cmake gmsh oce suite-sparse tetgen intel-tbb netgen bzip2 assimp boost gsl muparser dealii: C++ Finite Element Library Q&A on Slack: spackpm.herokuapp.com Tutorial materials: spack-tutorial.readthedocs.io 4 LLNL-PRES-806064 Software complexity in HPC is growing r-randomforest glm suite-sparse yaml-cpp r-labeling metis cmake ncurses parmetis pkgconf nalu r-lazyeval r-munsell hwloc libxml2 xz r-colorspace bzip2 openssl r-rcolorbrewer boost r-viridislite trilinos superlu openblas r-dichromat netlib-scalapack r-scales mumps r-tibble r-assertthat openmpi zlib netcdf hdf5 r-rlang r-gtable matio m4 libsigsegv parallel-netcdf r-ggplot2 r-plyr r-reshape2 r-rcpp r-modelmetrics Nalu: Generalized Unstructured Massively Parallel Low Mach Flow freetype bzip2 gperf r-r6 r-minqa fontconfig r-caret r-nloptr r-digest cairo libxml2 r-rcppeigen pixman r-lme4 harfbuzz libpng r-praise r-testthat pango adol-c automake sed xz gobject-introspection autoconf perl glib gettext r-mgcv r-crayon tar gmp libtool m4 libsigsegv libffi r-nlme p4est pkgconf libtiff hwloc libxml2 r-pbkrtest help2man xz r-mass zlib r-car r-foreach r-iterators flex netcdf-cxx netcdf r-sparsem curl r python ncurses nanoflann pkgconf openssl readline matio r-quantreg r-matrixmodels pcre hdf5 jdk libjpeg-turbo r-matrix cmake sundials r-lattice icu4c sqlite font-util r-adabag r-glmnet glm tk tcl nasm libxau gdbm xproto netlib-scalapack r-cubist r-mlbench libxdmcp openmpi xextproto mumps r-magrittr perl libpthread-stubs trilinos xtrans r-data-table libxcb util-macros hypre r-stringr xcb-proto r-xgboost libx11 inputproto r-rminer r-e1071 r-class r-codetools openblas kbproto gdbm bison r-kknn r-igraph r-irlba r-stringi r-rpart automake superlu-dist parmetis sqlite autoconf readline r-nnet zlib r-pkgconfig petsc gmp libtool m4 libsigsegv python dealii slepc ncurses arpack-ng openssl r-mda cmake metis r-kernlab gmsh oce suite-sparse r-strucchange r-zoo tetgen r-sandwich intel-tbb r-party netgen r-survival bzip2 r-multcomp r-th-data assimp r-plotrix boost r-coin r-mvtnorm r-modeltools gsl r-pls muparser dealii: C++ Finite Element Library R Miner: R Data Mining Library Q&A on Slack: spackpm.herokuapp.com Tutorial materials: spack-tutorial.readthedocs.io 5 LLNL-PRES-806064 What is the “production” environment for HPC? § Someone’s home directory? § LLNL? LANL? Sandia? ANL? LBL? TACC? — Environments at large-scale sites are very different. § Which MPI implementation? § Which compiler? § Which dependencies? § Which versions of dependencies? — Many applications require specific dependency versions. Real answer: there isn’t a single production environment or a standard way to build. Reusing someone else’s software is HARD. Q&A on Slack: spackpm.herokuapp.com Tutorial materials: spack-tutorial.readthedocs.io 6 LLNL-PRES-806064 The complexity of the exascale ecosystem threatens productivity. 5+ target architectures/platforms 15+ applications x 80+ software packages x Xeon Power KNL NVIDIA ARM Laptops? Up to 7 compilers 10+ Programming Models 2-3 versions of each package x Intel GCC Clang XL x OpenMPI MPICH MVAPICH OpenMP CUDA x PGI Cray NAG OpenACC Dharma Legion RAJA Kokkos + external dependencies = up to 1,260,000 combinations! § Every application has its own stack of dependencies. § Developers, users, and facilities dedicate (many) FTEs to building & porting. § Often trade reuse and usability for performance. We must make it easier to rely on others’ software! Q&A on Slack: spackpm.herokuapp.com Tutorial materials: spack-tutorial.readthedocs.io 7 LLNL-PRES-806064 What aBout containers? § Containers provide a great way to reproduce and distribute an already-built software stack § Someone needs to build the container! — This isn’t trivial — Containerized applications still have hundreds of dependencies § Using the OS package manager inside a container is insufficient — Most binaries are built unoptimized — Generic binaries, not optimized for specific architectures § HPC containers may need to be rebuilt to support many different hosts, anyway. — Not clear that we can ever build one container for all facilities — Containers likely won’t solve the N-platforms problem in HPC We need something more flexible to build the containers Q&A on Slack: spackpm.herokuapp.com Tutorial materials: spack-tutorial.readthedocs.io 8 LLNL-PRES-806064 Spack is a flexiBle package manager for HPC § How to install Spack: $ git clone https://github.com/spack/spack $ . spack/share/spack/setup-env.sh § How to install a package: $ spack install hdf5 § HDF5 and its dependencies are installed within the Spack directory. § Unlike typical package managers, Spack can also install github.com/spack/spack many variants of the same build. — Different compilers — Different MPI implementations @spackpm — Different build options Q&A on Slack: spackpm.herokuapp.com Tutorial materials: spack-tutorial.readthedocs.io 9 LLNL-PRES-806064 Who can use Spack? People who want to use or distribute software for HPC! 1. End Users of HPC Software — Install and run HPC applications and tools 2. HPC Application Teams — Manage third-party dependency libraries 3. Package Developers — People who want to package their own software for distribution 4. User support teams at HPC Centers — People who deploy software for users at large HPC sites Q&A on Slack: spackpm.herokuapp.com Tutorial materials: spack-tutorial.readthedocs.io 10 LLNL-PRES-806064 Spack is used worldwide! Over 4,300 software packages Over 2,900 monthly active users (on docs site) Over 600 contributors from labs, academia, industry Plot shows sessions on spack.readthedocs.io for one month Q&A on Slack: spackpm.herokuapp.com Tutorial materials: spack-tutorial.readthedocs.io 11 LLNL-PRES-806064 Users on our documentation site have also been increasing Q&A on Slack: spackpm.herokuapp.com Tutorial materials: spack-tutorial.readthedocs.io 12 LLNL-PRES-806064 Spack is Being used on many of the top HPC systems § Official deployment tool for the U.S. Exascale Computing Project § >= 5 of the top 10 supercomputers § High Energy Physics community — Fermilab, CERN, collaborators § Astra (Sandia) #1 Fugaku OSS software stack is deployed with Spack § Fugaku (Japanese National Supercomputer Project) Summit (ORNL), Sierra (LLNL) SuperMUC-NG (LRZ, Germany) Edison, Cori, Perlmutter (NERSC) Q&A on Slack: spackpm.herokuapp.com Tutorial materials: spack-tutorial.readthedocs.io 13 LLNL-PRES-806064 One month of Spack development is pretty Busy! Q&A on Slack: spackpm.herokuapp.com Tutorial materials: spack-tutorial.readthedocs.io14 14 LLNL-PRES-806064 ContriButions to Spack continue to grow! § In November 2015, LLNL provided most of the contributions to Spack § Since then, we’ve gone from 300 to over 4,000 packages § Most packages are from external contributors! § Many contributions in core, as well. § We are committed to sustaining Spack’s open source ecosystem! Q&A on Slack: spackpm.herokuapp.com Tutorial materials: spack-tutorial.readthedocs.io 15 LLNL-PRES-806064 Spack 0.15 was released at the end of June § Many new features: — Packages can specify how they should be found on the system • spack external find command — Source code mirror for all Spack packages (from 0.15.1) — Testing builds in GitLab for all packages in E4S — Better Cray support — Better compiler optimization support on macOS • apple-clang now its own compiler — Enhancements and simplification to configuration • spack config add / spack config remove § Over 4,300 packages — 430 new since 0.14 in November 2019 § Several bugfix releases since (0.15.1, 0.15.2, 0.15.3) Q&A on Slack: spackpm.herokuapp.com Tutorial materials: spack-tutorial.readthedocs.io 16 LLNL-PRES-806064 Spack is not the only tool that automates builds 1. “Functional” Package Managers — Nix https://nixos.org/ — GNU Guix https://www.gnu.org/s/guix/ 2. Build-from-source Package