Proposal for a Performance and Scaling Benchmark Set for Electronic Structure Calculations
William Huhn, Victor Yu, Wenhui Mi, Volker Blum MEMS Department, Duke University
ELSI Connector Meeting 2017 Duke University Durham, NC, USA August 16, 2017 What Solver To Use?
ELSI provides an interface layer between KS-DFT codes and a KS-DFT Codes number of solvers (ELPA, libOMM, PEXSI, SIPs, CheSS…) H S φ,ε DM
Matrix ELSI Conversion
H’ S’ φ’,ε DM’
Solvers
ELSI Connector Meeting Duke University [email protected] 1/12 What Solver To Use?
ELSI provides an interface layer between KS-DFT codes and a KS-DFT Codes number of solvers (ELPA, libOMM, PEXSI, SIPs, CheSS…) H S Solver φ,ε DM
Matrix but currently the user must specify ELSI Conversion the solver H’ S’ φ’,ε DM’
Solvers
ELSI Connector Meeting Duke University [email protected] 1/12 What Solver To Use?
ELSI provides an interface layer between KS-DFT codes and a KS-DFT Codes number of solvers (ELPA, libOMM, PEXSI, SIPs, CheSS…) H S Solver φ,ε DM
Matrix but currently the user must specify ELSI Conversion the solver H’ S’ φ’,ε DM’ What can happen if user does not select the optimal solver for their Solvers system?
ELSI Connector Meeting Duke University [email protected] 1/12 The Price for Choosing Incorrectly
From Victor’s talk several hours ago
ELSI Connector Meeting Duke University [email protected] 2/12 The Price for Choosing Incorrectly
From Victor’s talk several hours ago
ELSI Connector Meeting Duke University [email protected] 2/12 The Price for Choosing Incorrectly
Difference between PEXSI and ELPA: 200 seconds * 80 nodes * 24 cores/node
From Victor’s talk several hours ago
ELSI Connector Meeting Duke University [email protected] 2/12 The Price for Choosing Incorrectly
Difference between PEXSI and ELPA: 200 seconds * 80 nodes * 24 cores/node = 107 hours per scf step
From Victor’s talk several hours ago
ELSI Connector Meeting Duke University [email protected] 2/12 Proposed ELSI Decision Layer
KS-DFT Codes Our next major implementation in H S ELSI will be a “decision layer” which automatically recommends Decision Layer φ,ε a solver based on user-provided DM matrices H S Solver
Matrix ELSI Conversion
H’ S’ φ’,ε DM’ Solvers
ELSI Connector Meeting Duke University [email protected] 3/12 Proposed ELSI Decision Layer
KS-DFT Codes Our next major implementation in H S ELSI will be a “decision layer” which automatically recommends Decision Layer φ,ε a solver based on user-provided DM matrices H S Solver
Matrix ELSI Conversion How do we model the performance of a real-world H’ S’ φ’,ε DM’ calculation? Solvers
ELSI Connector Meeting Duke University [email protected] 3/12 Accuracy Benchmarking
Benchmarks for KS-DFT codes re: accuracy for materials are increasingly becoming popular in the materials science literature:
Lejaeghere et al., Science 351, aad3000 (2017) https://molmod.ugent.be/deltacodesdft, accessed 23 June 2017
ELSI Connector Meeting Duke University [email protected] 4/12 Accuracy Benchmarking
Benchmarks for KS-DFT codes re: accuracy for materials are increasingly becoming popular in the materials science literature:
71 E(V) curves 15 DFT codes
Lejaeghere et al., Science 351, aad3000 (2017) https://molmod.ugent.be/deltacodesdft, accessed 23 June 2017
ELSI Connector Meeting Duke University [email protected] 4/12 Accuracy Benchmarking
Benchmarks for KS-DFT codes re: accuracy for materials are increasingly becoming popular in the materials science literature:
https://molmod.ugent.be/deltacodesdft, accessed 15 August 2017
ELSI Connector Meeting Duke University [email protected] 4/12 Accuracy Benchmarking
Benchmarks for KS-DFT codes re: accuracy for materials are increasingly becoming popular in the materials science literature:
Huhn and Blum, Phys. Rev. Mater., in press (2017) arXiv:1705.01804v2
ELSI Connector Meeting Duke University [email protected] 4/12 Accuracy Benchmarking
Benchmarks for KS-DFT codes re: accuracy for materials are increasingly becoming popular in the materials science literature:
103 Band Structures WIEN2k and FHI-aims
Huhn and Blum, Phys. Rev. Mater., in press (2017) arXiv:1705.01804v2
ELSI Connector Meeting Duke University [email protected] 4/12 Accuracy Benchmarking
Benchmarks for KS-DFT codes re: accuracy for materials are increasingly becoming popular in the materials science literature:
http://nomad-repository.eu, accessed 15 August 2017
http://dx.doi.org/10.17172/NOMAD/2017.04.20-1, accessed 15 August 2017
ELSI Connector Meeting Duke University [email protected] 4/12 Performance/Scaling Benchmarking
What if we are interested in the performance/scaling of a code or solver library?
ELSI Connector Meeting Duke University [email protected] 5/12 Performance/Scaling Benchmarking
What if we are interested in the performance/scaling of a code or solver library? [for matrix inversion] matrix [for
sparsity pattern taken from BigDFT, non-zero values randomized
From CheSS Manuscript (arXiv:1704.00512v1)
ELSI Connector Meeting Duke University [email protected] 5/12 Performance/Scaling Benchmarking
What if we are interested in the performance/scaling of a code or solver library?
http://www.castep.org/CASTEP/Benchmarks, accessed August 16 2017
ELSI Connector Meeting Duke University [email protected] 5/12 Performance/Scaling Benchmarking
What if we are interested in the performance/scaling of a code or solver library?
From Victor’s talk several hours ago
ELSI Connector Meeting Duke University [email protected] 5/12 Performance/Scaling Benchmarking
What if we are interested in the performance/scaling of a code or solver library?
There is no systematic set of performance benchmark systems in the literature! Why?
From Victor’s talk several hours ago
ELSI Connector Meeting Duke University [email protected] 5/12 What Can Impact the Performance of a Solver?
The matrix properties of H and S can vary considerably based on: • The physics of the system (HOMO-LUMO gap) • Choice of basis set (size, sparsity, condition number…) • Pseudization/all-electron
ELSI Connector Meeting Duke University [email protected] 6/12 What Can Impact the Performance of a Solver?
The matrix properties of H and S can vary considerably based on: • The physics of the system (HOMO-LUMO gap) • Choice of basis set (size, sparsity, condition number…) Code • Pseudization/all-electron
ELSI Connector Meeting Duke University [email protected] 6/12 What Can Impact the Performance of a Solver?
The matrix properties of H and S can vary considerably based on: • The physics of the system (HOMO-LUMO gap) • Choice of basis set (size, sparsity, condition number…) Code • Pseudization/all-electron
Solver performance may be affected by: • Computing architecture (processors, network topology, accelerators) • Choice of compiler • Optimization flags for the compiler • Math libraries • MPI libraries • Network congestion
ELSI Connector Meeting Duke University [email protected] 6/12 What Can Impact the Performance of a Solver?
The matrix properties of H and S can vary considerably based on: • The physics of the system (HOMO-LUMO gap) • Choice of basis set (size, sparsity, condition number…) Code • Pseudization/all-electron
Solver performance may be affected by: • Computing architecture (processors, network topology, accelerators) • Choice of compiler Architecture • Optimization flags for the compiler • Math libraries • MPI libraries • Network congestion
ELSI Connector Meeting Duke University [email protected] 6/12 What Can Impact the Performance of a Solver?
The matrix properties of H and S can vary considerably based on: • The physics of the system (HOMO-LUMO gap) • Choice of basis set (size, sparsity, condition number…) Code • Pseudization/all-electron The core question is: What is the performance of a solver with code X running on architecture Y? Solver performance may be affected by: • Computing architecture (processors, network topology, accelerators) • Choice of compiler Architecture • Optimization flags for the compiler • Math libraries • MPI libraries • Network congestion
ELSI Connector Meeting Duke University [email protected] 6/12 Proposed ELSI Decision Layer
Proposal: Create a profile containing performance data
Profile for Code + Architecture
The profile contains timings for a given code and architecture for common materials encountered in KS-DFT
ELSI Connector Meeting Duke University [email protected] 7/12 Proposed ELSI Decision Layer
Proposal: Create a profile containing performance data KS-DFT Codes
Profile for Code + Architecture H S Decision Layer φ,ε DM The profile contains timings for a given code and architecture H S Solver for common materials Matrix encountered in KS-DFT ELSI Conversion
The ELSI decision layer uses H’ S’ φ’,ε DM’ the profile to estimate the best solver for the provided H and S Solvers
ELSI Connector Meeting Duke University [email protected] 7/12 What Should a Benchmark Set Contain?
To properly calibrate the profile for generic solvers and KS-DFT codes, we will need: • Periodic and non-periodic systems • Materials with different dimensionality (chain molecules, surfaces/slabs, bulk systems) • Gapped and metallic systems • Various chemical species
To be scientifically relevant and aesthetically pleasing, test set of materials should be both real(-sh) and interesting
Test set should have no bias towards a particular code or method
ELSI Connector Meeting Duke University [email protected] 8/12 A Performance Meta-Benchmark Set
“The ELSI benchmark set”
ELSI Connector Meeting Duke University [email protected] 9/12 A Performance Meta-Benchmark Set
“The ELSI benchmark set”
ELSI Connector Meeting Duke University [email protected] 9/12 A Performance Meta-Benchmark Set
HIC SVNT DRACONES “The ELSI benchmark set”
ELSI Connector Meeting Duke University [email protected] 9/12 A Performance Meta-Benchmark Set
HIC SVNT DRACONES “The ELSI benchmark set”
ELSI Connector Meeting Duke University [email protected] 9/12 A Performance Meta-Benchmark Set
CheSS libOMM PEXSI SIPs DGDFT
ELSI Connector Meeting Duke University [email protected] 9/12 A Performance Meta-Benchmark Set
We are creating a “meta-benchmark set” of materials used in performance/scaling benchmark for 20 different software packages:
CheSS libOMM PEXSI SIPs DGDFT
Five classes of systems emerge
ELSI Connector Meeting Duke University [email protected] 9/12 Large Molecules (1/5)
Buckminsterfullerene Cyanocobalamin
- DNA [BigDFT, CASTEP, PEXSI] - Polyacetylene [Yambo, molgw] - Polyalanine [FHI-aims, ELPA] - Buckyball [molgw] - Chlorophyll [BigDFT] - Crambin [CASTEP] - Vitamin B12 (cyanocobalamin) [BigDFT] - Bulk-Like Pentacene [CheSS]
ELSI Connector Meeting Duke University [email protected] 10/12 Bulk Systems (2/5)
Diamond Si, 10x10x10 supercell Well-Ordered Cubic MAPI, 8x8x8 supercell - Si [WEST!, SIESTA, Exciting, libOMM] - GaAs [FHI-aims] - Diamond [Exciting, SIPs] - LiH [CP2k] - MAPI, Well-Ordered Cubic [BigDFT, CheSS] - Si with 1.6% H impurity [PEXSI] - Al [PEXSI] - SiC [WEST!] - AlAs [WEST!] - SiO2 [Yambo] - BN, Cubic [Exciting] - ZnO [Berkeley GW] - Fe2O3, Hematite [NWChem] - Fe2SiO4, Fayalite [CP2k]
ELSI Connector Meeting Duke University [email protected] 10/12 2D Systems (3/5)
Sapphire (Al2O3) Surface, 6x6x1 supercell Bi2Se3 Slab, 10x10x1 supercell of 270-atom surface - Graphene [BigDFT, ELSI, libOMM] - Al2O3 surface [CASTEP] - Au(100) surface [FHI-aims] - BN surface [PEXSI] - Phosphorene monolayer [PEXSI] - Pt(100) reconstructed surface [ELPA] - TiN surface with adsorbed H [CASTEP] - W(110) surface [Exciting]
ELSI Connector Meeting Duke University [email protected] 10/12 Systems Containing Water (4/5)
512 water molecules in periodic cell DNA solvated by water Figure 2c, Lin et al., J. Phys. Condens. Matter. 2014 Figure 1a, Mohr et al.,arXiv:1704.005
- Water [BigDFT, CP2k, NWChem, Quantum ESPRESSO, PEXSI, libOMM, CheSS] - Aβ-peptide in water [Quantum ESPRESSO] - Au surface covered by thiols in interaction with water [Quantum ESPRESSO] - Buckyball solvated by water [BigDFT] - DNA solvated by water [PEXSI] - Si surface functionalized by H in interaction with water [WEST!] - Si surface functionalized by COOH in interaction with water [WEST!] 2+ - (UO2) + 122 H20 [NWChem] - Zn2 + 64 H20 [NWChem]
ELSI Connector Meeting Duke University [email protected] 10/12 Nano{crystals,tubes,wires} (5/5)
Si293H172 Nanocrystal (20,20) SWCNT Figure 12, Govoni and Galli, J. Chem. Theory Comput. 2015 Figure 4, Deslippe et al., Comp. Phys. Comm 2012
- Si nanocrystal (largest, Si293H172) [WEST!, BigDFT, CheSS] - SWCNT, (10,0) and (20,20) [Berkeley GW] - CNT functionalized with porphyrins [Quantum ESPRESSO] - Diamond Nanowire [SIPs] - Single-Wall Metallic Nanotubes [SIPs]
ELSI Connector Meeting Duke University [email protected] 10/12 Proposed Additional Systems (6/5)
CBTS, 6x6x6 supercell Bi2Se3 Slab, 10x10x1 supercell
- Au - Cu2BaSnS4 (CBTS) - SiC slab - TiO2 slab - Bi2Se3 slab
ELSI Connector Meeting Duke University [email protected] 10/12 Format of Geometries
Every material in the meta-benchmark set has a geometry in an XYZ format containing 5 lines of metadata including original source and an easily-scaled cell (when possible):
When applicable, original geometries will also be distributed
The meta-benchmark set is available on the ELSI Gitlab as a project:
https://git.elsi-interchange.org/, accessed 15 August 2017
ELSI Connector Meeting Duke University [email protected] 11/12 Status
The current benchmark set, while coming from 20 different codes, is still quite incomplete: • Molecules heavily biased towards hydrocarbons • Bulk systems biased towards compound semiconductors • Only water is simulated, no other liquids • ALL systems heavily biased towards light elements
We will be performing the first test run of the meta-benchmark set next week on our local cluster at Duke
Suggestions for additional systems (especially molecules) are welcome!
ELSI Connector Meeting Duke University [email protected] 12/12 ELSI Connector Meeting Duke University [email protected]