The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
Lazy matrices for contraction-based algorithms
Michael F. Herbst [email protected] https://michael-herbst.com
Interdisziplinäres Zentrum für wissenschaftliches Rechnen Ruprecht-Karls-Universität Heidelberg
4th October 2017
1 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
Contents
1 The storage problem Problems with conventional approaches contraction-based algorithms
2 Lazy matrices The lazyten lazy matrix library
3 Lazy matrices in quantum chemistry molsturm program package
4 Future work Outlook
1 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
Contents
1 The storage problem Problems with conventional approaches contraction-based algorithms
2 Lazy matrices The lazyten lazy matrix library
3 Lazy matrices in quantum chemistry molsturm program package
4 Future work Outlook
2 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
Problems with conventional approaches Processor vs. memory performance improvement
100000 CPU clock speed Memory bus speed 10000
1000 Processor vs. Memory 100 performance gap
10
1 1980 1985 1990 1995 2000 2005 2010
Data from https://dave.cheney.net/2014/06/07/five-things-that-make-go-fast 3 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
Problems with conventional approaches Hartree-Fock equations
Hartree-Fock equations 1 − ∆+ Vˆ + Vˆ {ψ } + Vˆ {ψ } − ε ψ (r) = 0 2 Nuc H f f∈I x f f∈I f f with 1 − ∆ Kinetic energy of electrons 2 VˆNuc Electron-nuclear interaction ˆ VH {ψf }f∈I Hartree potential ˆ Vx {ψf }f∈I Exchange potential
Non-linear system of partial differential equations
Non-linear eigenproblem for eigenpairs {(εf , ψf )}f∈I 4 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
Problems with conventional approaches Finite-element discretisation
Finite elements: Piecewise polynomials with support only on a few neighbouring cells ⇒ Need many finite elements (> 106) Typically sparse matrix structures:
0 1000 2000 3000 4000 5000 6000 7000 0
1000
0 2000 2 − 3000 4 − 6 4000 − 8 − 5000 10 − 12 6000 −
7000 1 ˆ ˆ Typical discretisation of − 2 ∆+ VNuc + VH 5 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
Problems with conventional approaches Finite-element discretisation Caveat
But . . . Vˆx is so-called non-local:
0 1000 2000 3000 4000 5000 6000 7000 0
1000
0 2000 2 − 3000 4 − 6 4000 − 8 − 5000 10 − 12 6000 −
7000
K: Discretisation of Vˆx
Building K takes Ω(N 2) time and O(N 2) storage Typically 106 · 106 elements ≈ 8 TB storage 6 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q contraction-based algorithms Finite-element discretisation contraction-based scheme
Iterative solvers only need matrix-vector products Matrix-vector product of K: Theoretically O(N)
⇒ contraction-based or matrix-free algorithm: Never build K in storage Use expression for K to directly form contraction of matrix to vectors
7 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q contraction-based algorithms contraction-based SCF Possible problems
For SCF we will need
F = T + VNuc + J + K + ···
Requirements differ: Optimal storage scheme Optimal contraction scheme Approximations / costs ⇒ contraction expression complicated to code ⇒ We would like to stay flexible
8 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q contraction-based algorithms Characteristics of contraction-based algorithms
Advantages Scaling (storage and time) reduced — in examples to O(N) Parallelisation easier ⇒ Less data management Hardware trends are in favour
Disadvantages Matrices more intuitive than contraction-functions More computations ⇒ Need efficient contraction schemes for the contraction ⇒ Algorithms more complicated
9 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q contraction-based algorithms Characteristics of contraction-based algorithms
Advantages Scaling (storage and time) reduced — in examples to O(N) Parallelisation easier ⇒ Less data management Hardware trends are in favour
Disadvantages Matrices more intuitive than contraction-functions More computations ⇒ Need efficient contraction schemes for the contraction ⇒ Algorithms more complicated
9 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q contraction-based algorithms Characteristics of contraction-based algorithms
100000 CPU clock speed Memory bus speed 10000
1000 Processor vs. Memory 100 performance gap
10
1 1980 1985 1990 1995 2000 2005 2010
Data from https://dave.cheney.net/2014/06/07/five-things-that-make-go-fast 9 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q contraction-based algorithms Characteristics of contraction-based algorithms
Advantages Scaling (storage and time) reduced — in examples to O(N) Parallelisation easier ⇒ Less data management Hardware trends are in favour
Disadvantages Matrices more intuitive than contraction-functions More computations ⇒ Need efficient contraction schemes for the contraction ⇒ Algorithms more complicated
9 / 26
The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
Contents
1 The storage problem Problems with conventional approaches contraction-based algorithms
2 Lazy matrices The lazyten lazy matrix library
3 Lazy matrices in quantum chemistry molsturm program package
4 Future work Outlook
10 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
The lazyten lazy matrix library Lazy matrices
Stored matrix: All elements reside in memory Lazy matrix: Generalisation of matrices State Non-linear Elements may be expressions ⇒ Obtaining elements expensive Evaluation of internal expression: Delayed until contraction For convenience: Offer matrix-like interface
11 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
The lazyten lazy matrix library Using lazy matrices
Program as usual D = A + B
Build expression tree internally
D = A + B
On application:
Dx = (Ax) + (Bx)
12 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
The lazyten lazy matrix library Using lazy matrices
Program as usual D = A + B
Build expression tree internally
+ D = A + B = A B
On application:
Dx = (Ax) + (Bx)
12 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
The lazyten lazy matrix library Using lazy matrices
Program as usual D = A + B
Build expression tree internally
+ D = A + B = A B
On application:
Dx = (Ax) + (Bx)
12 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
The lazyten lazy matrix library Notes and observations
lazyten: Bookkeeping for contraction-functions Programmer still sees matrices ⇒ Language for writing contraction-based algorithms
Lazy matrices allow layered responsibility for computation, e.g. (A + B)x Ax and Bx decided by implementation of A and B (Ax) + (Bx) done in linear algebra backend ⇒ Proper modularisation between Higher-level algorithms Lazy matrix implementations LA backends
13 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
The lazyten lazy matrix library lazyten: Lazy matrix library Structure of the library
Algorithms e.g. Integrals e.g. SCF Upper layers
common user-provided interface lazy matrices
lazyten built-in lazy LA backends solvers matrices
Armadillo ARPACK Iterative inverse Bohrium LAPACK
LA backend Eigen Anasazi Italic: planned 14 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
The lazyten lazy matrix library lazyten in practice: Example code
lazyten1: Prototype C++ implementation
1 typedef SmallVector
1https://lazyten.org 15 / 26
The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
Contents
1 The storage problem Problems with conventional approaches contraction-based algorithms
2 Lazy matrices The lazyten lazy matrix library
3 Lazy matrices in quantum chemistry molsturm program package
4 Future work Outlook
16 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q molsturm program package Lazy matrices for quantum chemistry codes
contraction-based algorithms ⇒ Lower memory footprint, scaling improvements
Readable code ⇒ Great for teaching or to play around
Abstraction between integrals and SCF algorithms ⇒ Plug and play integral libraries ⇒ Swap LA backends ⇒ Basis-type independent SCF / quantum chemistry
17 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q molsturm program package molsturm structure
Integral backends Post HF methods
MP2 Gaussians molsturm ADC Interface layer and python driver Sturmians Full CI gint gscf Ionis. sturmians Integral interface SCF algorithms Coupled Cluster Finite elements lazyten Finite differences Linear algebra interface
Wavelets krims Common utilities ...
Italic: planned 18 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q molsturm program package molsturm design
Enables contraction-based SCF routines Flexiblity as primary goal Behaviour controlled via python Keywords to change basis type or solver All computed data available in numpy format No input file, just a python script python utilities Import / export results Post-HF calculations
19 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q molsturm program package molsturm interface: CCD residual (parts)
1 X 1 X 1 X rab = − hmn||efi taf teb + hmn||efi tbf tea − hmn||efi tef tab ij 2 mn ij 2 mn ij 2 in mj mnef mnef mnef 1 X 1 X 1 X + hmn||efi tef tab + hmn||efi tab tef + hmn||efi tae tbf 2 jn mi 4 mn ij 2 im jn mnef mnef mnef 1 X 1 X 1 X − hmn||efi tae tbf − hmn||efi tbe taf + hmn||efi tbe taf 2 jm in 2 im jn 2 jm in mnef mnef mnef
eri_phys = state.eri.transpose((0, 2, 1, 3)) eri = eri_phys - eri_phys.transpose((1, 0, 2, 3)) res=\ - 0.5 * np.einsum("mnef,manf,iejb->iajb", eri.block("oovv"), t2, t2)\ + 0.5 * np.einsum("mnef,mbnf,ieja->iajb", eri.block("oovv"), t2, t2)\ - 0.5 * np.einsum("mnef,ienf,majb->iajb", eri.block("oovv"), t2, t2)\ + 0.5 * np.einsum("mnef,jenf,maib->iajb", eri.block("oovv"), t2, t2)\ + 0.25 * np.einsum("mnef,manb,iejf->iajb", eri.block("oovv"), t2, t2)\ + 0.5 * np.einsum("mnef,iame,jbnf->iajb", eri.block("oovv"), t2, t2)\ - 0.5 * np.einsum("mnef,jame,ibnf->iajb", eri.block("oovv"), t2, t2)\ - 0.5 * np.einsum("mnef,ibme,janf->iajb", eri.block("oovv"), t2, t2)\ + 0.5 * np.einsum("mnef,jbme,ianf->iajb", eri.block("oovv"), t2, t2) 20 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
Contents
1 The storage problem Problems with conventional approaches contraction-based algorithms
2 Lazy matrices The lazyten lazy matrix library
3 Lazy matrices in quantum chemistry molsturm program package
4 Future work Outlook
21 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
Outlook Lazy matrix expression optimisation
+ C + A + B + A B
+ = + + C + A B A B
Matrix expression tree ≡ abstract syntax tree ≡ DAG ⇒ May be optimised by standard methods
22 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
Outlook Lazy matrix expression optimisation
+ C + A + B + A B
+ + = + + = C · C + A B 2 + A B A B
Matrix expression tree ≡ abstract syntax tree ≡ DAG ⇒ May be optimised by standard methods
22 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
Outlook Selection of LA backend
Right now: LA backend is compiled in e.g. Bohrium backend Uses just-in-time (JIT) compilation Very specific for hardware Compilation takes time Better: Dynamic selection Load on the expression tree Availablility of backends Hardware specs
23 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
Outlook Extension to lazy tensors
contraction is a special tensor contraction ⇒ Lazy tensors: Delay all tensor contractions as long as possible
˜ X µ ν e.g. kbf = Cao cabIµν ccd CcoCdf acdoµν
Compare possible contraction schemes by complexity Execute cheapest evaluation scheme May incorporate and exploit symmetry ⇒ Determine optimal contraction sequence automatically
24 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
Acknowledgements
Dr. James Avery Prof. Andreas Dreuw and the Dreuw group
Prof. Guido Kanschat
HGS Mathcomp
25 / 26 The storage problem Lazy matrices Lazy matrices in quantum chemistry Future work A&Q
Questions?
EMail: [email protected] Website/blog: https://michael-herbst.com Projects: https://lazyten.org and https://molsturm.org
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International Licence.
26 / 26