Molecular Force Fields with Gradient-Domain Machine Learning (GDML): Comparison and Synergies with Classical Force Fields Huziel E. Sauceda,1, 2, 3, ∗ Michael Gastegger,2, 4, 3 Stefan Chmiela,2 Klaus-Robert M¨uller,2, 5, 6, 7, y and Alexandre Tkatchenko1, z 1Department of Physics and Materials Science, University of Luxembourg, L-1511 Luxembourg, Luxembourg 2Machine Learning Group, Technische Universit¨atBerlin, 10587 Berlin, Germany 3BASLEARN, BASF-TU joint Lab, Technische Universit¨atBerlin, 10587 Berlin, Germany 4DFG Cluster of Excellence \Unifying Systems in Catalysis" (UniSysCat), Technische Universit¨atBerlin, 10623 Berlin, Germany 5Department of Brain and Cognitive Engineering, Korea University, Anam-dong, Seongbuk-gu, Seoul 136-713, Korea 6Max Planck Institute for Informatics, Stuhlsatzenhausweg, 66123 Saarbr¨ucken,Germany 7Google Research, Brain team, Berlin, Germany (Dated: August 11, 2020) Modern machine learning force fields (ML-FF) are able to yield energy and force predictions at the accuracy of high-level ab initio methods, but at a much lower computational cost. On the other hand, classical molecular mechanics force fields (MM-FF) employ fixed functional forms and tend to be less accurate, but considerably faster and transferable between molecules of the same class. In this work, we investigate how both approaches can complement each other. We contrast the ability of ML-FF for reconstructing dynamic and thermodynamic observables to MM-FFs in order to gain a qualitative understanding of the differences between the two approaches. This analysis enables us to modify the generalized AMBER force field (GAFF) by reparametrizing short-range and bonded interactions with more expressive terms to make them more accurate, without sacrificing the key properties that make MM-FFs so successful. I. INTRODUCTION Recently, modern machine learning methods (ML) have emerged as an elegant solution to this dilemma [11{ Computational studies of materials and molecular sys- 13]. Due to the high functional flexibility of ML meth- tems at the atomic level of detail constitute a cen- ods they can be used to construct force fields (FFs) with tral tool in contemporary physics, biology, materials the accuracy of electronic-structure calculations, while science, and chemistry research. Standard electronic- retaining high computational efficiency albeit still be- structure theories (e.g. density-functional theory (DFT) hind conventional classical force fields. A plethora of or wavefunction-based methods) are ultimately limited in ML based FFs has been reported within the past few the size of systems and time scales they can treat, due to years, demonstrating the general utility of this approach. the high computational cost associated with modeling the In particular, a great amount of work has been done in electronic interactions. Conventional molecular mechan- FFs based on neural networks (NN) [14{24] and kernel- ics force fields (MM-FFs) overcome these limitations by based models [9, 25{33]. In order to understand bet- introducing a range of computationally efficient approx- ter how these models work and to improve their perfor- imations, circumventing the need to solve the electronic mance, other ML studies have been done on molecular problem explicitly [1{8]. This has made possible to simu- representations [34{48], data sampling [49{52], explana- late biologically relevant systems (such as proteins), gain- tion methods [21, 53], and inference of molecular prop- ing insights into their innermost workings at an atomistic erties [25, 27, 31, 32, 54{63], as well as software develop- level, culminating a Nobel prize in chemistry awarded to ment [64{66]. Yet, although it is generally assumed that Karplus, Levitt and Warshel. However, the high compu- ML-FFs have much better accuracy than MM-FFs, no tational efficiency of MM-FFs comes at the cost of ac- in-depth comparisons (e.g. potential-energy surfaces and curacy compared to electronic-structure methods. More- molecular dynamics predictions) have been performed up over, the basic form of the physical approximations em- to date. Furthermore, such comparison raises the ques- arXiv:2008.04198v1 [physics.chem-ph] 10 Aug 2020 ployed restricts their applicability to certain classes of tion of how the ML-FFs and MM-FFs could be combined problems, e.g. most of them are unable to model chemical to complement each other. reactions involving bond breaking/formation or do not The goal of the present work is to perform a detailed include crucial intramolecular interactions [9] or many- investigation of the differences between both approaches body effects [10]. based on a set of small molecules exhibiting different quantum-mechanical phenomena. Based on these re- sults, different alternatives are explored for improving ∗ [email protected] the data generation process and their applicability for y [email protected] expediting the force-field learning procedure. Further- z [email protected] more, improvement of the accuracy for MM-FFs is stud- 2 ied by reparameterising them based on more accurate mentioned physical properties constitute a set of mathe- reference data and test their limits and functional form matical constraints that narrows down the space of solu- flexibility. For this task, we use the sGDML frame- tion of the general ML approximator, thereby increasing work [31, 32] as ML-FF of choice, as it is able to effi- the data efficiency and accuracy in the reconstruction of ciently reconstruct the potential-energy surfaces (PES) the original data generator. However, all these consid- of medium sized molecules. The investigated systems are erations, and the desire for a global model in particular, the molecules ethanol, the keto form of malondialdehyde place limitations to the computational performance of (keto-MDA) as well as acetylsalicylic acid (Aspirin). In such an approach. the context of these systems, we study the performance A completely different strategy is pursued by molecular of MM-FFs and sGDML derived FFs based on the overall mechanical force fields, whose design follows chemical in- reliability of the generated PESs, as well as effects arising tuition and known physical approximations to intra- and from chemical phenomena such as interactions between intermolecular interactions. In this case, the globality of molecular orbitals. Although we restrict ourselves to the the model is replaced by a more scalable partitioning of sGDML approach, it can nevertheless be expected that the interactions in terms of interatomic pair distances, the results found here are equally valid for ML-FFs in angles and dihedral angles in local domains. The 3-body general. (angles) and 4-body (dihedral angles) terms are the ones responsible for the permutational invariance violation in MM-FFs, a problem that becomes less and less relevant II. THEORY as the size of the system increases from 100s to 1000s or even millions of atoms. On the other hand, these force Learning theory provides a framework to create a di- fields are energy conserving by construction, since the verse collection of universal approximators capable of molecular forces are derived as the derivative of a poten- to reconstruct any function. In practice, reconstructing tial in the form of the interaction energy. This strategy PESs from highly accurate reference calculations such as has been proven to be highly successful over the years in coupled cluster theory (CC) is a challenging task given many domains of molecular modeling [2]. that the amount of available data is limited by computa- Before comparing these two contrasting methodologies, tional resources that such level of theory requires. Conse- we briefly review both frameworks, which will be impor- quently, mathematically constrain the solution space of a tant for the interpretation and further improvement of ML approximator by enforcing universal physical laws it both approaches. is highly advantageous. In this manner, a data-efficient model capable of delivering physically meaningful pre- dictions can be obtained. Some of the basic desirable A. sGDML model properties of high accuracy ML-FFs, from the view of physics and chemistry, are: The requirements mentioned above served as a basis A global description of the system. The nature of for the symmetric Gradient Domain M achine Learning the quantum interactions resulting from the Schr¨odinger (sGDML) FF [32, 67]. This framework consisting in the equation (within the Born-Oppenheimer approximation) use of a permutationaly-symmetrized Hessian of the co- variance function, which then satisfy energy conserva- HΨ = VBOΨ and from the evaluation of the Hellmann- Feynman forces −F = hΨ∗j@H=@xjΨi is inherently tion and atomic indistinguishability by construction. The many-body. In practice, such global description of the global nature of the interactions is retained by using a system has to be enforced by avoiding the non-unique global molecular descriptor. In practice, the sGDML model consisting in the gra- partitioning of the total energy VBO in atomic contribu- tions. dient of a kernel ridge regression (KRR) estimator is Time invariance of the Hamiltonian. This constraint trained only on forces F, and the use of the Hessian ^ matrix of the kernel κ as a covariance structure, natu- requires a time-invariant Hamiltonian H = T +fE, hence ^ energy conservation must be enforced in the ML model, rally gives an analytically integrable force estimator fF, ^ R ^ ^ fE = fF ·dx+c [31, 32]. Then, the training of the model ^fF = −∇fE, yielding conservative forces. Permutational invariance of identical atoms. In quan- consists in solving the normal equation of the KRR esti- tum mechanics, the square of the wave function is in- mator in the gradient domain for the parameter-vectors variant to permuting two identical atoms in a molecule, ~α [31, 32]: a property that is carried over to the energy of the sys- KHess(κ) + λI ~α = rVBO = −F, (1) tem: VBO(:::; ~xi;:::; ~xj;:::) = VBO(:::; ~xj;:::; ~xi;:::). Fulfilling this spatial symmetry by the ML models often here KHess(κ) is the kernel matrix and I is the identity represents a big challenge, as well as in MM-FFs since matrix weighted by the regularization parameter λ.