)ORULGD6WDWH8QLYHUVLW\/LEUDULHV 2020 Machine Learned Force Fields Cole Nathaniel Sheridan Follow this and additional works at DigiNole: FSU's Digital Repository. For more information, please contact [email protected] ｄｯ｣ｵｓｩｧｮ＠ｅｮｶ･ｬｯｰ･＠ｉｄＺ＠ｄｃｅｅＸＶＹＲＭＸＰＱｃＭＴＴＷＴＭａｄＶＷＭＳｆｂｄＷＶＴＴＰＴｅｃ THE FLORIDA STATE UNIVERSITY COLLEGE OF ARTS & SCIENCES MACHINE LEARNED FORCE FIELDS By COLE SHERIDAN A Thesis submitted to the Department of Scientific Computing in partial fulfillment of the requirements for graduation with Honors in the Major Degree Awarded: Fall, 2020 ｄｯ｣ｵｓｩｧｮ＠ｅｮｶ･ｬｯｰ･＠ｉｄＺ＠ｄｃｅｅＸＶＹＲＭＸＰＱｃＭＴＴＷＴＭａｄＶＷＭＳｆｂｄＷＶＴＴＰＴｅｃ The members of the Defense Committee approve the thesis of Cole Sheridan defended on November 20, 2020. Dr. Chen Huang Thesis Director Dr. Wei Yang Outside Committee Member Dr. Albert DePrince Committee Member Dr. Xiaoqiang Wang Committee Member 1 ｄｯ｣ｵｓｩｧｮ＠ｅｮｶ･ｬｯｰ･＠ｉｄＺ＠ｄｃｅｅＸＶＹＲＭＸＰＱｃＭＴＴＷＴＭａｄＶＷＭＳｆｂｄＷＶＴＴＰＴｅｃ Acknowledgements I would like to express my sincerest thanks to Dr. Chen Huang, for his continuous support and encouragement, and his exceptional leadership throughout the course of this research. Without his assistance, this project may have never been realized. It has been an honor and a pleasure to do research with you throughout my undergraduate career. I would also like to express my gratitude to my defense committee, Dr. Chen Huang, Dr. Xiaoqiang Wang, Dr. Wei Yang, and Dr. Albert De Prince. While the circumstances surrounding the time of this prospectus may have caused difficulties in meeting, the continued support throughout this endeavor is greatly appreciated. Furthermore, a sincere thanks to Florida State University (FSU) and the FSU Department of Scientific Computing for providing the foundational education and resources required to complete this research project. Finally, a big thank you to my family and friends, who helped provide emotional support throughout these trying times. i ｄｯ｣ｵｓｩｧｮ＠ｅｮｶ･ｬｯｰ･＠ｉｄＺ＠ｄｃｅｅＸＶＹＲＭＸＰＱｃＭＴＴＷＴＭａｄＶＷＭＳｆｂｄＷＶＴＴＰＴｅｃ Table of Contents List of Figures ……………………………………..……………………………………………. iii List of Tables ……………………………………..……………………………………………. iv Abstract ………………………………………………………………………………………… v Chapter 1. Introduction to Density Functional Theory 1 1.1 Density Functional Theory ……………………………………………………………. 1 1.2 Gaussian Basis and Force Calculation ……………………………………………….… 2 1.3 Force Calculation …………………..………………………………………...………..… 3 1.4 Introduction to NWChem Program ..…………………………………………………..… 4 Chapter 2. Introduction to Molecular Dynamics 7 2.1 Basics of Molecular Dynamics Simulation ………………………………………….…. 7 2.2 Force Fields ……………………………………………………………….…………. 9 Chapter 3. Introduction to Neural Networks 11 3.1 Background ………………….……………………………………………………….. 11 3.2 ANN Force Fields ……………………………...……………………………………... 14 Chapter 4. Machine-Learned Force Fields for C-X Molecules and Radicals 17 4.1 Neural Networks for Describing Heterogeneous Molecular Systems ...........………… 17 4.2 Training Set …………………………………………………………………………… 18 4.3 Training the Neural Network …………………………………………………………… 22 Chapter 5. Machine-Learning for A Simple Three-Dimensional Molecule 27 5.1 Neural Networks for Describing Three-Dimensional Molecules……………..........….... 27 5.2 Application to Cyanopolyyne ………………………….............……………………...... 28 5.3 Results And Discussion ………………….……………............………………………... 30 Chapter 6 Conclusion 33 References 34 ii ｄｯ｣ｵｓｩｧｮ＠ｅｮｶ･ｬｯｰ･＠ｉｄＺ＠ｄｃｅｅＸＶＹＲＭＸＰＱｃＭＴＴＷＴＭａｄＶＷＭＳｆｂｄＷＶＴＴＰＴｅｃ List of Figures Figure 1.1 NWChem input file …………………………………………………….……………. 4 Figure 2.1 Flowchart of processes for performing molecular dynamics simulations …………. 7 Figure 3.1 Python code for a neural network using matrices ………….…………………..….... 14 Figure 4.1 ANN for diatomic molecule ……………………………………………………….. 17 Figure 4.2 Python code for reproducing NWChem input files ………………………………… 19 Figure 4.3 Job submission script for C2 …….……………………………….………………… 20 Figure 4.4 Graph of DFT energy vs bond length …………………………….………………… 21 Figure 4.5 Graph of partial charge vs bond length …………………………………………… 21 Figure 4.6 Outputs of the original neural network for DFT energies ……………………… 23 Figure 4.7 Outputs of the original neural network for partial charges ……………………… 23 Figure 4.8 Error from different initialization seeds for ANNs for two hidden layers ………..… 24 Figure 4.9 Error from different initialization seeds for ANNs for three hidden layers ……..… 25 Figure 4.10 Errors in calculation from different ANN sizes ………………………………… 26 Figure 5.1 Structure of a cyanopolyyne molecule …………………………….……………… 28 Figure 5.2 Results of training in with three MD points ………………………………………… 31 Figure 5.3 Results of training in with 30 MD points ………………………………………… 31 Figure 5.4 Results of training in with 110 MD points ………………………………………… 32 iii ｄｯ｣ｵｓｩｧｮ＠ｅｮｶ･ｬｯｰ･＠ｉｄＺ＠ｄｃｅｅＸＶＹＲＭＸＰＱｃＭＴＴＷＴＭａｄＶＷＭＳｆｂｄＷＶＴＴＰＴｅｃ List of Tables Figure 5.1 Input of cyanopolyyne molecule at the time step 90 ………..………….…………… 28 iv ｄｯ｣ｵｓｩｧｮ＠ｅｮｶ･ｬｯｰ･＠ｉｄＺ＠ｄｃｅｅＸＶＹＲＭＸＰＱｃＭＴＴＷＴＭａｄＶＷＭＳｆｂｄＷＶＴＴＰＴｅｃ Abstract Studying molecules through their potential energy surfaces using molecular dynamics have greatly advanced our understanding of chemical and physical processes in many exciting systems. To perform these simulations, we need atomic forces. Accurate atomic forces can be calculated using ab initio methods (such as the density functional theory). However, simulations based on ab initio methods are computationally costly. To reduce the cost, force fields that describe the interaction between atoms are often used. The accuracy of the simulations is then determined by the accuracy of force fields. Machine learning has become a promising method to generate force fields with an accuracy close to ab initio methods. Especially, artificial neural networks (ANN) have been shown to be an efficient and adaptable method for generating accurate force fields. One prominent challenge for ANN force fields is to represent different chemical elements in heterogeneous systems. In this thesis, we develop a new ANN force field that is capable of recognizing different chemical elements by using atomic numbers as additional descriptors. Our new method is demonstrated by generating ANN force fields for C-X systems in a one-dimensional space, where X stands for H, O, N, and C atoms. Afterwards, ANN methods are used to calculate the molecular force fiend of a cyanopolyyne molecule in three-dimensional space based on the partial energy of each atom in the molecule. v ｄｯ｣ｵｓｩｧｮ＠ｅｮｶ･ｬｯｰ･＠ｉｄＺ＠ｄｃｅｅＸＶＹＲＭＸＰＱｃＭＴＴＷＴＭａｄＶＷＭＳｆｂｄＷＶＴＴＰＴｅｃ Chapter 1. Introduction to Density Functional Theory 1.1 Density Functional Theory To solve quantum systems, we often adopt the Born-Oppenheimer approximation, meaning that the nuclei and electrons of molecules in the system are treated as separate cases due to the difference in mass between the nuclei and the electrons [1]. The system’s time-independent wave function can then be solved as which gives Above, is the system’s Hamiltonian, is the electronic energy, is the kinetic energy, is the potential energy from the nucleus, and is the electron-electron interaction energy. and are the same for all N-electron systems, while depends on the system. The above equation is an eigenvalue problem with the eigenvalue E and the eigenvector . For systems that have many electrons, the above equation is very difficult to solve due to the electron-electron interaction term . Density Functional Theory (DFT) is a method for calculating the electronic structure in materials and molecules efficiently by condensing the wave function to electron density. DFT is based on the Hohenberg-Kohn (HK) Theorem [2].The HK theorem proves that a non-degenerate quantum system is fully determined by its electron density, given as where is the number of electrons, is the system’s wave function, and is the coordinate of electron i. In other words, HK theorem proves that the external potential, , is determined by the system’s electron density, , and therefore the electron density determines the ground-state wave function and all other electronic properties [3]. On the other hand, the total energy of a quantum system can be obtained by minimizing the total energy with respect to its many-electron wave function Since the external potentialmin and the wave function are determined by the electron density, HK theorem also proved that the total energy is a functional of electron density, as 1 ｄｯ｣ｵｓｩｧｮ＠ｅｮｶ･ｬｯｰ･＠ｉｄＺ＠ｄｃｅｅＸＶＹＲＭＸＰＱｃＭＴＴＷＴＭａｄＶＷＭＳｆｂｄＷＶＴＴＰＴｅｃ where is a universal functional in terms of density and is dependent on the system. Though it is often difficult, if not impossible, to find , resulting in the development of various methods to perform DFT [3]. One such method is the Kohn-Sham (KS) DFT [4], which is considered one of the most popular methods available for simulating quantum systems. Under KS-DFT, the electronic structure of the system is evaluated by the potential acting on the system’s electrons as a sum of the external and effective potentials. This allows a material with N electrons to be represented by N one-electron KS equations [4]. KS equation is a single-electron Schrödinger equation for non- interacting electrons that move in a local effective potential, , (which is also known as the KS potential [4]). KS-DFT then seeks to solve the KS orbitals, , by solving the following ordinary differential equation where is the orbital energy of . The system’s electron density is calculated as where is the occupation number of orbital i. The total energy of the