Louisiana State University LSU Digital Commons LSU Doctoral Dissertations Graduate School 7-2-2020 Identifying Structure Transitions Using Machine Learning Methods Nicholas Walker Follow this and additional works at: https://digitalcommons.lsu.edu/gradschool_dissertations Part of the Condensed Matter Physics Commons, and the Data Science Commons Recommended Citation Walker, Nicholas, "Identifying Structure Transitions Using Machine Learning Methods" (2020). LSU Doctoral Dissertations. 5311. https://digitalcommons.lsu.edu/gradschool_dissertations/5311 This Dissertation is brought to you for free and open access by the Graduate School at LSU Digital Commons. It has been accepted for inclusion in LSU Doctoral Dissertations by an authorized graduate school editor of LSU Digital Commons. For more information, please [email protected]. IDENTIFYING STRUCTURE TRANSITIONS USING MACHINE LEARNING METHODS A Dissertation Submitted to the Graduate Faculty of the Louisiana State University and College of Science in partial fulfillment of the requirements for the degree of Doctor of Philosophy in The Department of Physics & Astronomy by Nicholas Walker B.Sc., Saint Louis University, 2015 August 2020 Acknowledgments I would like to thank the guidance and knowledge imparted to me by my late graduate advisor, Mark Jarrell, as well as Ka-Ming Tam, with whom I have worked very closely for these past years. Furthermore, I would like to thank my parents, Josh and Lindsey Walker for their support during my time as a doctoral candidate. ii Table of Contents ACKNOWLEDGMENTS......................................................... ii LIST OF TABLES...............................................................v LIST OF FIGURES.............................................................. vi ABSTRACT...................................................................... viii CHAPTER 1 INTRODUCTION.........................................................1 2 STATISTICAL MECHANICS OF THE ISING MODEL....................8 2.1 Introduction..........................................................8 2.2 Statistical Mechanics.................................................9 2.3 Weiss Mean Field Theory............................................. 12 2.4 Landau Theory....................................................... 27 2.5 Landau-Ginzburg Theory............................................. 37 2.6 The Renormalization Group.......................................... 61 2.7 The Ising Model in Two Dimensions.................................. 82 3 THE MELTING TRANSITION............................................ 115 3.1 Introduction.......................................................... 115 3.2 The Lindemann Parameter........................................... 116 3.3 The Radial Distribution Function..................................... 117 3.4 The Lennard-Jones Potential.......................................... 119 3.5 The Embedded Atom Model.......................................... 121 3.6 Numerical Calculation of the Melting Point........................... 123 4 MACHINE LEARNING WITH TRADITIONAL DATA SCIENCE......... 127 4.1 Introduction.......................................................... 127 4.2 Feature Space Scaling................................................. 128 4.3 Feature Space Reduction.............................................. 135 4.4 Cluster Analysis...................................................... 147 4.5 Linear Regression..................................................... 155 5 MACHINE LEARNING WITH NEURAL NETWORKS.................... 161 5.1 Artificial Neural Networks............................................ 161 5.2 Bias & Variance...................................................... 177 5.3 Types of Artifical Neural Networks.................................... 183 6 PREDICTION OF THE 2-DIMENSIONAL ISING MODEL CROSSOVER. 201 6.1 Introduction.......................................................... 201 6.2 Methods............................................................. 201 6.3 Results............................................................... 204 iii 6.4 Conclusions.......................................................... 208 7 PREDICTION OF THE MELTING POINT OF ALUMINUM.............. 210 7.1 Methods............................................................. 210 7.2 Results............................................................... 211 7.3 Conclusions.......................................................... 215 8 CONCLUSIONS........................................................... 217 REFERENCES.................................................................... 221 VITA............................................................................. 233 iv List of Tables 2.1 A table summarizing Landau-Ginzburg mean field critical exponents........ 60 2.2 A table summarizing critical exponents in the relevant dimensions........... 61 2.3 A table summarizing renormalization group scaling laws.................... 69 2.4 A table summarizing = 1 critical exponents............................... 82 2.5 A table summarizing the conformal transformations......................... 103 2.6 A table summarizing the critical exponents for d = 2........................ 114 3.1 A table summarizing the Lennard-Jones reduced units...................... 120 v List of Figures 2.1 A sketch of the intersections of m and tanh 2dβJ for 2dβJ < 1.............. 15 2.2 A sketch of the intersections of m and tanh 2dβJ for 2dβJ > 1.............. 15 2.3 A diagram depicting possible spin configurations in 2 dimensions............ 16 2.4 A sketch of m(T ) (vanishing field).......................................... 18 2.5 A sketch of m(T ) (non-vanishing field)..................................... 18 2.6 A sketch of the Free energy expansion when T > TC and H = 0............. 31 2.7 A sketch of the Free energy expansion when T < TC and H = 0............. 31 2.8 A sketch of the Free energy expansion when T > TC and H < 0............. 34 2.9 A sketch of the Free energy expansion when T > TC and H > 0............. 34 2.10 A sketch of the Free energy expansion when T < TC and H < 0............. 35 2.11 A sketch of the Free energy expansion when T < TC and H > 0............. 35 2.12 A diagram depicting the Ising lattice L and its dual LD..................... 91 2.13 A diagram depicting spin domain-separating polygons in LD................ 91 4.1 A diagram depicting feature scaling on Gaussian distributed data........... 132 4.2 A diagram depicting feature scaling on \blob-like clusters" of data.......... 133 4.3 A diagram depicting feature scaling on concentric circles of data............. 134 4.4 A diagram depicting PCA projections Gaussian distributed data............ 138 4.5 A diagram depicting PCA projections \blob-like clusters" of data........... 139 4.6 A diagram depicting PCA projections of concentric circles of data........... 140 4.7 A diagram depicting t-SNE manifolds of Gaussian distributed data.......... 143 4.8 A diagram depicting t-SNE manifolds of \blob-like clusters" of data......... 145 4.9 A diagram depicting t-SNE manifolds of concentric circles of data........... 146 4.10 A diagram depicting K-means clustering.................................... 148 4.11 A diagram depicting agglomerative clustering............................... 150 4.12 A diagram depicting spectral clustering..................................... 152 4.13 A diagram depicting DBSCAN clustering................................... 154 vi 5.1 A diagram depicting a linear regression as an ANN......................... 163 5.2 A diagram depicting an ANN with three hidden layers and one output...... 163 5.3 A diagram depicting a Boltzmann machine................................. 184 5.4 A diagram depicting a restricted Boltzmann machine....................... 187 5.5 A diagram depicting an AE................................................ 188 5.6 A diagram depicting a VAE................................................ 191 5.7 A diagram depicting the convolution operation.............................. 199 6.1 A diagram depicting the ensemble average encoding ν0(H; T )............... 205 6.2 A diagram depicting the ensemble average magnetization m(H; T ).......... 205 6.3 A diagram depicting the ensemble average encoding τ0(H; T )............... 206 6.4 A diagram depicting the ensemble average energy E(H; T ).................. 206 6.5 A diagram depicting the ensemble average encoding τ1(H; T )............... 207 6.6 A diagram depicting the ensemble specific heat C(H; T ).................... 207 6.7 A diagram depicting the spin prediction error distribution................... 208 6.8 A diagram depicting the spin prediction absolute error distribution.......... 208 6.9 A diagram depicting the latent encoding KLD distribution.................. 208 7.1 A diagram depicting a PCA decomposition of aluminum radial distributions. 212 7.2 A diagram depicting two PCs of aluminum radial distributions.............. 212 7.3 A diagram depicting t-SNE embeddings of aluminum radial distributions.... 213 7.4 A diagram depicting DBSCAN clustered aluminum radial distributions...... 213 7.5 A diagram depicting the temperature distributions of the clusters........... 214 7.6 A diagram depicting the average radial distributions of the clusters.......... 214 7.7 A diagram depicting PS(T ) for the heating and cooling stages separately.... 215 7.8 A diagram depicting PS(T )................................................. 215 vii Abstract Methodologies from data science and machine