Basic Concepts in Computational Physics Benjamin A

Basic Concepts in Computational Physics Benjamin A. Stickler • Ewald Schachinger

123 Benjamin A. Stickler Ewald Schachinger Faculty of Physics Institute of Theoretical and Computational University of Duisburg-Essen Physics Duisburg Graz University of Technology Germany Graz Austria

ISBN 978-3-319-02434-9 ISBN 978-3-319-02435-6 (eBook) DOI 10.1007/978-3-319-02435-6 Springer Cham Heidelberg New York Dordrecht London

Library of Congress Control Number: 2013949666

Ó Springer International Publishing Switzerland 2014 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein.

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com) Preface

Traditionally physics was divided into two fields of activities: theoretical and experimental. As a consequence of the stunning increase in computer power and of the development of more powerful numerical techniques, a new branch of physics was established over the last decades: Computational Physics. This new branch introduced as a spin-off what nowadays is commonly called computer simulations, which play an increasingly important role in physics and in related sciences as well as in industrial applications. They serve two purposes • Direct simulation of physical processes (theoretical-experimental-physics) such as for instance – molecular dynamics or – Monte Carlo simulation of physical processes. • Solution of complex mathematical problems such as – differential equations, – minimization problems as well as – high dimensional integrals or sums. This book addresses all these scenarios on a very basic level. It is addressed to lecturers who will have to teach a basic course/basic courses in Computational Physics or Numerical Methods and to students as a companion in their first steps into the realm of this fascinating field of modern research. Following these intentions, this book was divided into two parts. Part I deals with deterministic methods in Computational Physics. We discuss, in particular, numerical differentiation and integration, the treatment of ordinary differential equations, and we present some notes on the numerics of partial differential equations. Each chapter within this part of the book is complemented by numerous applications. Part II of this book provides an introduction to stochastic methods in Computational Physics. In particular, we will examine how to generate random numbers following a given distribution, summarize the basics of stochastics in order to establish the necessary background to understand techniques like MARKOV-Chain Monte Carlo. Finally, algorithms of stochastic optimization are discussed. Again, numerous examples out of physics like diffusion processes or the POTTS model are

v vi Preface investigated exhaustively. Finally, this book contains an Appendix that augments the main parts of the book with a detailed discussion of supplementary topics. This book is not meant to be just a collection of algorithms which can immediately be applied to various problems which may arise in Computational Physics. On the contrary, the scope of this book is to provide the reader with a mathematically well founded glance behind the scene of Computational Physics. Thus, particular emphasis is on a clear mathematical analysis of the various topics and to even provide, in some cases, the necessary mathematical means to understand the very background of these methods. Although there is a barely compre- hensible amount of excellent literature on Computational Physics, most of these books seem to concentrate either on deterministic methods or on stochastic methods. It is not our goal to compete with these rather speciﬁc works. On the contrary, it is the particular focus of this book to discuss deterministic methods on a par with stochastic methods. Nevertheless, a certain overlap with existing literature was unavoidable and we apologize if we were not able to cite appropriately all existing works which are of importance and which inﬂuenced this book. However, we believe that by putting the emphasis on an exact mathematical analysis of both the deterministic as well as the stochastic methods we created a stimulating presentation of the basic concepts applied in Computational Physics. If we assume two basic courses in Computational Physics to be part of the curriculum, nick-named here: Computational Physics 101 and Computational Physics 102, then we would like to suggest to present/study the various topics of this book according to the following syllabus: • Computational Physics 101: – Chapter 1: Some Basic Remarks. – Chapter 2: Numerical Differentiation. – Chapter 3: Numerical Integration. – Chapter 4:TheKEPLER Problem. – Chapter 5: Ordinary Differential Equations: Initial Value Problems. –(Chapter 6: The Double Pendulum). – Chapter 7: Molecular Dynamics. – Chapter 8: Numerics of Ordinary Differential Equations: Boundary Value Problems. –(Chapter 9: The One-Dimensional Stationary Heat Equation). – Chapter 10: The One-Dimensional Stationary SCHRÖDINGER Equation. – Chapter 12: Pseudo Random Number Generators. • Computational Physics 102: – Chapter 11: Partial Differential Equations. – Chapter 13: Random Sampling Methods. – Chapter 14: A Brief Introduction to Monte-Carlo Methods. – Chapter 15:TheISING Model. Preface vii

– Chapter 16: Some Basics of Stochastic Processes. – Chapter 17: The Random Walk and Diffusion Theory. – Chapter 18:MARKOV-Chain Monte Carlo and the POTTS Model. – Chapter 19: Data Analysis. – Chapter 20: Stochastic Optimization. The various chapters are augmented by problems of medium complexity which help to understand better the numerical part of the topics discussed within this book. Although the manuscript has been carefully checked several times, we cannot exclude that some errors escaped our scrutiny. We apologize in advance and would highly appreciate reports of potential mistakes or typos to Benjamin A. Stickler ([email protected]). Throughout the book SI-units are used except stated otherwise.

Graz, July 2013 Benjamin A. Stickler Ewald Schachinger Acknowledgments

The Authors are grateful to Prof. Dr. C. Gattringer (Karl-Franzens Universität, Graz), to Profs. Dr. W. von der Linden, Dr. H.-G. Evertz, and Dr. H. Sormann (Graz University of Technology, Austria). They inspired this book with their lectures on various topics of Computational Physics, computer simulation, and numerical analysis. Last but not least the Authors thank Dr. Chris Theis (CERN) for meticulously reading the manuscript, for pointing out inconsistencies, and for suggestions to improve the text.

ix Contents

1 Some Basic Remarks ...... 1 1.1 Motivation ...... 1 1.2 Rounding Errors ...... 6 1.3 Methodological Errors...... 8 1.4 Stability...... 9 1.5 Concluding Remarks...... 12 References ...... 13

Part I Deterministic Methods

2 Numerical Differentiation ...... 17 2.1 Introduction ...... 17 2.2 Finite Differences ...... 18 2.3 Finite Difference Derivatives ...... 20 2.4 A Systematic Approach: The Operator Technique ...... 22 2.5 Concluding Discussion ...... 25 Summary ...... 27 Problems...... 28 References ...... 28

3 Numerical Integration ...... 29 3.1 Introduction ...... 29 3.2 Rectangular Rule ...... 30 3.3 Trapezoidal Rule ...... 33 3.4 The SIMPSON Rule ...... 35 3.5 General Formulation: The NEWTON-COTES Rules ...... 36 3.6 GAUSS-LEGENDRE Quadrature...... 38 3.7 An Example...... 44 3.8 Concluding Discussion ...... 45 Summary ...... 47 Problems...... 48 References ...... 49

xi xii Contents

4 The KEPLER Problem ...... 51 4.1 Introduction ...... 51 4.2 The Problem ...... 52 4.3 Numerical Treatment ...... 54 Summary ...... 59 References ...... 59

5 Ordinary Differential Equations: Initial Value Problems ...... 61 5.1 Introduction ...... 61 5.2 Simple Integrators ...... 62 5.3 RUNGE-KUTTA Methods ...... 66 5.4 Hamiltonian Systems: Symplectic Integrators...... 71 5.5 An Example: The KEPLER Problem, Revisited ...... 73 Summary ...... 78 Problems...... 79 References ...... 79

6 The Double Pendulum ...... 81 6.1 HAMILTON’s Equations ...... 81 6.2 Numerical Solution...... 85 6.3 Numerical Analysis of Chaos...... 88 Summary ...... 95 Problems...... 95 References ...... 96

7 Molecular Dynamics...... 97 7.1 Introduction ...... 97 7.2 Classical Molecular Dynamics ...... 97 7.3 Numerical Implementation...... 102 Summary ...... 107 Problems...... 108 References ...... 109

8 Numerics of Ordinary Differential Equations: Boundary Value Problems ...... 111 8.1 Introduction ...... 111 8.2 Finite Difference Approach ...... 113 8.3 Shooting Methods...... 118 Summary ...... 121 References ...... 122

9 The One-Dimensional Stationary Heat Equation ...... 123 9.1 Introduction ...... 123 9.2 Finite Differences ...... 124 Contents xiii

9.3 A Second Scenario ...... 126 Summary ...... 128 Problems...... 129 Reference ...... 129

10 The One-Dimensional Stationary SCHRÖDINGER Equation ...... 131 10.1 Introduction ...... 131 10.2 A Simple Example: The Particle in a Box ...... 134 10.3 Numerical Solution...... 139 10.4 Another Case ...... 142 Summary ...... 145 Problems...... 146 References ...... 146

11 Partial Differential Equations ...... 147 11.1 Introduction ...... 147 11.2 The POISSON Equation ...... 148 11.3 The Time-Dependent Heat Equation ...... 151 11.4 The Wave Equation ...... 157 11.5 The Time-Dependent SCHRÖDINGER Equation ...... 160 Summary ...... 167 Problems...... 167 References ...... 168

Part II Stochastic Methods

12 Pseudo Random Number Generators ...... 171 12.1 Introduction ...... 171 12.2 Different Approaches ...... 174 12.3 Quality Tests ...... 178 Summary ...... 182 Problems...... 182 References ...... 183

13 Random Sampling Methods ...... 185 13.1 Introduction ...... 185 13.2 Inverse Transformation Method ...... 187 13.3 Rejection Method ...... 190 13.4 Probability Mixing ...... 193 Summary ...... 195 Problems...... 195 xiv Contents

14 A Brief Introduction to Monte-Carlo Methods...... 197 14.1 Introduction ...... 197 14.2 Monte-Carlo Integration ...... 198 14.3 The METROPOLIS Algorithm: An Introduction ...... 205 Summary ...... 208 References ...... 208

15 The ISING Model ...... 209 15.1 The Model...... 209 15.2 Numerics...... 218 15.3 Selected Results ...... 223 Summary ...... 227 Problems...... 228 References ...... 228

16 Some Basics of Stochastic Processes ...... 229 16.1 Introduction ...... 229 16.2 Stochastic Processes ...... 230 16.3 MARKOV Processes ...... 233 16.4 MARKOV-Chains ...... 241 16.5 Continuous-Time MARKOV-Chains...... 247 Summary ...... 249 Problems...... 250 References ...... 250

17 The Random Walk and Diffusion Theory ...... 251 17.1 Introduction ...... 251 17.2 The Random Walk ...... 253 17.3 The WIENER Process and Brownian Motion ...... 259 17.4 Generalized Diffusion Models ...... 265 Summary ...... 272 Problems...... 272 References ...... 273

18 MARKOV-Chain Monte Carlo and the POTTS Model...... 275 18.1 Introduction ...... 275 18.2 MARKOV-Chain Monte Carlo Methods ...... 276 18.3 The POTTS Model ...... 279 18.4 Advanced Algorithms for the POTTS Model ...... 284 Summary ...... 285 Problems...... 286 References ...... 286 Contents xv

19 Data Analysis ...... 287 19.1 Introduction ...... 287 19.2 Calculation of Errors...... 287 19.3 Auto-Correlations ...... 291 19.4 The Histogram Technique ...... 295 Summary ...... 296 Problems...... 297 Reference ...... 297

20 Stochastic Optimization ...... 299 20.1 Introduction ...... 299 20.2 Hill Climbing...... 301 20.3 Simulated Annealing...... 303 20.4 Genetic Algorithms...... 310 20.5 Some Further Methods ...... 312 Summary ...... 313 Problems...... 314 References ...... 314

Appendix A: The Two-Body Problem ...... 315

Appendix B: Solving Non-Linear Equations: The NEWTON Method . . . 321

Appendix C: Numerical Solution of Linear Systems of Equations .... 323

Appendix D: Basics of Probability Theory ...... 331

Appendix E: Phase Transitions ...... 345

Appendix F: Fractional Integrals and Derivatives in 1D ...... 349

Appendix G: Least Squares Fit ...... 351

Appendix H: Deterministic Optimization ...... 357

Index ...... 369 Acronyms

cdf Cumulative distribution function CFL COURANT-FRIEDRICHS-LEWY condition e-RK-4 Explicit RUNGE-KUTTA four stage method n. n. Nearest neighbors pdf Probability density function

xvii