• Abstract and Figures
• Public Full-text

This thesis has been submitted in fulfilment of the requirements for a postgraduate degree (e.g. PhD, MPhil, DClinPsychol) at the University of Edinburgh. Please note the following terms and conditions of use: This work is protected by copyright and other intellectual property rights, which are retained by the thesis author, unless otherwise stated. A copy can be downloaded for personal non-commercial research or study, without prior permission or charge. This thesis cannot be reproduced or quoted extensively from without first obtaining permission in writing from the author. The content must not be changed in any way or sold commercially in any format or medium without the formal permission of the author. When referring to this work, full bibliographic details including the author, title, awarding institution and date of the thesis must be given. Accelerated Sampling Schemes for High Dimensional Systems Anton Martinsson Doctor of Philosophy University of Edinburgh February 2020 Till Marie, Niklas och Marianna. 4 Lay Summary We construct and analyse accelerated sampling schemes for high dimensional systems. The development of these methods is fundamental for effective computational study of a large class of problems in statistics, statistical physics, chemistry and engineering. Models in these different areas associate a probabilistic likelihood to the values of the variables of the system. For example, a protein molecule may fold into one of several natural shapes which has a lower energy than others. The aim of accelerated schemes is to enhance the exploration of the most likely states that will be found under certain conditions. We focus on the design of methods based on the discretization of stochastic differential equations which result in sequential iterative procedures that generate state sequences when implemented on a computer. Of particular interest are methods that use temperature as a way of enhancing explo- ration of likely states. A contribution of this thesis is the design of numerical methods based on simulated tempering in the “infinite switch limit", in which the temperature is treated as a dynamical variable that fluctuates rapidly during the simulation. This approach can be generalised to allow different characteristics to be varied during sim- ulation; for example we develop a novel approach to constant pressure sampling and couple it with this idea. We also extend infinite switch dynamics to a general case and consider infinite switch sampling also of the pressure. 5 6 Abstract In this thesis we discuss accelerated sampling schemes for high dimensional systems, for example molecular dynamics (MD). The development of these methods is fundamental to the effective study of a large class of problems, for which traditional methods con- verge slowly to the system's underlying invariant probability distribution. Due to the complexity of the landscape defined by an energy function (or, in statistical models, the log likelihood of the target probability density), the exploration of the probability distribution is severely restricted. This can have detrimental effects on the conclusions drawn from numerical experiments when potentially important states and solutions are absent in the examination of the results as a consequence of poor sampling. The aim of accelerated sampling schemes is to enhance the exploration of the in- variant measure by improving the rate of convergence to it. In this work, we first focus our attention on numerical methods based on canonical sampling by studying Langevin dynamics, for which the convergence is accelerated by extending the phase-space. We introduce a scheme based on simulated tempering which makes temperature into a dynamical variable and allows switching the temperature up or down during the ex- ploration in such a way that the target probability distribution can be easily obtained from the extended distribution. We show that this scheme is optimal when operated in the infinite switch limit. We discuss the limitations of this method and demonstrate the excellent exploratory properties of it for a moderately complicated biomolecule, alanine-12. Next, we derive a novel approach to constant pressure simulation that forms the basis for a family of pure Langevin barostats. We demonstrate the excellent numer- ical performance of Lie-Trotter splitting schemes for these systems and the superior accuracy and precision of the simultaneous temperature and pressure control in com- parison to currently available schemes. The scientific importance of this method lies in the ability to control the simulation and to make better predictions for applications in both materials modelling and drug design. We demonstrate this method in simulations of state transitions in crystalline materials using the \Mercedes Benz" potential. In a final contribution, we extend the infinite switch schemes to incorporate a general class of collective variables. In particular this allows for tempering in both tempera- ture and pressure when combined with our new barostat. We conclude this thesis by presenting a numerical study of the computational prospects of these methods. 7 8 Acknowledgements It is difficult to sum up the large number of people that influences a mathematical and scientific education that culminates in the decision to study for a PhD in mathematics. First, I must thank my undergraduate supervisors from Manchester, Helen Gleeson and Matthias Heil, that shaped my initial experience of academic life and were elemental in encouraging me to continue my studies in Edinburgh. I foremost thank my advisor Ben Leimkuhler whose perspective on mathematics and numerical analysis molded my own understanding of where an applied mathematician's role fits within the larger scientific community. I am grateful for the large number of people he has introduced me to and the fruitful collaborations these generated. From these collaborations I must first thank Eric Vanden-Eijnden for his influence on much of my work and whose tenacity helped push my mathematical understanding further. From my time at Duke I am grateful to Jianfeng Lu and Jonathan Mattingly who showed me the power of non-intimidating mathematical presentation. I want to thank Gabriel Stoltz and Kostas Zygalakis for their helpful discussions and Michela Ottobre who has forged my understanding in stochastic analysis throughout my studies. I thank Charles Matthews for his Bayesian sampling example detailed in Section 1.3.1 and Greg Herschlag for his knowledge of C++. I would also like to thank Zofia Trstanova and Matthias Sachs whom without I never would have made it. 9 10 Contents Lay Summary 5 Abstract 7 Acknowledgements 9 1 Introduction and Fundamentals 13 1.1 Molecular Dynamics . 13 1.1.1 A Microscopic Perspective on Macroscopic Properties . 14 1.2 Thermodynamic Ensembles . 15 1.2.1 NVE, Constant Energy . 16 1.2.2 NVT, Constant Temperaure . 18 1.2.3 NPT, Constant Temperature and Pressure . 19 1.3 Statistical Models . 20 1.3.1 Bayesian Inference . 21 1.3.2 Sampling Political Districts . 23 1.4 Computational Challenges . 24 1.4.1 Long Term Stability and Accuracy . 25 1.4.2 Metastability . 26 1.5 Original Contributions . 27 2 Foundations of Stochastic 31 2.1 Foundations of Stochastic Processes . 31 2.1.1 Stochastic Processes . 32 2.1.2 Ergodicity . 34 2.1.3 Convergence and Error of Numerical Methods . 37 2.2 Methods for High Dimensional Sampling . 38 2.2.1 Metropolis Hastings Algorithm . 39 2.2.2 Continuous Stochastic Dynamics . 41 3 An Overview of Accelerated Sampling Schemes 49 3.1 Simulated Annealing . 50 3.2 Simulated Tempering . 52 3.3 Parallel Tempering / Replica Exchange . 56 3.3.1 Accelerating Accelerated Sampling . 59 4 ISST w. Adaptive Weight Learning 63 4.1 Foundations . 65 4.1.1 A Continuous Formulation of Simulated Tempering . 65 4.1.2 Averaged Equations of Motion . 65 11 4.1.3 Infinite Switch Limit . 67 4.1.4 Estimation of Canonical Expectations . 69 4.1.5 Plausibility Argument for the Choice of Temperature Weights . 70 4.1.6 Adaptive Learning of Temperature Weights . 71 4.2 Implementation details of the ISST algorithm . 72 4.3 Numerical Experiments . 74 4.3.1 Harmonic Oscillator . 74 4.3.2 Curie-Weiss Magnet . 78 4.3.3 Alanine-12 . 84 4.3.4 Accelerated Congressional District Sampling . 85 5 Isobaric{Isothermal Sampling 87 5.1 A stochastic barostat based on Langevin dynamics . 88 5.1.1 The Periodic Flexible Simulation Cell . 89 5.2 Splitting Schemes for NPT Dynamics . 90 5.2.1 Brief Aside on A, µ and Friction . 93 5.3 Numerical Experiments . 94 5.3.1 Pressure Conservation Under Simulated Tempering . 98 5.3.2 Merceds-Benz Potential . 98 6 Generalised Infinite Switch Simulations 105 6.1 Introduction . 105 6.2 Generalised Infinite Switch Simulation . 106 6.3 Applied Field Tempering . 109 6.3.1 Double Well . 109 6.3.2 Curie Weiss Magnet . 110 6.3.3 Mercedes-Benz Potential . 112 7 Conclusion 119 12 Chapter 1 Introduction and Fundamentals This thesis focuses on the study of numerical sampling methods for dynamical systems with a large number of degrees of freedom. Molecular dynamics (MD) is an example of one such problem, which finds widespread use in much of modern computational science and is the main application on which we test the methods developed herein. Additional applications where these techniques are relevant include large dimensional statistical models such as deep neural networks and Bayesian inference. The development of accelerated sampling schemes, particularly in the MD setting, has matured over many years and the techniques we study in this thesis are based on the incorporation of temperature, for which the physical interpretation is clearly understood through thermodynamics and statistical mechanics. Drawing on the wide body of work on molecular dynamics, which also represents the most immediate application for our work, we present most of our results in the molecular modelling context. This does not exclude the application of our results in the setting of statistical models.

Load more

  132

Copy Link