Introduction to Monte Carlo Methods

Introduction Classical Monte Carlo Quantum Monte Carlo Summary

Tim Schoof

3rd Graduate Summer Institute “Complex Plasmas”

August 6, 2012 Introduction Classical Monte Carlo Quantum Monte Carlo Summary Applications of Monte Carlo Introduction Classical Monte Carlo Quantum Monte Carlo Summary Outline

1 Introduction History Monte Carlo integration

2 Classical Monte Carlo Classical thermodynamics Metropolis algorithm Details

3 Quantum Monte Carlo Quantum thermodynamics Path integral Sign problem

4 Summary Introduction Classical Monte Carlo Quantum Monte Carlo Summary Table of contents

1 Introduction History Monte Carlo integration

2 Classical Monte Carlo Classical thermodynamics Metropolis algorithm Details

3 Quantum Monte Carlo Quantum thermodynamics Path integral Sign problem

4 Summary Introduction Classical Monte Carlo Quantum Monte Carlo Summary Monte Carlo

Monte Carlo methods are a class of computational algorithms that rely on repeated random sampling to compute their results Invented by Stanislaw Ulam and John von Neumann Named after the Monte Carlo Casino, where Ulam’s uncle often gambled away his money Introduction Classical Monte Carlo Quantum Monte Carlo Summary The basic idea

In 1946, being ill and playing solitaire, Ulam was interested in the chances to win the game. After spending a lot of time trying to estimate them by pure combinatorial calculation, he came up with the idea to simply lay out the cards 100 times and count the number of successful plays. Soon he realizes that this approach can be applied to the problem of neutron transport Stanislaw Ulam with FERMIAC Introduction Classical Monte Carlo Quantum Monte Carlo Summary The beginnings

Together with von Neumann the ﬁrst calculations were performed for the JOURNAL OF THE AMERICAN Manhatten Project at Los Alamos on the STATISTICAL ASSOCIATION

ENIAC Number 167 SEPTEMBER 1949 Volume 44 The ENIAC was the first electronic THE MONTE CARL0 METHOD NICHOLASMETROPOLIS AND S. ULAM Los Alamos Laboratory general-purpose computer We shall present here the motivation and a general description of a method dealing with a class of problems in mathematical physics. The method is, essentially, a statistical approach to the study of differential equations, or more The first unclassified paper on Monte Carlo generally, of integro-differential equations that occur in various branches of the natural sciences. was published in 1949 LREADY in the nineteenth century a sharp distinction began to ap- A pear between two different mathematical methods of treating physical phenomena. Problems involving only a few particles were studied in classical mechanics, through the study of systems of ordinary differential equations. For the description of systems with very many particles, an entirely different technique was used, namely, the method of statistical mechanics. In this latter approach, one does not concen- trate on the individual particles but studies the properties of sek of particles. In pure mathematics an intensive study of the properties of sets of points was the subject of a newTfield. This is the so-called theory of sets, the basic theory of integration, and the twentieth century development of the theory of probabilities prepared the formal apparatus for the use of such models in theoretical physics, i.e., description of properties of aggregates of points rather than of individual points and their coordinates. Soon after khe development of the calculus, the mathematical apparatus of partial differential equations was used for dealing with the problems of the physics of the continuum. Hydrodynamics is the most widely known field formulated in this fashion. A little later came the treatment of the problems of heat conduction and still later the field theories, like the electromagnetic theory of Maxwell. All this is very well known. It is of course important to remember that the study of the Introduction Classical Monte Carlo Quantum Monte Carlo Summary Basic example: Area of the unit circle

Choose M random points that are uniformly distributed within the enclosing square Count the number of points N that are within the unit circle An estimation for the area of the circle is given by

N A ≈ 22 M In the left picture it is A ≈ 3.15 Introduction Classical Monte Carlo Quantum Monte Carlo Summary Area of the m-dimensional unit hypersphere

m Exact 2 3.1415 3 4.1887 4 4.9348 5 5.2637 6 5.1677 7 4.7247 8 4.0587

Exact result

m m md π 2 r V = m Γ( 2 + 1) Introduction Classical Monte Carlo Quantum Monte Carlo Summary Area of the m-dimensional unit hypersphere

m Exact quad. time result 2 3.1415 0.00 3.13 3 4.1887 1.0 · 10−4 4.21 4 4.9348 1.2 · 10−3 4.97 5 5.2637 0.03 5.29 6 5.1677 0.62 5.20 7 4.7247 14.9 4.77 8 4.0587 369 4.09

Numerical integration by quadrature, e.g., for 3d Z q V 3d = 2 dx dy z(x, y) with z(x, y) = 1 − (x 2 + y 2)

x 2+y 2≤1 Introduction Classical Monte Carlo Quantum Monte Carlo Summary Area of the m-dimensional unit hypersphere

m Exact quad. time result MC time result 2 3.1415 0.00 3.13 0.07 3.14 3 4.1887 1.0 · 10−4 4.21 0.09 4.19 4 4.9348 1.2 · 10−3 4.97 0.12 4.93 5 5.2637 0.03 5.29 0.14 5.27 6 5.1677 0.62 5.20 0.17 5.17 7 4.7247 14.9 4.77 0.19 4.72 8 4.0587 369 4.09 0.22 4.07

Monte Carlo integration m m-dimensional points ~xi are uniformly sampled in volume V = 2

M 1 X V md = 2m Θ(~x ) with Θ(~x ) = 1 if ~x · ~x ≤ 1 M i i i i i=1 Introduction Classical Monte Carlo Quantum Monte Carlo Summary Area of the m-dimensional unit hypersphere

=⇒ Monte Carlo integration is faster for m > 6

Conclusion Monte Carlo integration is efﬁcient for high dimensional integrals Introduction Classical Monte Carlo Quantum Monte Carlo Summary Monte Carlo integration

Straightforward sampling Random points {xi } are chosen uniformly

M Z b b − a X I = f (x)dx ≈ f (x ) M i a i=1 Importance sampling {xi } are chosen with the probability p(x)

Z b M f (x) 1 X f (xi ) I = p(x)dx ≈ p(x) M p(xi ) a i=1

p(x) should be close to |f (x)| Introduction Classical Monte Carlo Quantum Monte Carlo Summary Table of contents

1 Introduction History Monte Carlo integration

2 Classical Monte Carlo Classical thermodynamics Metropolis algorithm Details

3 Quantum Monte Carlo Quantum thermodynamics Path integral Sign problem

4 Summary Introduction Classical Monte Carlo Quantum Monte Carlo Summary Thermodynamic expectation values

In thermodynamic equilibrium classical expectation values of observables O are given by Z dPdR hOi = O(R, P)p(R, P) h3N N!

with R = ~r1,...,~rN , P = ~p1,..., ~pN and the integral is over the whole phase space In the canonical ensemble for ﬁxed T , V , N the probability p(R, P) of a microstate (R, P) is given by 1 1 p(R, P) = e−βH(R,P), β = Z kBT Z dPdR Z(T , V , N) = e−βH(R,P) h3N N! It is Z dPdR p(R, P) = 1, p(R, P) ≥ 0 ∀ R, P h3N N! Introduction Classical Monte Carlo Quantum Monte Carlo Summary Metropolis algorithm

Developed by Marshall N. Rosenbluth THE 0 R Y 0 F T RAe KEF FEe T SIN R A D I 0 L Y SIS 0 F W ATE R 1087

instead, only water molecules with different amounts of paths (a) and (b) can be designated H 20* and those excitation energy. These may follow any of three paths: following path (c) can be designated H 20t. It seems Named after Nicholas Metropolis the head reasonable to assume for the purpose of these calcula (a) The excitation energy is lost without dissociation tions that the ionized H 0 molecules will become the into radicals (by collision, or possibly radiation, as in 2 H 20t molecules, but this is not likely to be a complete aromatic hydrocarbons). correspondence. (b) The molecules dissociate, but the resulting radi designer of the MANIAC In conclusion we would like to emphasize that the cals recombine without escaping from the liquid cage. qualitative result of this section is not critically de (c) The molecules dissociate and escape from the pendent on the exact values of the physical parameters cage. In this case we would not expect them to move used. However, this treatment is classical, and a correct more than a few molecular diameters through the dense treatment must be wave mechanical; therefore the medium before being thermalized. result of this section cannot be taken as an a priori First paper on 2d rigid spheres published in 1953 theoretical prediction. The success of the radical diffu In accordance with the notation introduced by Burton, Magee, and Samuel,22 the molecules following sion model given above lends some plausibility to the occurrence of electron capture as described by this One of the most important algorithms of the 20th 22 Burton, Magee, and Samuel, J. Chern. Phys. 20, 760 (1952). crude calculation. Further work is clearly needed.

century THE JOURNAL OF CHEMICAL PHYSICS VOLUME 21, NUMBER 6 JUNE, 1953 Equation of State Calculations by Fast Computing Machines

NICHOLAS METROPOLIS, ARIANNA W. ROSENBLUTH, MARSHALL N. ROSENBLUTH, AND AUGUSTA H. TELLER, Los Alamos Scientific Laboratory, Los Alamos, New Mexico Used in physics, statistics, economics and AND EDWARD TELLER, * Department of Physics, University of Chicago, Chicago, Illinois (Received March 6, 1953) computer science A general method, suitable for fast computing machines, for investigatiflg such properties as equations of state for substances consisting of interacting individual molecules is described. The method consists of a modified Monte Carlo integration over configuration space. Results for the two-dimensional rigid-sphere system have been obtained on the Los Alamos MANIAC and are presented here. These results are compared to the free volume equation of state and to a four-term virial coefficient expansion.

I. INTRODUCTION II. THE GENERAL METHOD FOR AN ARBITRARY POTENTIAL BETWEEN THE PARTICLES HE purpose of this paper is to describe a general T method, suitable for fast electronic computing In order to reduce the problem to a feasible size for machines, of calculating the properties of any substance numerical work, we can, of course, consider only a finite which may be considered as composed of interacting number of particles. This number N may be as high as individual molecules. Classical statistics is assumed, several hundred. Our system consists of a squaret con only two-body forces are considered, and the potential taining N particles. In order to minimize the surface field of a molecule is assumed spherically symmetric. effects we suppose the complete substance to be periodic, consisting of many such squares, each square contain These are the usual assumptions made in theories of ing N particles in the same configuration. Thus we liquids. Subject to the above assumptions, the method define dAB, the minimum distance between particles A is not restricted to any range of temperature or density. and B, as the shortest distance between A and any of This paper will also present results of a preliminary two the particles B, of which there is one in each of the dimensional calculation for the rigid-sphere system. squares which comprise the complete substance. If we Work on the two-dimensional case with a Lennard have a potential which falls off rapidly with distance, Jones potential is in progress and will be reported in a there will be at most one of the distances AB which later paper. Also, the problem in three dimensions is can make a substantial contribution; hence we need being investigated. consider only the minimum distance dAB.

* Now at the Radiation Laboratory of the University of Cali t We will use thl~ two-dimensional nomenclature here since it fornia, Livermore, California. is easier to visualize. The extension to three dimensions is obvious.

Downloaded 26 Jul 2012 to 134.245.67.230. Redistribution subject to AIP license or copyright; see http://jcp.aip.org/about/rights_and_permissions Introduction Classical Monte Carlo Quantum Monte Carlo Summary Markov chain

Starting from an initial state x0 a new microstate xi+1 is generated from previous microstate xi

The new state xi+1 is chosen with transition probability v(xi → xi+1)

For the states {xi } being distributed according to p(x), v(xi → xi+1) has to satisfy the following sufﬁcient conditions: X 1 Conservation law: v(xi xi+1) = 1 xi → ∀ xi+1 2 Positive semi-deﬁnite: v(xi xi+1) 0 xi , xi+1 → ≥ ∀

3 Detailed balance: p(xi )v(xi xi+1) = p(xi+1)v(xi+1 xi ) → →

4 Ergodicity: From each microstate x every other microstate x 0 can be reached in ﬁnite time with ﬁnite probability Introduction Classical Monte Carlo Quantum Monte Carlo Summary Metropolis-Hastings algorithm

Separate the transition step into 2 sub-steps

0 0 v(xi → xi+1) = Q(xi → x )Pacc(xi , x )

0 Proposal density Q(xi → x ) 0 Acceptance probability Pacc(xi , x ) Metropolis-Hastings

0 0 0 Q(x → xi ) p(x ) Pacc(xi , x ) = min 1, 0 Q(xi → x ) p(xi )

0 0 Original Metropolis: Q(xi → x ) = Q(x → xi ) Draw random number u ∈ [0, 1)

if u Pacc(xi , x 0): ≤ Accept proposed state xi+1 = x 0 ⇒ if u > Pacc(xi , x 0): Reject proposed state xi+1 = xi ⇒ Introduction Classical Monte Carlo Quantum Monte Carlo Summary Typical program

1 Choose initial microstate 0 0 2 Propose new state x according to proposal density Q(xi → x ) 0 3 Accept or reject new state according to Pacc(xi , x )

4 Repeat steps 2 and 3 Ncycle times to reduce autocorrelation 5 “Measure” quantities of interest in current microstate 6 Repeat steps 2 to 5 many times

7 Throw away the ﬁrst Neq measurements (equilibration time) 8 Compute averages from M remaining measurements

M 1 X hOi ≈ O(x ) M i i=1 Introduction Classical Monte Carlo Quantum Monte Carlo Summary Example: Dust clusters

The momenta P can be treated analytically Start with uniformly distributed particle coordinates R In the most simple case one just moves single particles 1 Choose uniformly a single particle i 1,..., N 2 ∈ { } 3 Choose uniformly a new position ~ri0 within sample volume Vs = (2δr)

x 0 = xi + (2u 1)δr, u [0, 1) − ∈ y 0 = yi + (2u 1)δr − z0 = zi + (2u 1)δr − 1 1 Proposal density Q(R R0) = Q(R0 R) = is symmetric → → N Vs 3 Calculate energy difference ∆E = H(...,~ri0,...) H(...,~ri ,...) −β∆E 4 Accept or reject with probability Pacc = min 1, e−

A good choice is Ncycle ≈ N and Neq ≈ 0.1M Calculate energy, heat capacity, n-particle correlation functions, . . . Introduction Classical Monte Carlo Quantum Monte Carlo Summary Optimizations

Efﬁciency of the algorithm depends strongly on the moves Proposal density should be chosen so that on average 20%-50% of the moves are accepted E.g. choose new position of a particle according to a Gaussian distribution centered on the old position, adapt σ to best acceptance ratio It is possible to use “unphysical” moves, MC is not a time dynamic It is possible to extend the conﬁguration space to “unphysical” microstates ⇒ just restrict the averaging to physical microstates e.g., use grand canonical ensemble to simulate canonical expectation values Use parallel tempering to avoid trapping in metastable states at low temperatures And many other methods . . . Introduction Classical Monte Carlo Quantum Monte Carlo Summary Random numbers

For reliable MC simulations the quality of the random numbers is crucial! In most cases random numbers from pseudorandom number generators are used Do not use linear congruential generators like C built-in rand() Instead use libraries (GSL, boost, . . . ) which implement well-tested high-quality rngs, e.g., Mersenne twister which has a period of 219997 − 1 Introduction Classical Monte Carlo Quantum Monte Carlo Summary Statistical Error

For a sample O1,..., OM of size M the mean is given by

M 1 X hOi = O M i i=1 Standard deviation v u M u 1 X σ = t (O − hOi)2 M − 1 i i=1

For uncorrelated data the error of the mean is given by σ ∆O = √ M √ The statistical error decreases ∼ M Introduction Classical Monte Carlo Quantum Monte Carlo Summary Correlation

Autocorrelation function

M−l 1 X C(l) = (O − hOi)(O − hOi) σ2(n − l) i i+l i=1 Integrated autocorrelation time

1 X τ = + C(l) int 2 l≥1

Statistical error with autocorrelation σ p ∆O = √ 2τint M

Reduced effective sample size M/2τint

Use e.g. binning analysis to estimate τint Introduction Classical Monte Carlo Quantum Monte Carlo Summary Table of contents

1 Introduction History Monte Carlo integration

2 Classical Monte Carlo Classical thermodynamics Metropolis algorithm Details

3 Quantum Monte Carlo Quantum thermodynamics Path integral Sign problem

4 Summary Introduction Classical Monte Carlo Quantum Monte Carlo Summary Quantum statistics

For quantum particles thermodynamic expectation values of observables Oˆ are given by

hOˆ i = TrOˆρˆ

Density operator X ρˆ = P(|ψi i) |ψi i hψi | i In canonical ensemble

1 ˆ ρˆ = e−βH Z ˆ Z = Tr e−βH

Trace can be evaluated in any basis, e.g., coordinate space |Ri Z hOˆ i = dR hR|Oˆρˆ|Ri Introduction Classical Monte Carlo Quantum Monte Carlo Summary Path integral I It is M −βHˆ h − β Hˆ i e = e M Rewrite density matrix

0 0 1 −βHˆ 0 ρ(R, R , β) := hR |ρˆ(β) | R i = hR|e |R i Z

M factors z }| { 1 − β Hˆ − β Hˆ 0 = hR| e M · ... · e M |R i Z ↑ ↑ Insert M−1 identities Iˆ=R dR |RihR| Z Z 1 − β Hˆ − β Hˆ 0 = dR ··· dR hR|e M |R i · · · hR |e M |R i Z 1 M−1 1 M−1

1 Z = dR dR ... dR ρ(R, R ; τ)ρ(R , R ; τ) ··· ρ(R , R0; τ) Z 1 2 M−1 1 1 2 M−1 β with τ = M Introduction Classical Monte Carlo Quantum Monte Carlo Summary Path integral II

Use high temperature approximations for matrix elements, e.g.

2 − β (Kˆ +Vˆ ) − β Kˆ β Vˆ β e M ≈ e M e M + O( ) M2 Primitive approximation more Z 0 1 − PM Sk ln ρ −S ρ(R, R ; β) = dR dR ... dR e k=1 , ρ = e ≡ e Z 1 2 M−1 π k = − [ρ( , ; τ)] ≈ λd·N + ( − )2 + τ ( ), S ln Rk−1 Rk ln τ 2 Rk Rk−1 V Rk λτ

Sk is called action Introduction Classical Monte Carlo Quantum Monte Carlo Summary Particle exchange

Symmetry of wavefunctions has to be taken into account

Ψ(...,~ri ,...,~rj ,...) = ±Ψ(...,~rj ,...,~ri ,...) Bosonic wavefunctions are symmetric (+) under particle permutation Fermionic wavefunctions are antisymmetric (−) To restore symmetry use projector on (anti-)symmetric states

ˆ 1 X P ˆ P ± = (±1) P N! P∈SN

with permutation operator Pˆ It is sufﬁcient to use projector only on one of the M states ρ±(R, R0) = Z Z 1 1 X β ˆ β ˆ P − M H − M H ˆ 0 (±1) dR ··· dR − hR|e |R i · · · hR − |e P |R i Z N! 1 M 1 1 M 1 P Introduction Classical Monte Carlo Quantum Monte Carlo Summary Example: 5 electrons in a harmonic trap

Pair exchange of particle 1 and 2 Introduction Classical Monte Carlo Quantum Monte Carlo Summary Sign problem

For fermions weight w(x) can be negative Possible solution: use |w(x)| Z p0(x) = , Z 0 = |w(x)|dx Z 0 Expectation values 0 R 0 ˆ ˆ O(x)s(x)p (x)dx hO si hO i = R = 0 , s(x) = sign w(x) s(x)p0(x)dx hsi Average sign

Z 0 hsi0 = = e−βN(f −f ) Z 0 Statistical error

∆O ∆s σs βN∆f ∼ 0 = √ e hOˆ i hsi L Sign problem is NP-hard1 1Matthias Troyer, PRL 94, 170201 (2005) Introduction Classical Monte Carlo Quantum Monte Carlo Summary Basic idea

In standard PIMC antisymmetry is included in density matrix

hOˆ i = TrOˆρˆAˆ = TrOˆρˆA

Trace is over simple product states Instead use antisymmetric states in trace

hOˆ i = TrA Oˆρˆ

A CONS of antisymmetric N-particle Hilbert space HN in occupation number representation

∞ X |{n}i := |n0n1n2 ...i , ni = N i=0

with ni = 0, 1 occupation of single particle state |φi i, |φi i CONS of H1 Introduction Classical Monte Carlo Quantum Monte Carlo Summary CPIMC

Canonical partition function in the interaction picture representation ∞ X X X X Z β Z β Z β Z (β) = ... dτ1 dτ2 ... dτK 0 τ τ K =0 {n} {n(1)} {n(K −1)} 1 K −1 K 6=1 (0) (K ) ={n }={n } 6={n(0)} 6={n(K −2)},6={n(K )}

( K ) K −1 K X Y (−1) exp − D{n(i)}(τi+1 − τi ) Y{n(i)},{n(i+1)} i=0 i=0 Visual representation (3)

) {n } τ (

} (0)

n {n } { {n(2)}, {n(4)}

state {n(1)} 0 τ1 τ2 τ3 τ4 τ5 β imaginary time τ Introduction Classical Monte Carlo Quantum Monte Carlo Summary Result

β = 2 1 β = 2 a) b)

1 N = 3 N = 6 10−

i NB = 45 NB = 45 s h

2 10− CPIMC(ideal) β = 5 CPIMC(HF) 3 10− DPIMC average sign

4 10− β = 5

5 10− 0.1 1 10 0.1 1 10 coupling parameter λ coupling parameter λ Reduced sign problem for highly degenerate, weakly interacting fermions Increased sign problem for non-degenerate, strongly interacting fermions from Hamiltonian matrix elements PIMC and CPIMC are complementary Transition region where exact simulations are difﬁcult Introduction Classical Monte Carlo Quantum Monte Carlo Summary Result

β = 2 1 β = 2 a) b)

1 N = 3 N = 6 10−

i NB = 45 NB = 45 s h

2 10− CPIMC(ideal) β = 5 CPIMC(HF) 3 10− DPIMC average sign

4 10− β = 5

1 Introduction History Monte Carlo integration

2 Classical Monte Carlo Classical thermodynamics Metropolis algorithm Details

3 Quantum Monte Carlo Quantum thermodynamics Path integral Sign problem

4 Summary Introduction Classical Monte Carlo Quantum Monte Carlo Summary Summary

MC can evaluate high dimensional integrals efﬁciently Thermodynamic expectation values in equilibrium can be calculated with arbitrary accuracy For bosons quantum statistics can be taken into account by efﬁcient PIMC simulation For fermions the sign problem remains a challenge No straightforward way to get real-time dynamics or non-equilibrium properties Introduction Classical Monte Carlo Quantum Monte Carlo Summary Outlook

Kinetic MC → Real time dynamics Variational MC, Diffusion MC → QM ground state Restricted PIMC, Direct PIMC → Fermion sign problem And many more . . . Thank you for your attention! Appendix

Results for kinetic and harmonic operators back

Potential energy density matrix in coordinate representation ˆ ~ri |~ri = ~ri |~ri ˆ D P ~ˆ ~ˆ ~ˆ E P ~ ~ ~ −βV 0 −β Vext(ri )+Vij (ri ,rj ) 0 −β Vext(ri )+Vij (ri ,rj ) 0 hR|e |R i = R e R = e δ(R − R )

Kinetic energy density matrix in coordinate representation Z ˆ 0 ˆ 0 0 ~pi |~ri = d~p ~pi |~p i h~p |~ri

∗ 1 −i~r·~p/ ~r ~p = h~p|~ri = e ~ (2π~)3/2 ˆ Z P ~pi D −βKˆ 0E 0 00 0 0 −β 00 00 0 R e R = dP dP hR | P i P e 2mi P hP | R i

π 0 2 − (Ri −R ) −3N λ2 i D = λD e

−1/2 Deﬁnitions: λD = 2π~(2πmkBT ) , |Ri = { | ~r1i | ~r2i · · · | ~rN i }.