Simulated Annealing A Basic Introduction Lecture 10 Olivier de Weck, Ph.D. Massachusetts Institute of Technology 1 Outline • Heuristics • Background in Statistical Mechanics – Atom Configuration Problem – Metropolis Algorithm • Simulated Annealing Algorithm • Sample Problems and Applications • Summary 2 Learning Objectives • Review background in Statistical Mechanics: configuration, ensemble, entropy, heat capacity • Understand the basic assumptions and steps in Simulated Annealing (SA) • Be able to transform design problems into a combinatorial optimization problem suitable to SA • Understand strengths and weaknesses of SA 3 Heuristics 4 What is a Heuristic? • A Heuristic is simply a rule of thumb that hopefully will find a good answer. • Why use a Heuristic? – Heuristics are typically used to solve complex (large, nonlinear, non- convex (i.e. contain local minima)) multivariate combinatorial optimization problems that are difficult to solve to optimality. • Unlike gradient-based methods in a convex design space, heuristics are NOT guaranteed to find the true global optimal solution in a single objective problem, but should find many good solutions (the mathematician's answer vs. the engineer’s answer) • Heuristics are good at dealing with local optima without getting stuck in them while searching for the global optimum. 5 Types of Heuristics • Heuristics Often Incorporate Randomization • Two Special Cases of Heuristics – Construction Methods • Must first find a feasible solution and then improve it. – Improvement Methods • Start with a feasible solution and just try to improve it. • 3 Most Common Heuristic Techniques – Simulated Annealing – Genetic Algorithms – Tabu Search – New Methods: Particle Swarm Optimization, etc… 6 Origin of Simulated Annealing (SA) • Definition: A heuristic technique that mathematically mirrors the cooling of a set of atoms to a state of minimum energy. • Origin: Applying the field of Statistical Mechanics to the field of Combinatorial Optimization (1983) • Draws an analogy between the cooling of a material (search for minimum energy state) and the solving of an optimization problem. • Original Paper Introducing the Concept – Kirkpatrick, S., Gelatt, C.D., and Vecchi, M.P., “Optimization by Simulated Annealing,” Science, Volume 220, Number 4598, 13 May 1983, pp. 671- 680. 7 MATLAB® “peaks” function • Difficult due to plateau at z=0, local maxima • SAdemo1 Peaks 3 • x =[-2,-2] o 2 • Optimum at Maximum 1 • x*=[ 0.012, 1.524] y 0 • z*=8.0484 -1 -2 Start Point -3 -3 -2 -1 0 1 2 3 x 8 “peaks” convergence • Initially ~ nearly random search • Later ~ gradient search SA convergence history 2 current configuration new best configuration 0 -2 -4 System Energy -6 -8 -10 0 50 100 150 200 250 300 Iteration Number 9 Statistical Mechanics 10 The Analogy • Statistical Mechanics: The behavior of systems with many degrees of freedom in thermal equilibrium at a finite temperature. • Combinatorial Optimization: Finding the minimum of a given function depending on many variables. • Analogy: If a liquid material cools and anneals too quickly, then the material will solidify into a sub-optimal configuration. If the liquid material cools slowly, the crystals within the material will solidify optimally into a state of minimum energy (i.e. ground state). – This ground state corresponds to the minimum of the cost function in an optimization problem. 11 Sample Atom Configuration Original Configuration Perturbed Configuration Atom Configuration - Sample Problem Atom Configuration - Sample Problem 7 7 6 6 5 5 4 3 4 3 4 3 4 2 2 2 3 2 1 1 1 1 0 0 -1 -1 -1 0 1 2 3 4 5 6 7 -1 0 1 2 3 4 5 6 7 E=133.67 E=109.04 Perturbing = move a random atom Energy of original (configuration) to a new random (unoccupied) slot 12 Configurations • Mathematically describe a configuration – Specify coordinates of each “atom” 1 3 4 5 r with i=1,..,4 r, r , r , r i 11 2 2 3 4 4 3 – Specify a slot for each atom rRi with i=1,..,4 1 12 19 23 13 Energy of a state • Each state (configuration) has an energy level associated with it H q,,,, q t qii p L q q t Hamiltonian i HTVEtot Energy of configuration = Objective function value of design kinetic potential Energy 14 Energy sample problem • Define energy function for “atom” sample problem 1 N 222 Ei mgy i x i x j y i y j potential ji energy kinetic energy N Absolute and relative position of E R Eii r i 1 each atom contributes to Energy 15 Compute Energy of Config. A • Energy of initial configuration Atom Configuration - Sample Problem 7 E 1 10 1 5 18 20 20.95 6 1 5 4 E2 1 10 2 5 5 5 26.71 3 3 4 E3 1 10 4 18 5 2 47.89 2 2 1 1 E4 1 10 3 20 5 2 38.12 0 -1 -1 0 1 2 3 4 5 6 7 Total Energy Configuration A: E({rA})= 133.67 16 Boltzmann Probability P! Number of configurations NR PN! P=# of slots=25 N =6,375,600 N=# of atoms =4 R What is the likelihood that a particular configuration will exist in a large ensemble of configurations? Er({ }) Boltzmann probability Pr{ } exp depends on energy kTB and temperature 17 Boltzmann Collapse at low T T=100 T=10 T=1 T=100 - E =100.1184 T=10 - E =58.9245 T=1 - E =38.1017 avg avg avg 14000 14000 12000 12000 12000 10000 10000 10000 8000 8000 8000 6000 Occurences 6000 Occurences 6000 Occurences 4000 4000 4000 2000 2000 2000 0 0 20 40 60 80 100 120 140 160 180 200 0 0 20 40 60 80 100 120 140 160 180 200 0 Energy 0 20 40 60 80 100 120 140 160 180 200 Energy Energy T high T low Boltzmann Distribution collapses to the lowest energy state(s) in the limit of low temperature Basis of search by Simulated Annealing 18 Partition Function Z • Ensemble of configurations can be described statistically • Partition function, Z, generates the ensemble average EENR Z Tr expii exp kBB Ti 1 k T Initially (at T>>0) equal to the number of possible configurations 19 Free Energy NR Average Energy of all EETETavg() i Configurations in an i 1 ensemble F T kB Tln Z E ( T ) TS NR ETi exp ETi i 1 kTB dZln EETavg () Z d1 kB T Relates average Energy at T with Entropy S 20 Specific Heat and Entropy • Specific Heat NR 1 2 2 ETETi () 2 2 dE() T NR i 1 ETET() CT 22 dT kBB T k T • Entropy T 1 CT() dS()() T C T S()() T S T1 dT T T dT T Entropy is ~ equal to ln(# of unique configurations) 21 Low Temperature Statistics Specific Heat C and Entropy S 5 10 10 T Entropy S 8 decreases 6 with lower T 4 2 # # of configurations, N # configs 0 Ht. Capacity 10 0 0 2 4 0 2 4 decreases 10 10 10 10 10 10 Temperature Temperature 150 150 g 100 v 100 a 50 50 0 0 -50 -50 Approaches Free Energy, F Free -100 Energy, Average E -100 E* from Approaches -150 -150 above 0 2 4 0 2 4 E* from 10 10 10 10 10 10 Temperature Temperature below 22 Minimum Energy Configurations • Sample Atom Placement Problem Uniqueness? Atom Configuration - Sample Problem Atom Configuration - Sample Problem 7 7 6 6 5 E*=60 5 E*=14.26 4 4 3 3 2 2 3 2 1 2 1 4 3 1 4 1 0 0 -1 -1 -1 0 1 2 3 4 5 6 7 -1 0 1 2 3 4 5 6 7 Strong gravity g=10 Low gravity g=0.1 23 Simulated Annealing 24 Dilemma • Cannot compute energy of all configurations ! – Design space often too large – Computation time for a single function evaluation can be large • Use Metropolis Algorithm, at successively lower temperatures to find low energy states – Metropolis: Simulate behavior of a set of atoms in thermal equilibrium (1953) – Probability of a configuration existing at T Boltzmann Probability P(r,T)=exp(-E(r)/T) 25 The SA Algorithm • Terminology: – X (or R or ) = Design Vector (i.e. Design, Architecture, Configuration) – E = System Energy (i.e. Objective Function Value) – T = System Temperature – = Difference in System Energy Between Two Design Vectors • The Simulated Annealing Algorithm 1) Choose a random Xi, select the initial system temperature, and specify the cooling (i.e. annealing) schedule 2) Evaluate E(Xi) using a simulation model 3) Perturb Xi to obtain a neighboring Design Vector (Xi+1) 4) Evaluate E(Xi+1) using a simulation model 5) If E(Xi+1)< E(Xi), Xi+1 is the new current solution 6) If E(Xi+1)> E(Xi), then accept Xi+1 as the new current solution with a (- /T) probability e where = E(Xi+1) - E(Xi). 7) Reduce the system temperature according to the cooling schedule. 8) Terminate the algorithm. 26 SA Block Diagram Define initial Evaluate energy Start To configuration Ro E(Ro) Compute energy Perturb configuration Evaluate energy End Tmin y T<Tmin? n _ difference Ri > Ri+1 E(Ri+1) E = E(Ri+1)-E(Ri) n Reduce temperature Reached y equilbrium Accept Ri+1 as y Ti+1 =Tj - T new configuration E < 0? at Tj ? y n Keep R as current i E Create random configuration n exp - > v? T number v in [0,1] Metropolis step SA Block Diagram Image by MIT OpenCourseWare. 27 MATLAB Function: SA.m Best Configuration Initial Configuration SA.m xbest Ebest xo Simulated Annealing Option Flags Algorithm History options xhist evaluation perturbation function function file_eval.m file_perturb.m 28 Exponential Cooling k • Typically (T1/To)~0.7-0.9 T TT1 Simulated Annealing Evolution kk1 10 C-specific heat To 9 S-entropy T/10-temperature 8 7 Can also do linear 6 or step-wise cooling 5 4 3 … 2 But exponential cooling 1 often works best.