A Fast Algorithm for Simulated Annealing

Physica Scripta. Vol. T38, 40-44, 1991. A Fast Algorithm for Simulated Annealing

Hong Guo, Martin Zuckermann, R. Harris and Martin Grant

Centre for the Physics of Materials, Department of Physics, McGill University, 3600 University Street, Montreal, Quebec, Canada H3A 2T8

Received September 24, 1990; accepted October 26, 1990

Abstract Gaussian distribution (Gaussian model), or takes values of We present a new deterministic algorithm for simulated annealing and fJ with equal probability ( f J model). The sum is over all demonstrate its applicability with several classical examples: the ground state the nearest neighbor pairs. The spin glass system is highly energies of the 2d and 3d short range king spin glasses, the traveling salesman frustrated and has a large number of metastable states with problem, and pattern recognition in computer vision. Our algorithm is based similar energies. Its ground state also has high degeneracy. At on a microcanonical Monte Carlo method and is shown to be a powerful tool low temperatures it can be trapped in one of the metastable for the analysis of a variety of problems involving combinatorial optimiz- states. Thermal noise then induces uphill steps in energy ation. We show that the deterministic method generates optimal solutions faster and often better than the standard Metropolis method. space which in principle eventually brings the system to equilibrium. The simulated annealing method starts with the system at high temperature and lowers the temperature in 1. Introduction small steps according to a prescribed annealing schedule. For Over the last two decades extensive analytical and numerical a SG, the system energy is used as the cost function. At each work has been performed in order to obtain approximate temperature, enough simulation steps (spin flip trials) must solutions to optimization problems involving many par- be performed in order to ensure the system is equilibrated ameters and conflicting constraints [ 11. Classic examples of [3, 41. At very low temperature, this process causes the SG such problems include the traveling salesman problem (TSP), system to freeze into a state which is at least an approxi- the three dimensional spin glass ground state, and the wiring mation to the true ground state. For other combinational of gates in chip design. There exists an entire class of these optimization problems, a temperature is also introduced as a problems which is termed “NP-complete” (non-deterministic control parameter measured in units of the cost function. polynomial time complete) because the computational effort Usually simulated annealing is performed using the used to find an exact solution increases exponentially as the Metropolis [5] algorithm with temperature T fixed at each total number of degrees of freedom, N, of the problem. This state of the annealing schedule. The total cost function of the implies that approximation methods are required for further system is lowered due to local ordering and the temperature analysis. Indeed, heuristic methods are widely used for such provides the activation necessary to bring the system out of practical optimization problems in computer science and its metastable states. However, for metastable states with engineering and they prove to be extremely fruitful. However, high energy barriers, it is very hard for the thermal noise to these methods are usually problem-oriented and their domain provide enough activation. Thus the system can be locked of application is restricted to the particular problem which into a local minimum of the cost function space. This is the they are designed to solve. As the value of N for a particular reason that the annealing schedule must be carefully designed problem increases, algorithms based on statistical method and the annealing steps must be small [3, 41. become useful, and allow unified treatment of the NP-complete In this paper we present a study of an algorithm for problems. The most successful statistical method to date is simulated annealing which directly and deterministically the stochastic model of simulated annealing introduced by minimizes the cost function. The method is based on a micro- Kirkpatrick, Gelatt, and Vecchi in 1983 [2]. canonical ensemble in which the system of interest is in In simulated annealing, a cost function is constructed to contact with an auxiliary system (a “demon”) and the cost characterize the optimization process and minimization of function (energy) is exchanged between both systems. In this this function gives an approximation to the optimal solution method the temperature is actually a derived quantity. This of the problem. A controlled thermal treatment followed by allows large local temperature fluctuations which are essen- slow cooling give the system the chance to jump out of local tial for bringing the system out of metastable states. The minima of the cost function to improve solutions. Construct- methodology of the microcanonical Monte Carlo was first ing the cost function can in principle be problematic, although proposed by Creutz [6] to study the Ising model and has in practice the choice is often straightforward. For instance, subsequently been applied to a variety of physical systems [7]. the distance the salesman travels is the cost function of TSP. Although equilibrium statistical mechanics using the canonical Simulated annealing has been applied to the problem of and microcanonical ensembles are equivalent, the dynamics finding the ground state of a spin glass (SG) which is an of the algorithms based on these ensembles are different. We NP-complete problem in three dimensions [3, 41. A short find that this microcanonical Monte Carlo method is very range Ising spin glass is described by the Hamiltonian naturally applicable to optimization problems because it is deterministic and allows large temperature fluctuations. To our knowledge, an early attempt to use microcanonical where Si = f 1 is the spin at site i of a cubic lattice; Jij is Monte Carlo to study low temperature properties of a spin- usually assumed to be a random number drawn from a glass was due to Dasgupta, Ma and Hu [8]. Recently, Sourlas

Physica Scripta T38 A Fast Algorithm for Simulated Annealing 41

[9] has also applied a microcanonical method to investigate We have applied our method to study both spin glass the ergodicity properties of a spin-glass. While the ideas are models. In 2d, the systems were on square lattices with up to similar, the algorithm to be presented below is most close to 100’ spins. In 3d, cubic lattices with up to 303spins were used. the one proposed by Clover [lo]. Periodic boundary conditions were used for all simulations. In Section 2 we present the method and apply it to several The units of energy were taken as J for the fJ model and 6J classic problems: the Ising spin glass, the STP and a short for the Gaussian model. discussion on its application to computer vision. Section 3 is For the 2d Gaussian model, we started with Er= 8 and reserved for a short summary and conclusion. All of our annealed down to Er= 0 with equal steps of 1. The simulations were performed on a SUN 3/50 workstation. systems used consisted of 100’ spins, and 400 or 800 Monte Carlo trials per spin were done for each Er.10 runs were 2. Method and results averaged to give the value Eo/GJ x - 1.30 f 0.004, which is in good agreement with that given by the transfer matrix Although the microcanonical Monte Carlo method has been method. Our results also compare favorably with those of the discussed in detail by Creutz [6], we briefly present it here Metropolis algorithm. for completenes. Consider a nearest neighbor Ising model For the 3d Gaussian model, the range of we used was described by Hamiltonian (1) with J,j = J, from 8 down to 0 with steps of 1 for the first 5 stages and 0.5 for the last 5 stages. For lattices with 163 spins, 600 Monte Elsing = - J SiS’. (2) 1 Carlo trials per spin were done for each of the r.Averaging To study its equilibrium properties, a microcanonical ensemble 30 independent runs gave Eo/SJ x - 1.674 f 0.0015 which is constructed by letting a demon with energy Ed interact with is somewhat higher than the transfer matrix estimate, although the spins such that the total energy E = Elsing+ Ed is con- within the error bars, but considerably lower than that found served. In a Monte Carlo simulation, the energy required to using the Metropolis Monte Carlo [3]. For lattices with 303 flip a spin, 6E, is compared with Ed, and if Ed 2 GE, the flip spins, test runs were also done using the same annealing is permitted and an amount of 6E is subtracted from Ed. schedule but with only 300 Monte Carlo trials per spin per Otherwise the trial is abandoned. It is easy to show [6] that the Er, we foundf Eo/6J = - 1.667 after averaging 5 runs. average demon energy measures the temperature of the Ising Since the transfer matrix calculation quoted above is per- system in equilibrium. formed on systems with 43 spins, and there is very little The microcanonical Monte Carlo method is generalized information about the ground state energy for larger systems, for simulated annealing as follows. We begin the simulation our results could provide a benchmark for further simulations. with the system at a disordered high temperature state and The 2d & J model was studied by using from 16 allow it to interact with the demon. At each stage of the down to 0 in steps of 4, with either 800 or 1600 Monte Carlo annealing schedule a maximum value of the demon cost steps per spin for each of the Eron lattices with 50’ spins. function, CY (e.g., Ed for SG), is specified such that if the Averaging 10 independent configurations gave Eo/J x demon cost function cd > Cy,cd is reset to Cr.This - 1.401 & 0.004, again in excellent agreement with the procedure brings the system very efficiently to lower and transfer matrix calculation. Test runs on systems with 100’ lower cost function states. For frustrated systems with local spins essentially gave the same result. The Metropolis algor- ordering, a “hot” demon can melt a local ordered region and ithm gives Eo/J x - 1.3974 while the extrapolation gives bring the system out of metastable state. In the simulations - 1.398 [4]. Our results agree very well with these values. reported below, we found that the choice of CyXwas rather The 3d f J model simulation used the range of Erfrom robust: any reasonable choice did equally well. This method 20 down to 0 in steps of 4. A total of 12 Y’Swere used for can also be used after a heuristic treatment of the problem. In a run, 600 or 1200 Monte Carlo steps were done for each that case, since the initial state is almost ordered, we start ETX.For lattices of 163 spins, 40 independent runs were with the demon possessing a large value of the cost function averaged to find the value of the ground state energy, and then anneal down. E,:J z - 1.773 f 0.001 (we have also performed test runs on lattices with 303spins; 5 independent runs averaged to give 2.1. Ground state energy of the Zsing spin glass - 1.772). Note that this value is actually lower than that of The short range Ising spin glass described by (1) has been the transfer matrix calculation, though within its error bars. extensively studied for many years [l 13. For the fJ model, Grest et al. [4] give a value of - 1.7706 using Metropolis transfer matrix calculations [ 121 yield the ground state Monte Carlo, which they extrapolate to - 1.791. energies: Eo/Jx - 1.40 f 0.01 for d = 2, and Eo/J x Figure 1 shows the decrease of energy as a function of time - 1.76 f 0.02 for d = 3, where d is the dimensionality of the for the 2d models using the parameters mentioned above. The system. For the Gaussian model, the transfer matrix method energy decreases continuously until Eris lowered and a [12] gives Eo/6J x - 1.31 f 0.01 for d = 2 and Eo/SJ x sudden drop of energy occurs. This is more apparent for the - 1.7 f0.03 for d = 3, where GJis the width of the Gaussian fJ model because the energy is discrete. Figure 2 compares distribution for Jij. Other numerical methods have also been approaches to the ground state as a function of time using used to obtain the ground state energies but usually yield both Creutz and Metropolis Monte Carlo methods for the 2d values higher than those quoted above, illustrating the diffi- Gaussian model. The same bond configuration (J,’) is used culty of reaching the true ground state. For example, simulated for the two simulations. Grest et al. [4] have shown that the annealing using Metropolis Monte Carlo gives, for the 2d ground state energies obtained by Metropolis Monte Carlo Gaussian spin glass [4], Eo/SJ x - 1.2867. Grest et al. then method depend on the cooling rate, thus we used the same extrapolate to the limit of infinitely slow cooling rate [4] to get parameters which gave the best results in their simulation. AS a value of - 1.308. shown in Fig. 2, our method gives a faster convergence to the

Physica Scripta T38 42 Hong Guo et al.

models. For the 3dmodels, lower energy states are also found using our method. 1 Gaussian model 2.2. The traveling salesman problem I+,...... model ...... I 1 The TSP has been studied for a long time as the representa- -0.50 1 tive problem for combinatorial optimizations [ 131. The TSP h tries to find the shortest path connecting N nodes (cities), with 2 the path ending at its starting node. Since this belongs to the 0 class of NP-complete problems, exact solutions have been 9 attempted only for cases with less than a few hundred nodes. -1.00 However, simulated annealing has been applied to this problem where up to several thousand nodes are present [2]. We have applied our algorithm to TSP. We first consider a system with N = 400 nodes randomly distributed on a *-*-.- ...... square of linear size fi = 20. The cost function, L, is the

I' length of the path connecting the N nodes. The length is -1.50 " ' ' I'' ' ' ' ' ' 0 2500 SO00 7500 measured in the "Manhattan" metric where the distance time between two nodes is the sum of their separations along the Cartesian coordinates. We number the nodes from 1 to N Fig. 1. Typical runs showing the approach to the ground state of the 2d and a path is represented as a particular permutation of models. Energy is measured in SJ for the Gaussian model and J for the f J { 1 . . . N}.The rearrangements of the path are done using the model. The time is measured in Monte Carlo steps. System sizes are 10O2 for strategy of Lin [14] where moves consist of reversal and the Gaussian model and 5d for the & J model. replacement [ 151. We used an annealing schedule in which the maximum of ground state, in addition to the better values obtained. Our the demon cost function Lystarts at fi/4 and is then results also show little variation of the energy, since in micro- usually lowered in equal steps. At most 45 Ly's are used in canonical Monte Carlo the total energy is a conserved quantity a simulation. A Monte Carlo step here corresponds to a and Ed has an upper bound. reconfiguration trial on the path, and we used up to 160000 In all of our simulations we found that the choice of erax (400N) Monte Carlo steps for each annealing stage. How- is rather robust. Provided we start with some large value and ever, if there are too many succussful reconfigurations, it goes down with reasonable steps, the final results are not either means Lyis too large or the system is too "hot", and significantly affected. Of course, careful design of the anneal- thus we can go to the next annealing stage with a lower value ing schedule could in principle improve the rate of convergence of Ly.In our simulation, if 4000 (10N) successful recon- to the ground state. The numbers quoted above are for figurations are made, we jump to the next annealing stage typical runs with a natural annealing schedule. Another directly. A typical measure of the TSP solution, denoted by advantage or our algorithm is that no extrapolation [4] is a, is the total path length divided by N. a will be independent needed to get the values for true ground state for the 2d of N if many runs are averaged [16]. For N = 400, and the nodes uniformly distributed, our algorithm gives a sz 0.98 f 0.03 by averaging 10 independent runs. We obtain the same value of a using the standard simulated annealing with Metropolis Monte Carlo, but more than twice the CPU time was needed. When using 40000 (100N) Monte Carlo steps per annealing stage, there is a slight increase in the value ------____. of a [17]. -0.50 Figure 3 shows a typical solution where 400 nodes are distributed in 9 equal size regions separated by empty space. After 15 stages of annealing, long paths acrossing the regions are infrequent while in each region the paths are still rather disordered (see Fig. 3(a)). Fig. 3b is the result after 45 stages of annealing which gives a = 0.72. Reference [2] gives a -1.00 larger value a = 0.784 for the case where the nodes are arranged similarly to ours, although it should be noted that the difference in the a's are presumably due to the details of the actual configurations.

-1.50 2.3. Restoration of corrupted binary images 0 4000 8000 In addition to the two examples discussed above, there are time many other practical problems which can be studied with this Fig. 2. Typical runs using the Creutz and Metropolis Monte Carlo methods algorithm. For example, a large class of computer vision and for the 2d Gaussian model. Energy is measured in SJ. The same bond configurations are used for the two curves. System size is 10O2. For the image interpretation problems can be described and dis- Metropolis method, the initial temperature is 1.5 and it is lowered in steps cussed within the framework of optimization theory [18, 191. of 0. I. For each temperature, 500 Monte Carlo steps per spin are performed. Forest [20] considered the following cost function for the

Physica Scripta T38 A Fast Algorithm for Simulated Annealing 43

is largest at corners), the annealing method has to be sup- plemented by other heuristics which preserve the corners [ 191. We are currently studying this problem and results will be presented elsewhere.

3. Conclusion t The main goal of this work has been to introduce a Creutz algorithm for simulated annealing. We found that this method is particularly powerful for problems involving frustration and local ordering. This was demonstrated in the short range Ising spin glass simulations. An obvious merit of the Creutz algorithm is that it is deterministic, so that programs can be made rather efficient. Although an exhaustive investigation of the parameter space was not carried out, we found that other reasonable annealing schedules give essentially the same performance as those presented in the text. This method can therefore provide an efficient approach to optimization problems where no good heuristic method is known. For problems where long range reconfigurations are needed, the performance enhancement of Creutz algorithm over Metropolis algorithm decreases. As mentioned above, we gain only a factor of less than 3 in CPU time for TSP. Improvement in this regard could be achieved by introducing multiple demons into the algorithm, so that at any instant during a simulation, many different demon energies are present to be consulted. A hybrid algorithm of combining the Creutz and Metropolis algorithm could also be useful in some instances. We have also tested another annealing method in which the total cost function, system plus demon, is held constant at each stage of the schedule. The annealing is achieved by systematically lowing the total cost function. Preliminary results show that this method performs as well as the one presented in the text. In conclusion, we have introduced a new algorithm for the simulated annealing of NP-complete problems, and shown that it provides a distinct improvement over previous methods.

Fig. 3. (a) A TSP path after 15 stages of annealing starting with LFu = 5 for 400 nodes distributed uniformly in 9 equal size regions: (b) The path after Acknowledgements 45 stages of annealing. a = 0.72 for this path. This work was supported by the Natural Sciences and Engineering Research Council of Canada, and le Fonds pour la Formation de Chercheurs et I’Aide binary image restoration problem a la Recherche de la Province du Quebec.

E = -.IC $4 - 1/2 1 Di(l + Si) In (p-’ - 1) (3) References where Si = f 1 denotes the binary values of the pixel i, I. Schwefel, H. P., Numerical Optimization of Computer Models, John 0,= f 1 are the initial binary values of the same pixel (i.e. Wiley and Sons (1981) (Edited by N. Christofides, A. Mingoui, P. Toth D, is the pixel in the corrupted image); and p is the noise and C. Sandi), Combinatorial Optimization London and New York, strength with a value between 0 and 1. The sum is over the Wiley-interscience (1979). nearest neighbor pairs of pixels. When an image is corrupted 2. Kirkpatrick, S., Gelatt, C. D. and Vecchi, M. P., Science 220, 671 (1983); Kirkpatrick, S., J. Stat. Phys. 34, 975 (1984). by white noise, it can be shown [20], that the configuration 3. Soukoulis, C. M., Levin, K. and Grest, G. S., Phys. Rev. B28, 1495 {S,)which minimizes E is an approximation to the original (1983); Reger, J. D., Binder, K. and Kinzel, W., Phys. Rev. B30,4028 uncorrupted image. The cost function (3) has the same form ( 1985). as a random field Ising model where Di is a space dependent 4. Grest, G. S., Soukoulis, C. M. and Levin, K., Phys. Rev. Lett. 56, 1148 (1986). random field. Thus we have applied our method of simulated 5. Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A. and annealing to determine the global minimum of (3) and fine Teller, E., J. Chem. Phys. 21, 1087 (1953). that the noise is annealed away very efficiently. For an image 6. Creutz, M., Phys. Rev. Lett. 50, 1411 (1983). (a Chinese character) of 50 000 pixels corrupted with noise 7. Bhanot, G., Creutz, M. and Neuberger, H., Nucl. Phys. B235, 417 strengthp = 0.25 (25% of the pixels are corrupted), only 3% (1984); Creutz, M., Gocksch, A., Ogilvie, M. and Okawa, M., Phys. noise is left after 120 trials per pixel annealing stages with Rev. Lett. 53, 875 (1984); Harris, R., Phys. Lett. 111A, 299 (1985). (6 8. Dasgupta, C., Ma, S. K. and Hu, C. K., Phys. Rev. B20, 3837 (1979). 20 trials each). However, since corners of a picture are not 9. Sourlas, N., Europhys. Lett. 6,. 561 (1988). preserved by thermal treatment (the probability of flipping IO. Clover, M., (unpublished). Physica Scripta T38 44 Hong Guo et al.

11. Binder, K. and Young, A. P., Rev. Mod. Phys. 58, 801 (1986) and and William T. Vetterling, Numerical Recipes. Cambridge University references therein. Press, New York (1987). 12. Cheung, H. and McMillan, W. L., J. Phys. (London) C16,7027 (1983); 16. Beardwook, J., Halton, J. H. and Hammersley, J. M., Proc. Cam- Morgenstern, I. and Binder, K., Phys. Rev. Lett. 43, 1615 (1979); Z. bridge Philos. Soc. 55, 299 (1959). Phys. B39, 227 (1980); Phys. Rev. B22, 288 (1980); J. Appl. Phys. 52, 17. Our value of LY is slightly higher than those of Ref. [2] where the 1692 (1981). simulated annealing is performed after treatment by a heuristic method. 13. Dantzig, G. B., Fulkerson, D. R. and Hohnson, S. M., Oper. Res. 2, 18. Buxton, B. F. and Murray, D. W., Image and Vision Computing 3, 393 (1954); Garey, M. R. and Johnson, D. S., Compters and Intrac- 163 (1985); Buxton, B. F., Buxton, H. and Kashko, A. in Parallel tability: A Guide to the theory of NP-Completeness, (Freeman, San Architectures and Computer Vision (Edited by Ian Page), Clarendon Francisco, 1979). Press, Oxford (1988). 14. Lin, S., Bell Syst. Tech. Jour. 44, 2245 (1965). 19. Geman, S. and Geman, D., IEEE Trans. PAM1 5, 721 (1984). 15. A FORTRAN program for the traveling salesman problem is provided 20. Forrest, B. M. in Parallel Architectures and Computer Vision (Edited in the book of William H. Press, Brian P. Flannery, Saul A. Teukolsky by Ian Page), Clarendon Press, Oxford (1988).

Physica Scripta T38