arXiv:cond-mat/0411428v1 [cond-mat.mtrl-sci] 17 Nov 2004 † ∗ ns,yedn yai piiainpoeuefe of free parameters. procedure selection optimization mech- dynamic Bak-Sneppen a yielding opti- the anism, extremal upon The draws algorithm changes. mization succes- adaptive are smallest for species with selected dependent one sively closest fit its least and the character- value and fitness value, is fitness species a each by certain where ized a 13], model[12, corresponding ele- Bak-Sneppen individual suc- fit the applied least in been the has cessfully eliminated that progressively principle are to ments The adapted rather best environment. species species, those their adapted breeding poorly expressly most by select- than few by the to progresses against driven evolution ing selectively example, are For elements extinction. inefficient when nature, emerge most In often their system. structure complex biological the focus specialized, of make let’s highly selection To concrete, natural very more the etc.. is EO on GP of which GA, mechanism well 11], SA, underlying some like 10, with algorithms comparing 9, known competitive 8, and sententious Percus[7, by proposed and is Boettcher optimization(EO) extremal named rithm on. so ge- and 5], programming(GP)[6], 4, netic algorithms(GA)[3, the genetic is simulated 2], including annealing(SA)[1, problems algorithms, so- optimization inspired high-quality special nature NP-hard the so-called One those finding to for frequently beyond. used lutions a algorithms and is of physics function class cost in some task to encountered respect with dom lcrncades [email protected] address: Electronic lcrncades [email protected] address: Electronic nrcn er,anvlntr nprdalgo- inspired nature novel a years, recent In free- of degrees many with system of optimization The ASnmes 36.40.-c,02.60.Pn,05.65.+b numbers: PACS a problem on optimization it clusters Lennerd-Jones demonstrate the We problem: procedures. p performance optimization CEO’s othe the stochastic the parameter, and adjustable searching one global only With for optimization( responsibility extremal with of is extension co one an called as considered method, be The can problems. optimization continuous to nti ae,w xlr eea-ups ersi algor heuristic general-purpose a explore we paper, this In .INTRODUCTION I. otnosetea piiainfrLnadJnsClust Lennard-Jones for optimization extremal Continuous a Zhou Tao 1 olna cec etradDprmn fMdr Physics, Modern of Department and Center Science Nonlinear 1 , ∗ nvriyo cec n ehooyo China, of Technology and Science of University nvriyo cec n ehooyo China, of Technology and Science of University e-i Bai Wen-Jie ee nu,202,P China PR 230026, Anhui, Hefei ee nu,202,P China PR 230026, Anhui, Hefei 2 Dtd coe 3 2018) 23, October (Dated: eateto Chemistry, of Department 2 ogjuCheng Long-jiu , grto o rn pngassse[4 7 9,adso and 19], 17, system[14, glass spin problem[7], Iring con- for salesman energy figuration lowest travelling finding 18], problem[17, 16], three-coloring 8, NP- graph famous finely including partitioning[7, some problems, or on optimization GA, discrete complicated or hard[15] SA more as such far algorithm, tuned some outperform often of EO[14]. probability of im- the remarkably efficiency which the reducing proved spins, by selected previously system flipping glass for pro- spin method has optimization(JEO) Ising Middleton extremal jaded example, the For posed EO. basic of formance n a einvrosfrsof forms problems, various optimization design can idiographic one at aiming course, Of k ainfo ai O In EO. basic from cation so-called hoeoeindividual one Choose individual blt function ability cettenwconfiguration new the Accept tt of state probability 1 hoea nta configuration initial an Choose (1) follows: 3 eeta tp()a oga desired. as long as Return (2) (4) step at Repeat (3) 2 vlaetefins value fitness the Evaluate (2) il set will; n if and ytmconfiguration system otelatfi n si h o.Use top. the in as is so one value fit fitness least its the to to according individual each rank omnmz h otfunction cost the minimize to hr h ytmcnit of consists system the where (2 h rvossuisidct htE loih can algorithm EO that indicate studies previous The h ffiinyo Oagrtmi estv oteprob- the to sensitive is algorithm EO of efficiency The eew osdragnrlotmzto problem, optimization general a consider we Here ≤ swt epniiiyfrlclsearching. local for responsibility with is r k . C O n scnitdo w components, two of consisted is and EO) 2 n igHn Wang Bing-Hong and , tnosetea optimization(CEO), extremal ntinuous j ≤ ( tmfrfidn ihqaiysolutions high-quality finding for ithm S oe opttv ihmr elaborate more with competitive roves S τ n epalohridvdas tt unaltered. state individuals’ other all keep and best elkoncniuu optimization continuous known well E,cnb bandtruhasih modifi- slight a through obtained be can -EO, N ) i S srn,cery h es toei frn 1. rank of is one fit least the clearly, rank, ’s P C < ), best ( := k P j P ( ,adte,ol admycag the change randomly only then, and ), ( S and S k ( k . best .Amr ffiin loih,the algorithm, efficient more A 0. = ) .I ai EO, basic In ). C S hnset then ) h Oagrtmpoed as proceeds algorithm EO The . ( j S τ htwl ecagdwt the with changed be will that best -EO, ). S f N i ′ P P 1 C o ahindividual each for unconditionally † S ( lmns n ewish we and elements, ( ( k best k S ) P oipoeteper- the improve to ) ers eedn nthe on depending ) S ∼ 1 n o any for and 1 = (1) := ftesse at system the of k k − i S τ odnt the denote to . where S > τ := i and S 0. ′ , 2 on. However, many practical problems can not be ab-
stracted to discrete form, thus to investigate EO’s ef- (a)
-181.0 ficiency on continuous optimization problems[20] is not -128.21 only of theoretic interest, but also of prominent practical -128.22
-128.23 worthiness. -181.5
-128.24
In this paper, a so-called continuous extremal op- -128.25 timization(CEO) algorithm aiming at continuous opti- -128.26 -182.0 mization problem will be introduced, which can be con- -128.27 sidered as a mixing algorithm consisting of two com- 1.0 1.2 1.4 1.6 1.8 2.0
With only one adjustable parameter, the CEO’s perfor- -183.0 mance proves competitive with more elaborate stochastic optimization procedures. We demonstrate it on a well
-183.5 known continuous optimization problem: the Lennerd- 1.0 1.2 1.4 1.6 1.8 2.0 Jones(LJ) clusters optimization problem. This paper is organized as follows: in section 2, the LJ clusters optimization problem will be briefly intro- duced. In section 3, we will give the algorithm proceeds of CEO. Next, we give the computing results about the (b) 0.40 performance of CEO on LJ clusters optimization prob- lem. Finally, in section 5, the conclusion is drawn and 0.35 the relevances of the CEO to the real-life problems are discussed.
0.30
0.25
II. LENNERD-JONES CLUSTERS 0.85 R OPTIMIZATION PROBLEM 0.80
0.20 0.75
0.70
0.65
0.60 R
0.15 Continuous optimization problem is ubiquitous in ma- 0.55
0.50
terials science: many situation involve finding the struc- 0.45
0.40 ture of clusters and the dependence of structure on size is 0.10
1.0 1.2 1.4 1.6 1.8 2.0 particularly complex and intriguing. In practice, we usu- ally choose a potential function to take the most steady 1.0 1.2 1.4 1.6 1.8 2.0 structure since it’s considered to be in possession of the minima energy. However, in all but the simplest cases, these problem are complicated due to the presence of many local minima. Such problem is encountered in FIG. 1: The details of τ-CEO for τ ∈ [1, 2]. Figure 1a shows many area of science and engineering, for example, the the average energies obtained by CEO over 200 runs, and notorious protein folding problem[21]. figure 1b exhibits the success rate of hitting the global minima As one of the simplest models that exhibits such be- in 200 runs[27]. For both figure 1a and 1b, the main plot and havior [22] one may consider the problem of finding the inset represent the case N = 40 and N = 30 respectively. One ground-state structure of nanocluster of atoms interact- can find that, the best τ corresponding lowest average energy ing through a classical Lennerd-Jones pair potential, in and highest success rate is approximate to 1.5. reduced units given by
1 1 and the total energy for N atoms is V (r)= 12 6 (1) r − r E E = X i (3) where r is the distance between two atoms. This po- i 6 tential has a single minimum at r = √2, which is the The optimization task is to find the configuration with equilibrium distance of two atoms. It can, of course, eas- minimum total potential energy of a system of N atoms, ily be reduced to an arbitrary LJ-potential by a simple each pair interacting by potential of the form (1). Clearly, rescaling of length and energy units. The ith atom has a trivial lower bound for the total energy is N(N 1)/2, energy obtained when one assumes that all pairs− are at− their equilibrium separation. For N =2, 3, 4 the lower bound 1 E = V(r ) (2) can actually be obtained in three-dimensional space, cor- i 2 X ij j6=i responding respectively to a dimer, equilateral triangle, 3 and regular tetrahedron, with all interatomic distance
equal to 1. However, from N = 5 onwards it is not possi- (a) ble to place all the atoms simultaneously at the potential 0 minimum of all others and the ground-state energy is strictly larger than the trivial lower bound. This system -100 has been studied intensely [23] and is known to have an exponential increasing number of local minima, growing -200 2 roughly as e0.36N+0.03N near N = 13, at which point
-300 there are already at least 988 minima [23]. If this scaling E continues, more than 10140 local minima exist when N -400 approach 100.
-500
The global minimum enengy
The average energy obtained by CEO
III. CONTINUOUS EXTREMAL -600
OPTIMIZATION 0 20 40 60 80 100
N The continuous extremal optimization algorithm is consisted of two components, one is the classical EO al- gorithm with responsibility for global searching, and the (b) other is a certain local searching algorithm. We give the general form of CEO algorithm by way of the LJ clusters 1.0 optimization problem as follows: 0.10
0.08 (1) Choose an initial state of the system, where all 0.8
the atoms are placed within a spherical container with 0.06 R
radius[24, 25] 0.04
0.6
1 3N 0.02 1/3 R
0.00 radius = re[ + ( ) ], (4) 0.4 2 4π√2 50 60 70 80 90 100
N 6 0.2 where re = √2 is the equilibrium distance and N denotes the number of atoms. Set the minimal energy Emin = 0.
(2) Use a certain local searching algorithm to find the 0.0 local minimum from the current configuration of system. If the local minimal energy is lower than Emin, then re- 0 20 40 60 80 100 place Emin by the present local minimum. N (3) Rank each atom according to its energy obtained by Equ.(2). Here, the atom who has highest energy is the least fit one and is arranged in the top of the queue. FIG. 2: The performance of CEO algorithm on LJ clusters Choose one atom j that will be changed with the prob- optimization problem. In figure 2a, the red circles represent ability P (kj ) where kj denotes the rank of atom j, and the average energies obtained by CEO over 200 runs, where then, only randomly change the coordinates of j and keep the black squares represent the global minima. Figure 2b all other atoms’ positions unaltered. Accept the new con- shows the success rate of hitting the global minima in 200 figuration unconditionally. runs, the inset is the success rate for N > 50 that may be (4) Repeat at step (2) as long as desired. unclear in the main plot. (5) Return the minimal energy Emin and the correspond- ing configuration. For an idiographic problem, one can attempt various the previous M steps. The number M determines the local searching algorithms and pitch on the best one. amount of storage required by the routine, which is spec- ified by the user, usually 3 M 7 and in our compu- In this paper, for the LJ clusters optimization problem, ≤ ≤ we choose limited memory BFGS method(LBFGS) qua tation M is fixed as 4. the local searching algorithm. The BFGS method is an optimization technique based on quasi-Newton method proposed by Broyden, Fletcher, Goldfard and Shanno. IV. COMPUTING RESULTS LBFGS proposed by Liu and Nocedal[25, 26] is espe- cially effective on problems involving a large number of Similar to τ-EO, we use τ-CEO algorithm for the LJ clusters optimization problem, where the probability variables. In this method, an approximation Hk to the 2 inverse of the Hessian is obtained by applying M BFGS function of CEO is P (k) k−τ . Since there are N pairs ∼ 2 updates to a diagonal matrix H0, using information from of interactional atoms in a LJ cluster of size N, we require 4
by CEO over 200 runs, where the black squares repre- sent the global minima. One can see that the deviation from global minimum becomes greater and greater when
100000 the cluster size getting larger and larger, which indicates that for very large LJ cluster, CEO may be a poor al- gorithm. Figure 2b shows the success rate of hitting the 10000 global minima in 200 runs, the inset is the success rate for N > 50 that may be unclear in the main plot. For
1000 both the case N = 95 and N = 100, the global optimal solutions appears only once in 200 runs. CPU Time (ms)CPU
Slope=3.886 Although CEO is not a all-powerful algorithm and it 100 may perform poorly for very large LJ clusters, we demon- strate that it is competitive or even superior over some
10 100 more elaborate stochastic optimization procedures like
N SA[28], GA[29] in finding the most stable structure of LJ clusters with minimal energy. Finally, we investigate the average CPU time over FIG. 3: The average CPU time(ms) over 200 runs va the size 200 runs vs the size of LJ clusters. The computations of LJ clusters. In the log-log plot, the data can be well fitted were carried out in a single PentiumIII processor(1GHZ). by a straight line with slope 3.886 ± 0.008, which indicates From figure 3, in the log-log plot, the data can be well fit- that the increasing tendency of CPU time T vs cluster size is ted by a straight line with slope 3.326 0.008, which indi- 3.886 approximate to power-law form as T ∼ N . cates that the increasing tendency of CPU± time T vs clus- ter size is approximate to power-law form as T N 3.886. That means the CEO is a polynomial algorithm∼ of order 4 αN 2 updates where α is a constant and fixed as 100 in O(N ). the following computation. In order to avoid falling into the same local minimum too many times, before run- ning LBFGS, we should make the system configuration V. CONCLUSION AND DISCUSSION far away from last local minimum, thus we run LBFGS every 20 time steps. That is to say, for a LJ cluster of In this paper, we explore a general-purpose heuris- size N, the present algorithm runs EO 100N 2 times and 2 tic algorithm for finding high-quality solutions to con- LBFGS 5N times in total. tinuous optimization problems. The computing results We have carried out the τ-CEO algorithm so many indicate that this simple approach is competitive and times for different τ and N, and find that the algorithm sometimes can outperform some far more complicated performs better when τ is in the interval [1,2]. In figure 1, or finely tuned nature inspired algorithm including sim- we report the details for 1 τ 2, where figure 1a shows ulated annealing and genetic algorithm, on a well-known ≤ ≤ the average energies obtained by CEO over 200 runs, and NP-hard continuous optimization problem for LJ clus- figure 1b exhibits the success rate R of hitting the global ters(see reference[28, 29] for comparison). According to minima[27]. For both figure 1a and 1b, the main plot and EO’s updating rule, it is clear that EO has very high abil- inset represent the case N = 40 and N = 30 respectively. ity in global searching, thus to combine EO and a strong The readers should note that, although the difference of local searching algorithm may produce a high efficient average energies between two different τ is great in the algorithm for continuous optimization problems. plot, it is very very small in fact. For the case N = 40, Recently, several novel algorithms aiming at LJ clus- the best value of τ is τ = 1.6 corresponding the lowest ters optimization problem have been proposed, such average energy and highest success rate, however, the as fast annealing evolutionary algorithm[25], conforma- performance of CEO for τ =1.5 is almost the same as τ = tional space annealing method[30], adaptive immune 1.6 in this case but obviously better than τ =1.6 in the optimization algorithm[31], cluster similarity checking case N = 30. Therefore, in the following computation, method[32], and so forth. These algorithms consider we set τ =1.5. We have also compared the performance more about the special information about LJ clusters of CEO on larger LJ clusters for τ =1.5 and τ =1.6, the and perform better than CEO. However, we have not two cases are pretty much the same thing and τ =1.5 is found a compellent evidence indicating that there exists a a little better. general-purpose algorithm like SA or GA entirely prepon- We demonstrate that for all the LJ clusters of size N derate over CEO on LJ cluster optimization problem. It not more than 100, the global minima can be obtained is worthwhile to emphasize that, in this paper, we do not by using CEO algorithm. In figure 2, we report the want to prove that the CEO is an all-powerful algorithm, performance of CEO on LJ clusters optimization prob- even do not want to say that the CEO is a good choice lem according to 200 independent runs. In figure 2a, for chemists on LJ cluster optimization problem since a the red circles represent the average energies obtained general-purpose method often perform poorer than some 5 special methods aiming at an idiographic problem. The (NNSFC) under Grant No. 20325517, and the Teaching only thing we want to say is the CEO, an extension of and Research Award Program for Outstanding Young nature inspired algorithm EO, is a competitive algorithm Teacher (TRAPOYT) in higher education institutions of and needs more attention. the Ministry of Education (MOE) of China, the State Further more, to demonstrate the efficiency of CEO, Key Development Programme of Basic Research of China much more experiments on various hard continuous op- (973 Project), the NNSFC under Grant No. 10472116, timization problems should be achieved. 70471033 and 70271070, and the Specialized Research Fund for the Doctoral Program of Higher Education (SRFDP No.20020358009). Acknowledgments
This work is supported by the outstanding youth fund from the National Natural Science Foundation of China
[1] S. Kirkpatrick, C. D. Gelatt and M. P. Vecchi, Science 066703(2004). 220, 671(1983). [19] S. Boettcher and M. Grigni, J. Phys. A 35, 1109(2002). [2] E. H. L. Aarts and J. H. M. Korst, Simulated annealing [20] In discrete optimization problem, the set of all the states and Boltzmann machines (John Wiley and Sons, 1989). of system is denumerable, while in continuous optimiza- [3] J. Holland, Adaption in natural and artificial systems tion problem, the corresponding set is a continuum. (The university of Michigan Press, Ann Arbor, MI, 1975). [21] T. P. Martin, T.Bergmann, H.G¨ohilich and T. Lange, [4] D. G. Bounds, Nature 329, 215(1987). Chem. Phys. Lett. 172,209 (1990). [5] D. E. Goldberg, Genetic algorithms in search, optimiza- [22] L. T. Wille, in: D. Stauffer(Ed.), Annual Reviewsof tion, and machine learning (Addison-Wesly, MA, 1989). Computational Physics VII, World Scientific, Singna- [6] W. Banzhaf, P. Nordin, R. Keller and F. Francone, Ge- pore, 2000. netic programming - an introduction (Morgan Kaufmann, [23] M. R. Hoare, Advan. Chem. Phys. 40 (1979) 49, and San Francisco, CA, 1998). references therein. [7] S. Boettcher and A. G. Percus, Artificial Intelligence 199, [24] B. Xia, W. Cai, X. Shao, Q. Guo, B. Maigret and Z. Pan, 275(2000). J. Mol. Struct.(Theochem) 546, 33(2001). [8] S. Boettcher, J. Phys. A 32, 5201(1999). [25] W. Cai, Y. Feng, X. Shao and Z. Pan, J. Mol. [9] S. Boettcher, Comput. Sci. Eng. 2, 6(2000). Struct.(Theochem) 579, 229(2002). [10] S. Boettcher, Comput. Sci. Eng. 2, 75(2000). [26] D. C. Liu and J. Nocedal, Math. Progm. B 45, 503(1989). [11] S. Boettcher, A. G. Percus and M. Grigni, Lect. Notes. [27] The complete up-to-date list of the global min- Comput. Sci. 1917, 447(2000). ima of LJ clusters can be download from the web [12] P. Bak and K. Sneppen, Phys. Rev. Lett. 71, 4083(1993). http://brian.ch.cam.ac.uk/. [13] K. Sneppen, P. Bak, H. Flyvbjerg and M. H. Jensen, [28] L. T. Wille, Computational Materials Science 17, Proc. Natl. Acad. Sci. 92, 5209(1995). 551(2000). [14] A. A. Middleton, Phys. Rev. E 69, 055701(R)(2004). [29] D. M. Deaven, N. Tit, J. R. Morris and K. M. Ho, Chem. [15] M. R. Garey and D. S. Johnson, Computers and in- Phys. Lett. 256, 195(1996). tractability: a guide to the theory of NP-completeness [30] J. Lee, I. -H. Lee and J. Lee, Phys. Rev. Lett. 91, (Freeman, New York, 1979). 080201(2003). [16] S. Boettcher and A. G. Percus, Phys. Rev. E 64, [31] X. Shao, L. Cheng and W. Cai, J. Chem. Phys. 120, 026114(2001). 11401(2004). [17] S. Boettcher and A. G. Percus, Phys. Rev. Lett. 86, [32] L. Cheng, W. Cai and X. Shao, Chem. Phys. Lett. 389, 5211(2001). 309(2004). [18] S. Boettcher and A. G. Percus, Phys. Rev. E 69,