Improving Selection Methods for Evolutionary Algorithms by Clustering
Total Page:16
File Type:pdf, Size:1020Kb
Improving Selection Methods for Evolutionary Algorithms by Clustering Kaikuo Xu1, ChangjieTang1, Yintian Liu1, Chuan Li1, Jiang Wu1, Jun Zhu2, Li Dai2 1School of Computer Science, Sichuan University, Chengdu, 610065, China 2National Center for Birth Defects Monitoring, Chengdu, 610065, China {xukaikuo, tangchangjie}@cs.scu.edu.cn Abstract characteristics of the population, hence the performance is still far from practice requirement. This study applies clustering in population selection to In this work, we apply clustering to selection improve the efficiency of evolutionary algorithms. The methods to better balance these two primary factors. (a) main contributions include: (a) Proposes a novel This selection framework is a combination of clustering selection framework that uses the number of clusters for a [7, 8] and classic selection schemes [6]. It first clusters the population as the measurement the population diversity. population according to the fitness distribution, then (b) Proposes clustering-ranking selection, an instance of carries out selection on the clusters using the classic this framework, and discusses its mathematical principle selection methods, at last selects individuals from the by PD-SP equation. (c) Gives experiments over CLPSO selected clusters according to uniform distribution; (b) (Comprehensive Learning Particle Swarm Optimization). Ranking selection is chosen to form an instance of the Experiment result shows that the proposed selection framework. The mathematical analysis shows that since method outperforms canonical exponential ranking on all the results of the clustering vary with the evolutional the sixteen-benchmark functions for both 10-D and 30-D progress, the selective pressure and population diversity problems except a function for 30-D problem. change dynamically; (c) An equation called PD-SP equation is generated to describe the relation; (d) CLPSO 1. Introduction was chosen to optimize 16 benchmark functions in experiment. We demonstrate that the genetic search under Evolutionary data mining is an amalgamation the guidance of the clustering-ranking selection yields direction of data mining and evolutionary computing better results than that of the canonical exponential inspired by biological evolution. Unlike data mining using ranking selection. evolutionary algorithms [1], it first uses existing The rest of the paper is organized as follows. Section techniques in data mining, such as trend analysis, 2 introduces the framework for selection with clustering. clustering and frequent pattern mining, to analysis the Section 3 describes an instance of this framework: data generated in the evolution process, and then utilize clustering-ranking selection method and discusses it from the discovered rule to direct the evolution [2-4]. a mathematical view. Section 4 gives experiments. The subject of evolutionary data mining, evolutionary Conclusion is given in Section 5. algorithms, have been widely researched and many methods have been proposed. Despite their different 2. A framework for selection with clustering representation and genetic operators [1], natural selection 2.1. Symbols and Terms is a common and essential part for all of them. Selection process plays a major role in evolutionary algorithms Clustering is an important human activity and it is since it determines the direction of search whereas other heuristic for our selection: in a population, there are many genetic operators propose new search points in an groups in which the fitness of the individuals within a undirected way. In the literature [5], Whitley argues that group being similar while dissimilar to those of population diversity and selective pressure is the only two individuals in other groups. Therefore, these groups can primary factors in genetic search and that the main be treated differently. problem consists in the inverse relation. A good selection Let P = {I1, I2, …, IN} be a population containing N should balance the two primary factors well. Although individuals Ii, F(Ii) be the fitness of Ii, si be a non-null many selection schemes [6] have been proposed to subset of P and S = {s1, s2, …, s2n-1} be the set of all improve the performance, they don’t adapt to the subsets. Denote the individual whose fitness is the largest in si as max(si) and the smallest one as min(si). Definition 1 (Similarity) Let Ii and Ij as two individuals. Supported partly by the 11th Five Years Key Programs for Science and The similarity between Ii and Ij is given as equation 1. min(F (I ),F (I )) Technology Development of China under grant No.2006038002003; sim(I , I ) = i j (1) Youth Foundation, Sichuan University, 06036: JS20070405506408. Jun i j max(F (Ii ),F (I j )) Zhu is the associate author. Third International Conference on Natural Computation (ICNC 2007) 0-7695-2875-9/07 $25.00 © 2007 Definition 2 (Similar) Let Ii and Ij be two individuals, σ example, roulette wheel selection may require the average be a user-specified threshold, if sim( Iii , I ) ≥ σ, then Ii is fitness of all the individuals inside a cluster. Theoretically this won’t change the time complexity of algorithm 1. said to be similar to Ij and vice versa. Definition 3 (Cluster) Let s be a non-null subset of P, if Algorithm 2 Selection with clustering min(s) is similar to max(s), then s is called a Cluster, and Input: the population P ’ max(s) is the Core of cluster s denoted as Core(s). Output: the population after selection, P 1. Clustering the population Definition 4 (Maximal Cluster) Let s be a cluster in P, 2. Call ClusterSelection (C) ’ if ∀I∈P and I∉s, I is not similar to the core of s, then s is 3. Call IndividualSelecction (C ) 4. return a Maximal Cluster in P. Procedure ClusterSelection (C) The clustering algorithm described in Algorithm 1 Procedure IndividualSelection (C’) 14. P’ ← ∅ consists of two main procedures: Sorting and Clustering. 15. for each s’∈ C’ do Sorting can be performed in O(nlog(n)) steps by standard 16. r ← random[0, |s’| ] 17. P’ = P’ ∪ s’[r] techniques. Since the population has been sorted 18. return P’ according to fitness in ascending order, we can scan the population in a fitness descending order in cluster. We According to procedure IndividualSelection, given a find max(P) from P, which is the first element we scan cluster s∈C and its selection probability ps, the probability and remove it from P. Then we find all the elements that of the individuals in s is given by formula 2. are similar to max(P), remove them from P (in lines 10 - p '= ps (i∈{1, …, |s|}) (2) 13). This generates a maximal cluster, which takes max(P) j |s| as the core. The step is repeated until P is null. Because To further illustrate this framework, exponential the population is sorted ahead, the time complexity of ranking selection is selected to carry out ClusterSelection. cluster is O(n). It requires the rank of each cluster, which is mentioned above. This new formed selection method is called Algorithm 1 Clustering the population clustering-ranking selection. Input: population P Output: a cluster set C 1. Sort P on fitness (value ascending order) 3. An instance of the selection framework 2. Call Cluster (P) 3. return 3.1. Clustering-ranking Selection (CRS) Procedure Cluster (P) //P is sorted by fitness 4. C ← ∅ 5. count ← 0 This selection consists of two steps. It first selects 6. while P ≠ ∅do clusters using exponential ranking selection and then 7. s ← ∅ 8. scan P and find I = max (P) selects individuals in the selected clusters according to the 9. s ← s∪{I} uniform distribution. Let q denote the base of exponent. 10. remove I from P Given the cluster set C generated by Algorithm 1, the 11. while ∋I’∈ P and I’ is similar to I do 12. s ← s∪{I’} probability of the clusters is given by formula 3. 13. remove I’ from P |C|−i 15. s.rank = count q = ( i∈{1, …, |C|}) (3) 16. count ++ pi |C| − 17. C ← C∪{s} q|C| j 18. for each s∈ C do ∑ j=1 19. s.rank = |C| - s.rank Algorithm 3 Clustering-ranking selection 20. C[s.rank] = s ∈ 21. return C Input: the population P and the ranking base q (0, 1] Output: the population after selection P’ 1. Clustering the population Each cluster is a set of individuals whose attributes 2. Call ClusterSelection (q,C) in the solution space are similar. Therefore the meaning of 3. Call IndividualSelecction (C’) 4. return |C|, the cardinal number of the output, is clear: it could be Procedure ClusterSelection (q,C) //C is sorted by fitness a measurement of the population diversity. |C| will 5. C’ ← ∅ vary with the evolution of the population. 6. sc0 ← 0 7. for i ← 1 to |C| do 8. sci ← sci-1 + pi //(Equation 3) 9. for i ← 1 to |C| do 2.2. Selection with clustering 10. r ← random[0, sc] 11. si’ ← sl such that sl-1 ≤ r < sl 12. C’ = C’ ∪ {si’} The framework for selection with clustering is 13. return C’ described in Algorithm 2. The process is composed of |C| − three parts, of which ClusterSelection is the key. All the The sum q|C| j normalizes the probabilities to ∑ j=1 classic selection schemes, such as tournament selection, |C| |C| − |C| − truncation selection, ranking selection, can be used to ensure that p = 1. For q|C| j = q 1 [6], we ∑ j=1 j ∑ j=1 q−1 carry out this procedure. Different schemes require different information provided by algorithm 1. For can rewrite the above equation as formula 4. Third International Conference on Natural Computation (ICNC 2007) 0-7695-2875-9/07 $25.00 © 2007 q −1 − M *A2 = |C| i ∈ MIN( ) = σ . (proof skipped) pi |C| q (i {1, …, |C|}) (4) q −1 M *A3 According to equation 4 and 5, given a population P Theorem 1 shows that the degree of the difference and its cluster set C, ∀I∈P, I ∈ s, s ∈ C, s.rank = i, the depends on σ.