Heuristics for Multi-Population Cultural Algorithm
Total Page:16
File Type:pdf, Size:1020Kb
University of Windsor Scholarship at UWindsor Electronic Theses and Dissertations Theses, Dissertations, and Major Papers 10-19-2015 Heuristics for Multi-Population Cultural Algorithm Xinyu He University of Windsor Follow this and additional works at: https://scholar.uwindsor.ca/etd Recommended Citation He, Xinyu, "Heuristics for Multi-Population Cultural Algorithm" (2015). Electronic Theses and Dissertations. 5639. https://scholar.uwindsor.ca/etd/5639 This online database contains the full-text of PhD dissertations and Masters’ theses of University of Windsor students from 1954 forward. These documents are made available for personal study and research purposes only, in accordance with the Canadian Copyright Act and the Creative Commons license—CC BY-NC-ND (Attribution, Non-Commercial, No Derivative Works). Under this license, works must always be attributed to the copyright holder (original author), cannot be used for any commercial purposes, and may not be altered. Any other use would require the permission of the copyright holder. Students may inquire about withdrawing their dissertation and/or thesis from this database. For additional inquiries, please contact the repository administrator via email ([email protected]) or by telephone at 519-253-3000ext. 3208. Heuristics for Multi-Population Cultural Algorithm by Xinyu He A Thesis Submitted to the Faculty of Graduate Studies through the School of Computer Science in Partial Fulfillment of the Requirements for the Degree of Master of Science at the University of Windsor Windsor, Ontario, Canada 2015 c 2015, Xinyu He Heuristics for Multi-Population Cultural Algorithm by Xinyu He APPROVED BY: K. Tepe, External Reader Department of Electrical and Computer Engineering D. Wu, Internal Reader School of Computer Science Z. Kobti, Advisor School of Computer Science October 15, 2015 Authors Declaration of Originality I hereby certify that I am the sole author of this thesis and that no part of this thesis has been published or submitted for publication. I certify that, to the best of my knowledge, my thesis does not infringe upon anyones copyright nor violate any proprietary rights and that any ideas, techniques, quotations, or any other material from the work of other people included in my thesis, published or otherwise, are fully acknowledged in accordance with the standard referencing practices. Furthermore, to the extent that I have included copyrighted material that surpasses the bounds of fair dealing within the meaning of the Canada Copyright Act, I certify that I have obtained a written permission from the copyright owner(s) to include such material(s) in my thesis and have included copies of such copyright clearances to my appendix. I declare that this is a true copy of my thesis, including any final revisions, as approved by my thesis committee and the Graduate Studies office, and that this thesis has not been submitted for a higher degree to any other University or Institution. iii ABSTRACT Cultural Algorithm (CA) is one of the Evolutionary Algorithms (EAs) which de- rives from the cultural evolution process in nature. As an extended version of the CA, the Multi-population Cultural Algorithm (MPCA) has multiple population spaces. Since the evolutionary information can be exchanged among the sub-populations, the MPCA can obtain better results than the CA in optimization problems. In this thesis, we introduce heuristics to improve the MPCA. The heuristic strate- gies target the existing weaknesses in MPCAs. Four strategies are developed address- ing these weaknesses, including the individual memory heuristic, the social interaction heuristic, the dynamic knowledge migration interval heuristic and the population dis- persion based knowledge migration interval heuristic.Five standard benchmark opti- mization functions with different characteristics are taken to test the efficiency of the heuristics. Simulation results show that each heuristic, to varying degrees, improves the MPCA in convergence speed, stability and precision. We compared different combinations of the strategies, and the results show that the MPCAs with social interaction based knowledge selection, as well as dynamic knowledge migration inter- val/population dispersion based knowledge migration interval, outperform the other combinations in both low-dimension functions and high-dimension functions. iv DEDICATION Dedicated to My Family and Friends v ACKNOWLEDGEMENTS I would like to express my gratitude to all those who helped me during my graduate studies. My deepest gratitude goes first and foremost to my supervisor, Dr. Ziad Kobti for his constant support and guidance all throughout my graduate studies. He has walked me through all the stages of the writing of this thesis. Without his consistent and illuminating instruction, this thesis could not have reached its present form. My heartfelt gratitude goes to Mrs. Gloria Mensah, secretary to the director, who has always helped setup meetings with my supervisor. In addition, my sincere thanks to Ms. Karen Bourdeau, who has always been there to take care of other issues related to my masters degree. I am also indebted to NSERC Discovery Grants Program for partially supporting my research. Finally I would like to thank my parents for their unconditional support and love. Xinyu He vi TABLE OF CONTENTS AUTHOR'S DECLARATION OF ORIGINALITY iii ABSTRACT iv DEDICATION v ACKNOWLEDGEMENTS vi LIST OF TABLES x LIST OF FIGURES xi 1 INTRODUCTION 1 1.1 Overview . 1 1.2 Motivation . 3 1.3 Problem Statement . 4 1.4 Organization of Thesis . 5 2 LITERATURE REVIEW 6 2.1 Evolutionary Algorithm . 6 2.1.1 Genetic Algorithm . 6 2.1.2 Particle Swarm Optimization . 8 2.1.3 Ant-colony Optimization . 9 2.1.4 Other EAs . 11 2.2 Cultural Algorithm . 13 2.3 Multi-Population Cultural Algorithm . 16 vii 3 PROPOSED ALGORITHM 19 3.1 Algorihtm Flowchart . 19 3.2 Knowledge Sources . 23 3.2.1 Normative Knowledge . 23 3.2.2 Topographic Knowledge . 26 3.2.3 Historic Knowledge . 29 3.3 Heuristic Strategies . 31 3.3.1 Strategy One (S1): Knowledge Selection Based on Individual Memory . 31 3.3.2 Strategy Two (S2): Knowledge Selection Based on Social In- teraction . 34 3.3.3 Strategy Three (S3): Dynamic Knowledge Migration Interval . 36 3.3.4 Strategy Four (S4): Population Dispersion Based Knowledge Migration Interval . 38 4 EXPERIMENT AND RESULT ANALYSIS 40 4.1 Benchmark Optimization Functions . 40 4.2 Experimental Setup . 45 4.3 Experimental Results and Analysis . 49 4.3.1 M1 vs. M2 vs. M3 . 49 4.3.2 M1 vs. M4 vs. M5 . 54 4.3.3 M1 vs. M6 vs. M7 vs. M8 vs. M9 . 55 4.4 Discussion . 59 4.5 A Real-World Application . 66 5 CONCLUSIONS AND FUTURE WORK 68 REFERENCES 70 viii VITA AUCTORIS 75 ix LIST OF TABLES 4.1 Dimensions, search ranges,and global optimum values of the test func- tions . 40 4.2 Target values of 5 test problems [1] . 47 4.3 Main parameters for M1 vs. M2 vs. M3 . 49 4.4 Test result for M1 vs. M2 vs. M3 . 50 4.5 Main parameters for M1 vs. M4 vs. M5 . 54 4.7 Main parameters for M1 vs. M6 vs. M7 vs. M8 vs. M9 . 55 4.6 Test result for M1 vs. M4 vs. M5 . 56 4.8 Test result for M1 vs. M6 vs. M7 vs. M8 vs. M9(10-dimension) . 57 4.9 Test result for M1 vs. M6 vs. M7 vs. M8 vs. M9 (50-dimension) . 58 4.10 Test result for spring design problem . 67 4.11 Main parameters for spring design problem . 67 x LIST OF FIGURES 1.1 What is culture[2] . 2 2.1 Ants find the shortest path to the food[3] . 10 2.2 CA framework[4] . 13 2.3 MPCA framework . 16 3.1 Flowchart of the proposed MPCA . 20 3.2 Solution individual . 21 3.3 Cooperate mechanism of normative knowledge (m = 2) . 26 3.4 Topographic knowledge(2-dimension) . 27 3.5 Cooperate mechanism of topographic knowledge (m = 2) . 29 3.6 The memory can affect the individual's decisions . 33 3.7 Individuals select knowledge source based on social influence . 35 3.8 Dynamic knowledge migration interval (ηmax = 50, N = 1:02) . 37 4.1 3-D image for 2-D Sphere Function[5]. 41 4.2 3-D image for 2-D Schwefel Problem 1.2 Function[5]. 42 4.3 3-D image for 2-D Ackleys Function[5]. 43 4.4 3-D image for 2-D Rastrigin Function[5]. 44 4.5 3-D image for 2-D Rosenbrock Function[5]. 45 4.6 Ring topology, mesh topology and square topology . 48 4.7 Number of times three different knowledge sources being selected by individuals for F1 . 51 4.8 Number of times three different knowledge sources being selected by individuals for F2 . 52 4.9 Number of times three different knowledge sources being selected by individuals for F3 . 52 4.10 Number of times three different knowledge sources being selected by individuals for F4 . 53 xi 4.11 Number of times three different knowledge sources being selected by individuals for F5 . 53 4.12 Convergence performance of M1,M3 and M9 for F1(10-D) . 60 4.13 Convergence performance of M1(constant knowledge migration inter- val = 1), M1(constant knowledge migration interval = 6) and M8 for F2(50-D) . 61 4.14 Convergence performance of M1,M7 and M9 for F3 (50-D) . 62 4.15 Convergence performance of M1,M2 and M3 for F4(10-D) . 64 4.16 Convergence performance of M1, M8 and M9 for F5(10-D) . 65 4.17 Spring design problem[6] . 66 xii 1 INTRODUCTION 1.1 Overview Optimization problems are crucial in Computer Science. Mathematical approaches often meet failures in solving large-scale optimization problems. In addition to the mathematical optimization, linear programming techniques and dynamic program- ming techniques were developed to search for optimal solutions.