Particle Swarm Optimization
Total Page:16
File Type:pdf, Size:1020Kb
PARTICLE SWARM OPTIMIZATION Thesis Submitted to The School of Engineering of the UNIVERSITY OF DAYTON In Partial Fulfillment of the Requirements for The Degree of Master of Science in Electrical Engineering By SaiPrasanth Devarakonda UNIVERSITY OF DAYTON Dayton, Ohio May, 2012 PARTICLE SWARM OPTIMIZATION Name: Devarakonda, SaiPrasanth APPROVED BY: Raul Ordonez, Ph.D. John Loomis, Ph.D. Advisor Committee Chairman Committee Member Associate Professor Associate Professor Electrical & Computer Engineering Electrical & Computer Engineering Robert Penno, Ph.D. Committee Member Associate Professor Electrical & Computer Engineering John G. Weber, Ph.D. Tony E. Saliba, Ph.D. Associate Dean Dean, School of Engineering School of Engineering & Wilke Distinguished Professor ii ABSTRACT PARTICLE SWARM OPTIMIZATION Name: Devarakonda, SaiPrasanth University of Dayton Advisor: Dr. Raul Ordonez The particle swarm algorithm is a computational method to optimize a problem iteratively. As the neighborhood determines the sufficiency and frequency of information flow, the static and dynamic neighborhoods are discussed. The characteristics of the different methods for the selection of the algorithm for a particular problem are summarized. The performance of particle swarm optimization with dynamic neighborhood is investigated by three different methods. In the present work two more benchmark functions are tested using the algorithm. Conclusions are drawn by testing the different benchmark functions that reflect the performance of the PSO with dynamic neighborhood. And all the benchmark functions are analyzed by both Synchronous and Asynchronous PSO algorithms. iii This thesis is dedicated to my grandmother Jogi Lakshmi Narasamma. iv ACKNOWLEDGMENTS I would like to thank my advisor Dr.Raul Ordonez for being my mentor, guide and personally supporting during my graduate studies and while carrying out the thesis work and offering me excellent ideas. I also wish to express my deepest gratitude to Dr.Veysel Gazi, who along with my advisor offered me his help while working on my thesis. I would also like to thank Dr. John Loomis and Dr. Robert Penno for being the committee members. I would like to express my appreciation to my brother who has helped with my work. I would like to thank everyone in the Electrical Department for making me feel comfortable in the two and half year's of study at University of Dayton. I finally thank my family for their support and love in all activities during my time in the graduate program. v TABLE OF CONTENTS Page ABSTRACT . iii DEDICATION . iv ACKNOWLEDGMENTS . v LIST OF FIGURES . viii LIST OF TABLES . xiii CHAPTER: 1. INTRODUCTION . 1 1.1 Particle Swarm Optimization Algorithm . 2 1.1.1 Particle Swarm Optimization with Constriction Factor . 5 1.2 Hybrid Particle Swarm Optimization Algorithms . 7 1.3 Parallel and Distributed Implementation . 10 1.4 Multi Objective Optimization . 11 1.5 Stability and Convergence Analysis . 13 1.6 Application Areas . 14 1.6.1 Neural Network Training . 14 1.6.2 Dynamic Tracking . 14 1.6.3 Multi-Agent Search . 15 1.6.4 Wireless-Sensor Networks . 19 1.6.5 Optimal Design of Power Grids . 20 1.6.6 PSO for Multi User Detection in CDMA . 20 2. NEIGHBORHOOD TOPOLOGIES . 22 2.1 Static Neighborhood . 23 2.2 Dynamic Neighborhood . 25 vi 2.2.1 Nearest Neighbors in Search Space . 26 2.2.2 Nearest Neighbors in Function Space . 27 2.2.3 Random Neighborhood . 27 2.3 Synchronous and Asynchronous PSO . 29 2.3.1 Synchronous PSO . 29 2.3.2 Asynchronous PSO . 30 3. RESULTS - I . 32 4. RESULTS - II . 55 5. RESULTS - III . 77 5.1 Synchronous PSO . 77 5.2 Asynchronous PSO . 82 6. CONCLUSIONS . 90 6.1 Design Guidelines . 95 6.2 Future Work . 95 BIBLIOGRAPHY . 96 Appendices: A. MATLAB CODE FOR SYNCHRONOUS PSO ALGORITHM FOR DYNAMIC NEIGHBORHOOD FOR DEJONGF4 FUNCTION . 106 B. MATLAB CODE FOR ASYNCHRONOUS PSO ALGORITHM FOR DYNAMIC NEIGHBORHOOD FOR DEJONGF4 FUNCTION . 110 C. MATLAB CODE FOR SYNCHRONOUS PSO ALGORITHM FOR NO OF PARTICLES AS PARAMETER FOR DEJONGF4 FUNCTION . 114 D. MATLAB CODE FOR ASYNCHRONOUS PSO ALGORITHM FOR DYNAMIC NEIGHBORHOOD FOR DEJONGF4 FUNCTION . 118 vii LIST OF FIGURES Figure Page 2.1 Static Neighborhood Topologies. 24 2.2 Nearest neighbors in search space. 27 2.3 Nearest neighbors in function space. 28 3.1 Contour plots of all six benchmark functions. 33 3.2 Distance between particles in search space against average global value for a Sphere function. 35 3.3 Distance between particles in search space against average global value for a Griewank function. 35 3.4 Distance between particles in search space against average global value for Rastrigin function. 37 3.5 Distance between particles in search space against average global value for a Rosenbrock function. 37 3.6 Distance between particles in search space against average global value for a Ackley function. 38 3.7 Distance between particles in search space against average global value for a DejonF4 function. 41 viii 3.8 Distance between particles in function space against average global value for a Sphere function. 41 3.9 Distance between particles in function space against average global value for a Griewank function. 42 3.10 Distance between particles in function space against average global value for a Rastrigin function. 43 3.11 Distance between particles in function space against average global value for a Rosenbrock function. 45 3.12 Distance between particles in function space against average global value for a Ackley function. 45 3.13 Distance between particles in function space against average global value for a DejonF4 function. 48 3.14 Probability of particles being neighbors against mean global best value for a Sphere function. 48 3.15 Probability of particles being neighbors against mean global best value for a Griewank function. 49 3.16 Probability of particles being neighbors against mean global best value for a Rastrigin function. 50 3.17 Probability of particles being neighbors aganist mean global best value for a Rosenbrock function. 51 3.18 Probability of particles being neighbors against mean global best value for a Ackley function. 51 3.19 Probability of particles being neighbors against mean global best value for a DejongF4 function. 52 ix 3.20 Distance between particles in search space against average global best value for Synchronous PSO . 53 3.21 Distance between particles in search space against average global best value for Asynchronous PSO . 54 4.1 Distance between particles in search space against average global value for a Sphere function. 56 4.2 Distance between particles in search space against average global value for a Griewank function. 57 4.3 Distance between particles in search space against average global value for a Rastrigin function. 58 4.4 Distance between particles in search space against average global value for a Rosenbrock function. 59 4.5 Distance between particles in search space against average global value for a Ackley function. 61 4.6 Distance between particles in search space against average global value for a DejonF4 function. 61 4.7 Distance between particles in function space against average global value for a Sphere function. 63 4.8 Distance between particles in function space against average global value for a Griewank function. 64 4.9 Distance between particles in function space against average global value for a Rastrigin function. 65 4.10 Distance between particles in function space against average global value for a Rosenbrock function. 66 x 4.11 Distance between particles in function space against average global value for a Ackley function. 68 4.12 Distance between particles in function space against average global value for a DejonF4 function. 68 4.13 Distance between particles in Random Neighborhood against average global value for a Sphere function. 70 4.14 Distance between particles in Random Neighborhood against average global value for a Griewank function. 71 4.15 Distance between particles in random neighborhood against average global value for a Rastrigin function. 72 4.16 Distance between particles in random neighborhood against average global value for a Rosenbrock function. 73 4.17 Probability of particles being neighbors against mean global best value for a Ackley function. 73 4.18 Probability of particles being neighbors against mean global best value for a DejongF4 function. 74 4.19 Neighborhood size expressed as percentage of function space against Average global best value for Synchronous PSO . 75 4.20 Neighborhood size expressed as percentage of function space against Average global best value for Asynchronous PSO . 76 5.1 Average global best value versus No.of Neighbors for Sphere function. 79 5.2 Average global best value versus No.of Neighbors for Griewank function. 79 5.3 Average global best value versus No.of Neighbors for Rastrigin function. 81 5.4 Average global best value versus No.of Neighbors for Rosenbrock function. 81 xi 5.5 Average global best value versus No.of Neighbors for Ackley function. 82 5.6 Average global best value versus No.of Neighbors for DejongF4 function. 83 5.7 Average global best value versus No.of Neighbors for Sphere function. 84 5.8 Average global best value versus No.of Neighbors for Griewank function. 85 5.9 Average global best value versus No.of Neighbors for Rastrigin function. 86 5.10 Average global best value versus No.of Neighbors for Rosenbrock function. 87 5.11 Average global best value versus No.of Neighbors for Ackley function. 88 5.12 Average global best value versus No.of Neighbors for DejongF4 function. 89 6.1 Comparison of Synchronous PSO and Asynchronous PSO in Static Neighborhood . 93 6.2 Comparison of Synchronous PSO and Asynchronous PSO in Dynamic Neighborhood . 94 xii LIST OF TABLES Table Page 3.1 Results for Neighborhood determination based on nearest neighbors in the search space . 34 3.2 Results for Neighborhood determination based on nearest neighbors in the search space .