Iowa State University Capstones, Theses and Retrospective Theses and Dissertations Dissertations 2008 Improving solution characteristics of particle swarm optimization through the use of digital pheromones, parallelization, and graphical processing units (GPUs) Vijay Kiran Kalivarapu Iowa State University Follow this and additional works at: https://lib.dr.iastate.edu/rtd Part of the Mechanical Engineering Commons Recommended Citation Kalivarapu, Vijay Kiran, "Improving solution characteristics of particle swarm optimization through the use of digital pheromones, parallelization, and graphical processing units (GPUs)" (2008). Retrospective Theses and Dissertations. 15700. https://lib.dr.iastate.edu/rtd/15700 This Dissertation is brought to you for free and open access by the Iowa State University Capstones, Theses and Dissertations at Iowa State University Digital Repository. It has been accepted for inclusion in Retrospective Theses and Dissertations by an authorized administrator of Iowa State University Digital Repository. For more information, please contact [email protected]. Improving solution characteristics of particle swarm optimization through the use of digital pheromones, parallelization, and graphical processing units (GPUs) by Vijay Kiran Kalivarapu A dissertation submitted to the graduate faculty in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Co-majors: Mechanical Engineering; Human Computer Interaction Program of Study Committee: Eliot Winer (Co-major Professor) Adin Mann James Oliver Judy Vance Julie Dickerson Iowa State University Ames, Iowa 2008 Copyright © Vijay Kiran Kalivarapu, 2008. All rights reserved 3316222 3316222 2008 ii To mom, sis and my wife Kavita iii TABLE OF CONTENTS List of Figures .................................................................................................... vi List of Tables ................................................................................................... viii Acknowledgements ........................................................................................... ix Abstract .............................................................................................................. xi 1 Introduction .................................................................................................. 1 1.1 Formulation of Optimization Problems ...................................................................... 1 1.2 Classification of Optimization Problems ................................................................... 5 1.3 Numerical and Evolutionary Methods ....................................................................... 7 1.4 Genetic Algorithms .................................................................................................. 11 1.5 Simulated Annealing ................................................................................................ 15 1.6 Particle Swarm Optimization ................................................................................... 18 2 Background................................................................................................. 22 2.1 Particle Swarm Optimization ................................................................................... 22 2.2 Digital Pheromones .................................................................................................. 23 2.3 Parallelization ........................................................................................................... 24 2.4 Computations Using Graphics Hardware ................................................................. 28 2.5 Research Issues ........................................................................................................ 33 3 Digital Pheromones in PSO ....................................................................... 36 3.1 Rationale ................................................................................................................... 36 3.2 Method Overview ..................................................................................................... 38 3.3 Digital Pheromone Initialization and Merging Process ........................................... 39 3.4 Proximity Analysis to Determine Target Pheromone .............................................. 42 3.5 Velocity Vector Update ............................................................................................ 43 3.6 Geometric Interpretation of Target Pheromone and Confidence Parameter, c3 ....... 44 3.7 Move Limits ............................................................................................................. 46 3.8 Statistical Significance of Digital Pheromones ........................................................ 47 3.8.1 Statistical Hypothesis Testing ........................................................................... 47 3.8.2 Hypothesis Testing Procedure .......................................................................... 49 3.9 Further Improvements .............................................................................................. 52 4 Parallelization on Computer Clusters...................................................... 53 4.1 Rationale for Parallelization ..................................................................................... 53 4.2 Synchronous Coarse Grain Parallelization ............................................................... 54 4.3 Shared Pheromone Parallelization ........................................................................... 57 5 Parallelization on Commodity Graphics Hardware ............................... 61 5.1 GPU Parallelization .................................................................................................. 61 5.2 Choice of GLSL as Shading Language .................................................................... 64 5.3 Vertex and Fragment Shaders .................................................................................. 65 5.4 Formulation for GPU Computations ........................................................................ 65 5.5 GPU Implementation................................................................................................ 67 5.6 Percentage of GPU Vs CPU Usage .......................................................................... 70 5.7 Implementation Specifics ......................................................................................... 70 iv 6 Constrained Optimization ......................................................................... 72 6.1 Methods to Solve Constrained Problems ................................................................. 72 6.1.1 Exterior Penalty Function Method (EPF) ......................................................... 75 6.1.2 Interior Penalty Function Method (IPF) ........................................................... 77 6.2 Augmented Lagrange Multiplier (ALM) Method .................................................... 78 7 Results and Discussion ............................................................................... 84 7.1 Overview .................................................................................................................. 84 7.2 Test Problem Description ......................................................................................... 85 7.2.1 Six-hump Camelback 2D function .................................................................... 85 7.2.2 Himmelblau 2D function .................................................................................. 86 7.2.3 Rosenbrock 5D function ................................................................................... 87 7.2.4 Ackley’s 10D Path Function ............................................................................. 88 7.2.5 Dixon and Price 15D function .......................................................................... 89 7.2.6 Ackley’s 20D Path Function ............................................................................. 89 7.2.7 Levy 25D Function ........................................................................................... 90 7.2.8 Sum of Squares 30D Function .......................................................................... 90 7.2.9 Sphere 40D Function ........................................................................................ 91 7.2.10 Griewank’s 50D Function ................................................................................. 92 7.2.11 One Dimensional Two Inequality Constrained Problem .................................. 93 7.2.12 Two Dimensional Single Inequality Problem ................................................... 94 7.2.13 Two Dimensional Two Inequality Problem ...................................................... 94 7.2.14 Four Dimensional Eight Inequality Constrained Weld Beam Problem ............ 94 7.2.15 Golinski’s Speed Reducer Problem .................................................................. 96 7.2.16 Himmelblau 5D Constrained Problem .............................................................. 99 7.3 Results from Digital Pheromone Implementation in PSO ..................................... 100 7.3.1 Test Problem Settings ..................................................................................... 100 7.3.2 Results and Discussion ................................................................................... 102 7.3.3 Simulating Realistic Objective Functions ......................................................