New Approaches in Parallel Algorithm Portfolios for Metaheuristic Optimization
Total Page：16
File Type：pdf, Size：1020Kb
New Approaches in Parallel Algorithm Portfolios for Metaheuristic Optimization A Dissertation submitted to the designated by the General Assembly of Special Composition of the Department of Computer Science and Engineering Examination Committee by Dimitrios Souravlias in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY University of Ioannina June 2017 Advisory Committee • Konstantinos E. Parsopoulos, Associate Professor, Department of Computer Science & Engineering, University of Ioannina, Greece • Enrique Alba, Professor, Department of Computer Science & Programming Languages, University of Málaga, Spain • Konstantina Skouri, Associate Professor, Department of Mathematics, Univer- sity of Ioannina, Greece Examination Committee • Konstantinos E. Parsopoulos, Associate Professor, Department of Computer Science & Engineering, University of Ioannina, Greece • Enrique Alba, Professor, Department of Computer Science & Programming Languages, University of Málaga, Spain • Konstantina Skouri, Associate Professor, Department of Mathematics, Univer- sity of Ioannina, Greece • Isaac Lagaris, Professor, Department of Computer Science & Engineering, Uni- versity of Ioannina, Greece • Ilias Kotsireas, Professor, Department of Physics & Computer Science, Wilfrid Laurier University, Canada • Vassilios Dimakopoulos, Associate Professor, Department of Computer Science & Engineering, University of Ioannina, Greece • Dimitrios Papageorgiou, Associate Professor, Department of Materials Science & Engineering, University of Ioannina, Greece Dedication To my family Acknowledgements First and foremost I would like to express my deepest gratitude to my advisor, Professor Konstantinos E. Parsopoulos, for his valuable guidance and continuous support from the beginning of my PhD studies. Our excellent collaboration has given me the opportunity to acquire important knowledge and skills that have helped me in my research endeavor. Our long discussions and his constant encouragement have greatly contributed to my scientific and personal development. I would feel grateful if I have managed to acquire even the minimum of the professionalism and academic ethics that distinguish him. I would like to express my sincere thanks to the members of the Advisory Com- mittee Professor Enrique Alba and Professor Konstantina Skouri. Professor Enrique Alba and his research team at the University of Málaga have been exceptional collab- orators on solving difficult optimization problems met in the environment of smart cities. His advice and constructive comments have been of great importance. Profes- sor Konstantina Skouri and I have collaborated on modeling and solving operations research problems. Her constructive guidance has been fundamental in my work in the field of operations research. Her advice has always been invaluable. I am also very grateful to Professor Ilias Kotsireas with whom I have extensively collaborated on combinatorial optimization problems. Professor Ilias Kotsireas has al- ways supported, motivated me, and assisted me to all my research needs. I would like to thank Professor Gerasimos Meletiou for our excellent collaboration on addressing challenging optimization problems in cryptography. I would also like to thank the members of the Examination Committee, Professor Isaac Lagaris, Professor Vassilios Dimakopoulos, and Professor Dimitrios Papageor- giou for their insightful comments and critical view on my work. Many thanks to my friends Dimitris Kitsakis, Vasilis Bourgos, Katerina Pandrem- menou, Nikos Tziortziotis, Kostas Tziortziotis, Marina Drosou, Grigoris Tzortzis and Kostas Stefanidis for their unconditional support during my postgraduate studies. Also, I would like to thank my friend Vasilis Tatsis for our long discussions and the pleasant atmosphere at the lab. I would like to deeply thank my parents Nikos and Katerina and my brother Vasilis for their support. Their advice, encouragement, and endless love have always helped me move forward. Finally, a special thanks goes to my life-partner Anastasia, who has always been by my side and patiently supported me at every step. Her continuous presence in my life has been a driving force within me. Table of Contents List of Figures iv List of Tables vi List of Algorithms ix List of Abbreviations x Abstract xii Εκτεταμένη Περίληψη xv 1 Introduction 1 1.1 Overview ................................... 1 1.2 Thesis contribution .............................. 3 1.3 Thesis Layout ................................. 6 2 Metaheuristic Optimization 7 2.1 Intoduction .................................. 7 2.2 Simulated Annealing ............................. 8 2.3 Tabu Search .................................. 9 2.4 Iterated Local Search ............................. 10 2.5 Variable Neighborhood Search ....................... 12 2.6 Differential Evolution ............................ 13 2.7 Particle Swarm Optimization ........................ 16 2.8 Computational Budget Allocation ...................... 21 2.8.1 Neighborhood-based Budget Allocation .............. 21 2.8.2 Experimental Results ......................... 31 2.9 Synopsis .................................... 56 i 3 Algorithm Portfolios 57 3.1 Introduction .................................. 57 3.2 Literature review ............................... 58 3.3 Parallel Algorithm Portfolios ........................ 62 3.3.1 Standard Model ........................... 62 3.3.2 Algorithm Portfolio for Population-based Algorithms ...... 63 3.4 Application in Design of S-boxes ...................... 64 3.4.1 Problem Formulation ........................ 66 3.4.2 Implementation Details ....................... 69 3.4.3 Experimental Results ......................... 71 3.5 Application in Traffic Light Scheduling ................... 86 3.5.1 Problem Formulation ........................ 89 3.5.2 Employed algorithms ........................ 90 3.5.3 Experimental Results ......................... 92 3.6 Synopsis .................................... 99 4 New Parallel Trading-based Algorithm Portfolio 101 4.1 Introduction .................................. 101 4.2 Proposed model ............................... 102 4.3 Application in Circulant Weighing Matrices ................ 104 4.3.1 Problem Formulation ........................ 105 4.3.2 Employed Algorithms ........................ 106 4.3.3 Experimental results ......................... 108 4.4 Application in Production Scheduling ................... 112 4.4.1 Problem Formulation ........................ 113 4.4.2 Employed Algorithms ........................ 114 4.4.3 Experimental Results ......................... 116 4.5 Application in Humanitarian Logistics ................... 117 4.5.1 Problem Formulation ........................ 117 4.5.2 Employed Algorithms ........................ 120 4.5.3 Experimental Results ......................... 121 4.6 Synopsis .................................... 129 5 New Parallel Forecasting-based Algorithm Portfolio 131 5.1 Introduction .................................. 131 ii 5.2 Time Series Forecasting ........................... 132 5.3 Proposed model ............................... 133 5.4 Application in Circulant Weighing Matrices ................ 137 5.4.1 First stage of experiments: proof of concept ............ 146 5.4.2 Second stage of experiments: sensitivity analysis ......... 148 5.5 Synopsis .................................... 152 6 Concluding Remarks and Future Work 153 6.1 Concluding Remarks ............................. 153 6.2 Future work ................................. 154 Bibliography 155 A Appendix 171 A.1 Test Problems ................................. 171 A.1.1 Standard Test Suite .......................... 171 A.1.2 Nonlinear Systems .......................... 172 iii List of Figures 2.1 Neighborhood topologies: ring (left) and star (right). ........... 17 2.2 Number of wins, draws, and losses for all SOBA variants. ........ 37 2.3 Aggregate number of wins, draws, and losses for different combinations of quality criteria and selection probability in SOBA-based variants. .. 38 2.4 Aggregate number of wins, draws, and losses for pairs of variants with the same selection probability scheme (L or NL) but different parame- ters in SOBA-based variants. ........................ 40 2.5 Number of wins, draws, and losses for the MOBA-based variants LWA and DWA. ................................... 42 2.6 Aggregate number of wins, draws, and losses for different combinations of quality criteria and selection probability in LWA and DWA approaches. 43 2.7 Aggregate number of wins, draws, and losses for pairs of LWA and DWA variants with identical selection probability scheme (L or NL), but different parameters. ........................... 44 2.8 Number of wins, draws, and losses for the PFA variant of MOBA strategy. 46 2.9 Aggregate number of wins, draws, and losses for different tournament sizes in PFA variants. ............................ 47 2.10 Cumulative number of hits for different accuracy levels (linear case). .. 53 2.11 Cumulative number of hits for different accuracy levels (nonlinear case). 54 2.12 Portion of time spent on algorithmic procedures vs function evaluations. 55 3.1 Number of wins per algorithm and test problem. ............. 79 3.2 Required execution time to achieve the reported median solutions. ... 81 3.3 Nonlinearity of the best solution during the algorithms’ execution. ... 82 3.4 Autocorrelation of the best solution during the algorithms’ execution. 83 3.5 Standard deviation of running time for all algorithms and problems.