From Fitness Landscape Analysis to Efficient Local Search Algorithms
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Guided Local Search
GUIDED LOCAL SEARCH Christos Voudouris, Edward P. K. Tsang and Abdullah Alsheddy Abstract Combinatorial explosion problem is a well known phenomenon that pre- vents complete algorithms from solving many real-life combinatorial optimization problems. In many situations, heuristic search methods are needed. This chapter de- scribes the principles of Guided Local Search (GLS) and Fast Local Search (FLS) and surveys their applications. GLS is a penalty-based meta-heuristic algorithm that sits on top of other local search algorithms, with the aim to improve their efficiency and robustness. FLS is a way of reducing the size of the neighbourhood to improve the efficiency of local search. The chapter also provides guidance for implement- ing and using GLS and FLS. Four problems, representative of general application categories, are examined with detailed information provided on how to build a GLS- based method in each case. Key words: Heuristic Search, Meta-Heuristics, Penalty-based Methods, Guided Local Search, Tabu Search, Constraint Satisfaction. 1 INTRODUCTION Many practical problems are NP-hard in nature, which means complete, construc- tive search is unlikely to satisfy our computational demand. For example, suppose Christos Voudouris Group Chief Technology Office, BT plc, Orion Building, mlb1/pp12, Marthlesham Heath, Ipswich, IP5 3RE, United Kingdom, e-mail: [email protected] Edward P. K. Tsang Department of Computer Science, Univeristy of Essex, Wivenhoe Park, Colchester, CO4 3SQ, United Kingdom, e-mail: [email protected] Abdullah Alsheddy Department of Computer Science, Univeristy of Essex, Wivenhoe Park, Colchester, CO4 3SQ, United Kingdom, e-mail: [email protected] 1 2 Christos Voudouris, Edward P. -
(Stochastic) Local Search Algorithms
DM841 DISCRETE OPTIMIZATION Part 2 – Heuristics (Stochastic) Local Search Algorithms Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Local Search Algorithms Basic Algorithms Outline Local Search Revisited 1. Local Search Algorithms 2. Basic Algorithms 3. Local Search Revisited Components 2 Local Search Algorithms Basic Algorithms Outline Local Search Revisited 1. Local Search Algorithms 2. Basic Algorithms 3. Local Search Revisited Components 3 Local Search Algorithms Basic Algorithms Local Search Algorithms Local Search Revisited Given a (combinatorial) optimization problem Π and one of its instances π: 1. search space S(π) I specified by the definition of (finite domain, integer) variables and their values handling implicit constraints I all together they determine the representation of candidate solutions I common solution representations are discrete structures such as: sequences, permutations, partitions, graphs (e.g., for SAT: array, sequence of truth assignments to propositional variables) Note: solution set S 0(π) ⊆ S(π) (e.g., for SAT: models of given formula) 4 Local Search Algorithms Basic Algorithms Local Search Algorithms (cntd) Local Search Revisited 2. evaluation function fπ : S(π) ! R I it handles the soft constraints and the objective function (e.g., for SAT: number of false clauses) S(π) 3. neighborhood function, Nπ : S ! 2 I defines for each solution s 2 S(π) a set of solutions N(s) ⊆ S(π) that are in some sense close to s. (e.g., for SAT: neighboring variable assignments differ in the truth value of exactly one variable) 5 Local Search Algorithms Basic Algorithms Local Search Algorithms (cntd) Local Search Revisited Further components [according to [HS]] 4. -
Appendix H: MOEA Software Availability
H MOEA Software Availability H.1 Introduction This appendix describes some of the main public-domain MOEA software that is currently available. This description includes the following information: Name: Name of the software system Description: Any relevant information about the system Environment: Software environment for which the software is intended Language: Programming language in which the software is written Availability: Software availability Original sources of each of the systems analyzed are cited, but readers should be aware of the fact that many of these systems are also mirrored at the EMOO repository located at (see under Software): http://delta.cs.cinvestav.mx/~ccoello/EMOO/ with a mirror at: http://www.lania.mx/~ccoello/EMOO Note that although commercial software for multi-objective optimization also exists (e.g., iSIGHT, which is briefly described in Chapter 5), such systems are not included here, since we limited the contents of this appendix to public- domain software. 678 H MOEA Software Availability Table H.1: MOEA Software Name Description Environment Availability Metaheuristic Includes imple- Any platform Antonio J. Nebro Algorithms in mentations in supporting Java ([email protected]), or download Java (JMetal) Java of: NSGA-II at: http://neo.lcc.uma.es/metal/ [358], SPEA2 index.html [1712], PAES [858], OMOPSO [1546], AbYSS [1137], MOCell [1136]. It also includes several test functions Multiple- Includes: Pareto Linux/Unix; Andrzej Jaszkiewicz Objective Simulated An- Standard C++ ([email protected]) MetaHeuristics -
Exploring High-Order Neighborhoods by Pattern Mining and Injection
PILS: Exploring high-order neighborhoods by pattern mining and injection Florian Arnolda, ´Italo Santanab, Kenneth S¨orensena, Thibaut Vidalb∗ a University of Antwerp, Department of Engineering Management, Belgium fflorian.arnold,[email protected] bDepartamento de Inform´atica,Pontif´ıciaUniversidade Cat´olicado Rio de Janeiro (PUC-Rio) fisantana,[email protected] Abstract. We introduce pattern injection local search (PILS), an optimization strategy that uses pattern mining to explore high-order local-search neighborhoods, and illustrate its application on the vehicle routing problem. PILS operates by storing a limited number of frequent patterns from elite solutions. During the local search, each pattern is used to define one move in which 1) incompatible edges are disconnected, 2) the edges defined by the pattern are reconnected, and 3) the remaining solution fragments are optimally reconnected. Each such move is accepted only in case of solution improvement. As visible in our experiments, this strategy results in a new paradigm of local search, which complements and enhances classical search approaches in a controllable amount of computational time. We demonstrate that PILS identifies useful high-order moves (e.g., 9-opt and 10-opt) which would otherwise not be found by enumeration, and that it significantly improves the performance of state-of-the-art population-based and neighborhood-centered metaheuristics. Keywords. Local search, Pattern mining, Combinatorial optimization, Vehicle routing problem ∗ Corresponding author 1. Introduction Since the \no free lunch" theorem [46], it is known that no method can | on average | perform arXiv:1912.11462v1 [cs.AI] 24 Dec 2019 better than any other method on all possible problems. -
Metaheuristic Optimization Frameworks: a Survey and Benchmarking
Soft Comput DOI 10.1007/s00500-011-0754-8 ORIGINAL PAPER Metaheuristic optimization frameworks: a survey and benchmarking Jose´ Antonio Parejo • Antonio Ruiz-Corte´s • Sebastia´n Lozano • Pablo Fernandez Ó Springer-Verlag 2011 Abstract This paper performs an unprecedented com- and affordable time and cost. However, heuristics are parative study of Metaheuristic optimization frameworks. usually based on specific characteristics of the problem at As criteria for comparison a set of 271 features grouped in hand, which makes their design and development a com- 30 characteristics and 6 areas has been selected. These plex task. In order to solve this drawback, metaheuristics features include the different metaheuristic techniques appear as a significant advance (Glover 1977); they are covered, mechanisms for solution encoding, constraint problem-agnostic algorithms that can be adapted to incor- handling, neighborhood specification, hybridization, par- porate the problem-specific knowledge. Metaheuristics allel and distributed computation, software engineering have been remarkably developed in recent decades (Voß best practices, documentation and user interface, etc. A 2001), becoming popular and being applied to many metric has been defined for each feature so that the scores problems in diverse areas (Glover and Kochenberger 2002; obtained by a framework are averaged within each group of Back et al. 1997). However, when new are considered, features, leading to a final average score for each frame- metaheuristics should be implemented and tested, implying work. Out of 33 frameworks ten have been selected from costs and risks. the literature using well-defined filtering criteria, and the As a solution, object-oriented paradigm has become a results of the comparison are analyzed with the aim of successful mechanism used to ease the burden of applica- identifying improvement areas and gaps in specific tion development and particularly, on adapting a given frameworks and the whole set. -
On the Design of (Meta)Heuristics
On the design of (meta)heuristics Tony Wauters CODeS Research group, KU Leuven, Belgium There are two types of people • Those who use heuristics • And those who don’t ☺ 2 Tony Wauters, Department of Computer Science, CODeS research group Heuristics Origin: Ancient Greek: εὑρίσκω, "find" or "discover" Oxford Dictionary: Proceeding to a solution by trial and error or by rules that are only loosely defined. Tony Wauters, Department of Computer Science, CODeS research group 3 Properties of heuristics + Fast* + Scalable* + High quality solutions* + Solve any type of problem (complex constraints, non-linear, …) - Problem specific (but usually transferable to other problems) - Cannot guarantee optimallity * if well designed 4 Tony Wauters, Department of Computer Science, CODeS research group Heuristic types • Constructive heuristics • Metaheuristics • Hybrid heuristics • Matheuristics: hybrid of Metaheuristics and Mathematical Programming • Other hybrids: Machine Learning, Constraint Programming,… • Hyperheuristics 5 Tony Wauters, Department of Computer Science, CODeS research group Constructive heuristics • Incrementally construct a solution from scratch. • Easy to understand and implement • Usually very fast • Reasonably good solutions 6 Tony Wauters, Department of Computer Science, CODeS research group Metaheuristics A Metaheuristic is a high-level problem independent algorithmic framework that provides a set of guidelines or strategies to develop heuristic optimization algorithms. Instead of reinventing the wheel when developing a heuristic -
A Novel LP-Based Local Search Technique – Fast and Quite Good –
A Novel LP-based Local Search Technique – Fast and Quite Good – Vom Fachbereich Informatik der Technische Universit¨at Darmstadt zur Erlangung des akademischen Grades eines Dr. rer. nat. genehmigte Dissertation von Herrn Alaubek Avdil (M.Sc.) aus der Ulaanbaatar, Mongolei Referent: Professor Dr. Karsten Weihe Korreferent: Professor Dr. Matthias M¨uller-Hannemann Tag der Einreichung: 04. Juni 2009 Tag der m¨undlichen Pr¨ufung: 17. Juli 2009 Darmstadt 2009 D 17 ii Acknowledgments I want to express my deep and sincere gratitude to my supervisor Prof. Karsten Weihe for enabling my doctoral study, inspiring with scientific work and motivating and providing me pleasant research and work atmosphere. Without his trust and encouragement my research would not be done and this theses would not exist. I am extremely grateful to Prof. Matthias M¨uller-Hannemann for his invaluable support and advice on my research, excellent and critical review to improve the presentation of my work, and for his friendship, and making it possible for my young family be close with me during my doctoral study in Germany. I would like to express my warm and sincere thanks to the colleagues and members of Algorithmics Group at the Department of Computer Science of Technische Universit¨at Darmstadt for their support and valuable comments on my research. My special ackno- wledgment goes to Dr. Roland Martin for his unforgettable support and encouragement during my doctoral study, and his detailed review and constructive criticism during the writing process of my dissertation. I wish to extend my warmest thanks to all people, who have helped me with my work at the Department of Computer Science at Technische Universit¨at Darmstadt. -
Coupling Ant Colony System with Local Search
UNIVERSITE´ LIBRE DE BRUXELLES Ecole´ Polytechnique de Bruxelles IRIDIA - Institut de Recherches Interdisciplinaires et de D´eveloppements en Intelligence Artificielle Coupling Ant Colony System with Local Search Luca Maria Gambardella Promoteur de Th`ese: Prof. Marco DORIGO Th`esepr´esent´eeen vue de l'obtention du titre de Docteur en Sciences de l'Ing´enieur Ann´eeacad´emique2014-2015 2 Summary In the last decades there has been a lot of interest in computational models and algorithms inspired by the observation of natural systems. Nature seems to be very good at inventing processes that have many of the qualities we would like to be able to put into artificial systems. Consider for example genetic algorithms and evolutionary computation (Holland 1975): by reproducing in a simplified way the approach taken by Nature to develop living beings, researchers were able to develop both control systems for learning autonomous agents, and robust optimiza- tion algorithms for function optimization or combinatorial optimization. We can also think of work in reinforcement learning (Kaelbling et al. 1996): again, the observation of how animals learn to accomplish some simple tasks has suggested a useful and productive set of reinforcement- based learning algorithms. There are many such examples, some already very widespread, like simulated annealing (Kirkpatrick et al. 1983), ge- netic algorithms (Holland 1975), neural networks (Rumelhart et al. 1986), and others that still exist mostly in research labs, like immune networks (Bersini & F.Varela 1993) and classifier systems (Holland & Reitman 1978). In this thesis we present another ethological approach to combinatorial optimization, which has been inspired by the observation of ant colonies: the Ant Colony System (ACS, Gambardella & Dorigo 1996; Dorigo & Gambardella 1997). -
Clever Algorithms Nature-Inspired Programming Recipes Ii
Jason Brownlee Clever Algorithms Nature-Inspired Programming Recipes ii Jason Brownlee, PhD Jason Brownlee studied Applied Science at Swinburne University in Melbourne, Australia, going on to complete a Masters in Information Technology focusing on Niching Genetic Algorithms, and a PhD in the field of Artificial Immune Systems. Jason has worked for a number of years as a Consultant and Software Engineer for a range of Corporate and Government organizations. When not writing books, Jason likes to compete in Machine Learning competitions. Cover Image © Copyright 2011 Jason Brownlee. All Reserved. Clever Algorithms: Nature-Inspired Programming Recipes © Copyright 2011 Jason Brownlee. Some Rights Reserved. Revision 2. 16 June 2012 ISBN: 978-1-4467-8506-5 This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 2.5 Australia License. The full terms of the license are located online at http://creativecommons.org/licenses/by-nc-sa/2.5/au/legalcode Webpage Source code and additional resources can be downloaded from the books companion website online at http://www.CleverAlgorithms.com Contents Foreword vii Preface ix I Background1 1 Introduction3 1.1 What is AI.........................3 1.2 Problem Domains...................... 10 1.3 Unconventional Optimization............... 13 1.4 Book Organization..................... 17 1.5 How to Read this Book.................. 20 1.6 Further Reading...................... 21 1.7 Bibliography........................ 22 II Algorithms 27 2 Stochastic Algorithms 29 2.1 Overview.......................... 29 2.2 Random Search....................... 30 2.3 Adaptive Random Search................. 34 2.4 Stochastic Hill Climbing.................. 40 2.5 Iterated Local Search.................... 44 2.6 Guided Local Search.................... 50 2.7 Variable Neighborhood Search.............. -
A Guided Tabu Search for the Vehicle Routing Problem with Two-Dimensional Loading Constraints
Available online at www.sciencedirect.com European Journal of Operational Research 195 (2009) 729–743 www.elsevier.com/locate/ejor A Guided Tabu Search for the Vehicle Routing Problem with two-dimensional loading constraints Emmanouil E. Zachariadis a, Christos D. Tarantilis b,*, Christos T. Kiranoudis a a Department of Process Analysis and Plant Design, Department of Chemical Engineering, National Technical University of Athens, Athens, Greece b Department of Management Science and Technology, Athens University of Economics and Business, Hydras Street 28, Athens 11362, Greece Received 12 January 2007; accepted 31 May 2007 Available online 21 November 2007 Abstract We present a metaheuristic methodology for the Capacitated Vehicle Routing Problem with two-dimensional loading constraints (2L- CVRP). 2L-CVRP is a generalisation of the Capacitated Vehicle Routing Problem, in which customer demand is formed by a set of two- dimensional, rectangular, weighted items. The purpose of this problem is to produce the minimum cost routes, starting and terminating at a central depot, to satisfy the customer demand. Furthermore, the transported items must be feasibly packed into the loading surfaces of the vehicles. We propose a metaheuristic algorithm which incorporates the rationale of Tabu Search and Guided Local Search. The loading aspects of the problem are tackled using a collection of packing heuristics. To accelerate the search process, we reduce the neigh- bourhoods explored, and employ a memory structure to record the loading feasibility information. Extensive experiments were con- ducted to calibrate the algorithmic parameters. The effectiveness of the proposed metaheuristic algorithm was tested on benchmark instances and led to several new best solutions. -
Ant Colony Optimization Marco Dorigo and Thomas Stützle Ant Colony Optimization
Ant Colony.qxd 6/9/04 12:15 PM Page 1 Ant Colony Optimization Marco Dorigo and Thomas Stützle Optimization Ant Colony The complex social behaviors of ants have been much studied by science, and computer scientists are now finding that these behavior patterns can provide models for solving difficult combinatorial optimization problems. The attempt to develop algorithms inspired by one aspect of ant behavior, the ability to find what computer scientists would call shortest paths, has become the field of ant colony optimization (ACO), the most successful and widely recognized algorithmic Ant Colony technique based on ant behavior. This book presents an overview of this rapidly growing field, from its theoretical inception to practical applications, including descriptions of many available ACO algorithms and their uses. The book first describes the translation of observed ant behavior into working optimization algorithms. The ant Optimization colony metaheuristic is then introduced and viewed in the general context of combinatorial optimization. This is followed by a detailed description and guide to all major ACO algorithms and a report on current theoretical findings. The book surveys ACO applications now in use, including routing, assignment, scheduling, subset, machine learning, and bioin- formatics problems. AntNet, an ACO algorithm designed for the network routing problem, is described in detail. The authors conclude by summarizing the progress in the field and outlining future research directions. Each chapter ends Marco Dorigo and Thomas Stützle with bibliographic material, bullet points setting out important ideas covered in the chapter, and exercises. Ant Colony Optimization will be of interest to academic and industry researchers, graduate students, and practitioners who wish to learn how to implement ACO algorithms. -
Paradiseo-MO: from Fitness Landscape Analysis to Efficient
ParadisEO-MO: From Fitness Landscape Analysis to Efficient Local Search Algorithms Jérémie Humeau, Arnaud Liefooghe, El-Ghazali Talbi, Sébastien Verel To cite this version: Jérémie Humeau, Arnaud Liefooghe, El-Ghazali Talbi, Sébastien Verel. ParadisEO-MO: From Fitness Landscape Analysis to Efficient Local Search Algorithms. [Research Report] RR-7871, 2012. hal- 00665421v1 HAL Id: hal-00665421 https://hal.inria.fr/hal-00665421v1 Submitted on 1 Feb 2012 (v1), last revised 4 Jun 2013 (v2) HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. ParadisEO-MO: From Fitness Landscape Analysis to Efficient Local Search Algorithms Jérémie Humeau, Arnaud Liefooghe, El-Ghazali Talbi, Sébastien Verel RESEARCH REPORT N° 7871 February 2012 Project-Team Dolphin ISSN 0249-6399 ISRN INRIA/RR--7871--FR+ENG ParadisEO-MO: From Fitness Landscape Analysis to Efficient Local Search Algorithms Jérémie Humeau∗, Arnaud Liefooghe†, El-Ghazali Talbi‡, Sébastien Verel§ Project-Team Dolphin Research Report n° 7871 — February 2012 — 34 pages Abstract: This document presents a general-purpose software framework dedicated to the design, the analysis and the implementation of local search algorithms: ParadisEO-MO. A sub- stantial number of single-solution based local search metaheuristics has been proposed so far, and an attempt of unifying existing approaches is here presented.