Metaheuristics : from Design to Implementation E-G. Talbi Professor – Polytech’Lille - Team manager –INRIA – University of Lille - CNRS – France [email protected] Wiley, 2009 (596pp) ISBN: 978-0-470-27858-1 Metaheuristics E-G. Talbi Outline Metaheuristics E-G. Talbi Outline of the book Common concepts for Metaheuristics Single-solution based metaheuristics z Common concepts for S-metaheuristics z Local search z Landscape analysis z Advanced local search (Simulated annealing, Tabu search, VNS, ILS, GLS, …) Population-based metaheuristics z Common concepts for P-metaheuristics z Evolutionary algorithms (Genetic algorithms, GP, ES, EDA, …) z Swarm inteeligence: Ant colonies, Particle swarm, … z Bess, Immune systems, … Metaheuristics for Multi-objective Optimization Hybrid Metaheuristics Parallel metaheuristics Metaheuristics E-G. Talbi Outline of the chapter 1 Optimization models Optimization methods z Exact algorithms / Approximate algorithms z When using metaheuristics? z Greedy heuristics Main common concept of metaheuristics z Representation z Objective function Constraint handling Parameter tuning Performance analysis of metaheuristics Software frameworks for metaheuristics z ParadisEO framework Metaheuristics E-G. Talbi From modeling to decision making In practice, we find solutions for models representing problems Usually models are simplifications of the reality Importance of models Metaheuristics E-G. Talbi Optimization problem Definition (minimisation problem) : couple (S,f) z Given a search space S which represents feasible solutions z Given an objective function f: S Æ R z Finds*∈ S such as : f( s *)≤ f ( s )∀s∈ S s* : global optimum z Large scale and complex optimization problems in many areas of science and industry (Telecommunications, Biology, Transportation- Logistics, Environment, Finance, Design, ...). z Problem size more and more important (combinatorial explosion) and/or Delays more and more reduced. Metaheuristics E-G. Talbi Optimization models Metaheuristics E-G. Talbi Complexity theory: Algorithms Polynomial-time algorithms: shortest paths, spanning tree, network flow, … Exponential-time algorithms Metaheuristics E-G. Talbi Complexity theory: Algorithms • Two other notations to analyze algorithms Metaheuristics E-G. Talbi Complexity theory: Problems Complexity of a problem = complexity of the best algorithm solving that problem Tractable (easy) = there exists a polynomial-time algorithm to solve it Intractable = if no polynomial-time algorithm exists Most of real-life problems are intractable Metaheuristics E-G. Talbi Decision / Optimization problems Decision problem has always « yes » or « no » answer Optimization problem can always be reduced to a decision problem Metaheuristics E-G. Talbi Complexity classes P class: decision problems solved by a deterministic machine in polynomial time NP class: decision problems solved by a non- deterministic algorithm in polynomial time NP-complete: if all problems of the class NP are reduced polynomially to the problem P=NP? Important open question Metaheuristics E-G. Talbi Complexity classes: Examples P class: z Shortest path, network flow z Spanning tree, bipartite matching, linear programming, … NP-complete: z Sequencing and scheduling problems: flow-shop, job-shop, open-shop, … z Assignment and location problems: QAP, GAP, p-median, … z Grouping problems: clustering, graph partitioning, coloring, … z Routing and covering problems: set covering, VRP, … z Knapsack, cutting and packing problems Open problems: e.g. graph isomorphism Metaheuristics E-G. Talbi Other models for optimization Optimization under uncertainty z Input data subject to noise z e.g VRP – stochastic demands, travel times, … Dynamic optimization z Deterministic objective function, varies over time z e.g. VRP – new demand Æ Track the optimal solution z Multi-periodic optimization: periodic change (change known a priori) z e.g. planning problem (traffic, incoming technology) Robust optimization: engineering design problems Multi-objective optimization problems Metaheuristics E-G. Talbi “Generic” problems Ex: SAT, TSP, and NLP are 3 canonical forms of models than can be applied to solve different problems. Quadratic Assignment Problem Bin Packing and Generalised Assignment Problems Hub allocation problems Graph Colouring & Partitioning Vehicle Routing Single & Multiple Knapsack Set Partitioning & Set Covering Problems Processor Allocation Problem Various Staff Scheduling Problems Job Shop Scheduling Metaheuristics E-G. Talbi Size of the search space SAT Problem : Boolean Satisfiability Problem (First NP-hard problem). Many applications : timetabling, routing, … Find a set of boolean variables XXX(= 1 ,...,n ) such as the boolean expression F = TRUE Boolean expression F = conjunction of clauses Clauses are disjunctions of k variables (k-SAT) : FXXXXXXX(= ) ( ∨1 3 ) ∧ ( 1 ∨ 2 ) ∧ ( 4 ∨ 2 ) ... ∧ For n=100, size of the search space = 2100 ≈ 1030, Evaluation of 1000 solutions / sec ; 15 billions of years (Big Bang) to evaluate less than 1% of the search space. k>2, NP-hard problem ; k=2, Polynomial problem Metaheuristics E-G. Talbi Size of the search space TSP Problem : n cities, find a circuit which minimize the total distance ; all cities must be visited once. Symmetric TSP dist(x,y)=dist(y,x) |S| = n!/2n = (n-1)! / 2 ; n>6 TSP > SAT 10 cities = 181 000 solutions 20 cities = 10 000 000 000 000 000 solutions 50 cities = 100 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 solutions There is 1 000 000 000 000 000 000 000 litres of water in the planet http://www.ing.unlp.edu.ar/cetad/mos/TSPBIB_home.html Metaheuristics E-G. Talbi The traveling salesman problem (from Chvatal’s page) • 1800-1900: first descriptions of the problem; • 1920-1930: problem becomes well defined; • 1940-50: it starts to be recognized as “hard”; • 1954: an instance with 42 cities is solved to optimality. Metaheuristics E-G. Talbi Size of the search space NLP problem : Non-Linear Programming Problem Many applications : mechanics, aeronautics, … Classical methods don’t give “good” esultsr n n cos (4 )− 22 cos ( )Non linear Function ∑∏xi xi G2 ( x= ) i==11i Global Optimum unknown n 2 ∑ixi i=1 n n 0≥ . 75 ≤7 .n 5 0≤ ≤ 10 ∏ xi ∑xi i i=1 i=1 x Non-linear Constraint + linear constraint Metaheuristics E-G. Talbi Size of the search space Treated as a mathematical problem = infinite number of solutions / dimension. On machine : suppose a precision of six decimals, each variable may take 10 000 000 different values. |S| = 10 000 000n = 107n Search space larger than TSP Evaluation function and constraints ? Maximize G2(x) ; Non feasible solutions = 0 Metaheuristics E-G. Talbi Constraints Most of the problems have constraints. Ex : Timetabling : z List of courses, classes, students / classes, professor / classes, available rooms, capacity of rooms, logistics of rooms (video, computer, microphone, …). z Hard constraints (must be satisfied) : z Each class must be assigned to a room with a sufficient number of seats and required facilities. z Students assigned to different classes must not be scheduled at the same time. z Professors must not be scheduled to courses in parallel. Metaheuristics E-G. Talbi Optimization + Simulation: Cellular network design Economical Challenge (Cost, Quality of service) Network Design (Positioninig of antenna, Configuration of antenna Objectives : Constraints : Covering z Min (number of sites) Handover z Min (interferences) z Min (traffic loss) z Max (resource use) Metaheuristics E-G. Talbi Parameters of antennaes Decision variables Data Bounds Power [26, 55] dBm Diagram 3 types Height [30, 50] m Azim ut [0, 359] ° Tilt [-15, 0] ° Transmitters [1, 7] Search space : 568 sites candidates Î 23689160 solutions and ~ 600.109 choices for one antenna ! 5 Cost Evaluation (pop : 100, Gen : 10 , cost = more than one year !!) 1 omni or 1-3 sectorial Metaheuristics E-G. Talbi Impact of the parameters via simulation Cell = zone covered by a BTS Limited Traffic for a site (technological constraint) Many sites are necessary Omni-directional Sectorial type 1 Sectorial type 2 Propagation Model ?: 60 dBm Free space Azimut 50° PIRE -10 dBm Tilt -15° Metaheuristics E-G. Talbi Model : Constraints A set of BTS satisfying the constraints : z Covering P z Handover (mobility) Metaheuristics E-G. Talbi Model : Objectives Min Number of sites (cost) Min Interferences Max Trafic handled Zone géographique Used Sites Non-used Sites Covered Zone Handover Zone Metaheuristics E-G. Talbi Other standard problems i Integer programming problem (IP); Commercially available. i (Weighted) constraint satisfaction problem (CSP, WCSP) i Maximum satisfiability problem (MAX SAT) i Set covering problem (SCP) Generalized assignment problem (GAP) i Generalized quadratic assignment problem (GQAP) i Resource constrained project scheduling problem (RCPSP) i Vehicle routing problem (VRP) i Cutting stock problem (CSTP) i 2-Dimensional Packing Problem (2PP) i … Metaheuristics E-G. Talbi Ex : Knapsack Problem The classic Knapsack problem is typically put forth as: z A thief breaks into a store and wants to fill his knapsack with as much value in goods as possible before making his escape. Given the following list of items available, what should he take? z Item A, weighting wA pounds and valued at vA z Item B, weighting wB pounds and valued at vB z Item C, weighting wC pounds and valued at vC z ••• Metaheuristics E-G. Talbi Ex : Knapsack Problem Input