DOCSLIB.ORG

New Variants of Variable Neighbourhood Search for 0-1 Mixed Integer Programming and Clustering

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
New Variants of Variable Neighbourhood Search for 0-1 Mixed Integer Programming and Clustering New Variants of Variable Neighbourhood Search for 0-1 Mixed Integer Programming and Clustering A thesis submitted for the degree of Doctor of Philosophy Jasmina Lazi´c School of Information Systems, Computing and Mathematics Brunel University c August 3, 2010 Contents Contents i List of Figures vi List of Tables viii List of Abbreviations ix Acknowledgements xi Related Publications xiii Abstract 1 1 Introduction 3 1.1 CombinatorialOptimisation . ........ 3 1.2 The0-1 MixedInteger ProgrammingProblem . ......... 11 1.3 Clustering...................................... ... 17 1.4 ThesisOverview .................................. ... 18 2 Local Search Methodologies in Discrete Optimisation 21 2.1 BasicLocalSearch ................................ .... 21 2.2 A Brief Overview of Local Search Based Metaheuristics . ............. 24 2.2.1 SimulatedAnnealing. ... 24 2.2.2 TabuSearch ................................... 26 2.2.3 Greedy Randomised Adaptive Search . ..... 28 2.2.4 GuidedLocalSearch............................. .. 29 2.2.5 IteratedLocalSearch . ... 31 2.3 VariableNeighbourhoodSearch . ........ 33 2.3.1 BasicSchemes .................................. 33 2.3.2 AdvancedSchemes ............................... 40 2.3.3 Variable Neighbourhood Formulation Space Search . ........... 40 2.3.4 Primal-dualVNS................................ 41 2.3.5 Dynamic Selection of Parameters and/or NeighbourhoodStructures . 42 2.3.6 VeryLarge-scaleVNS .. .. .. .. .. .. .. .. .. .. .. .. .. 43 i ii Contents 2.3.7 ParallelVNS ................................... 44 2.4 Local Search for 0-1 Mixed Integer Programming . ........... 44 2.4.1 LocalBranching ................................ 47 2.4.2 VariableNeighbourhoodBranching. ....... 47 2.4.3 Relaxation Induced Neighbourhood Search . ........ 49 2.5 Future Trends: the Hyper-reactive Optimisation . ............. 50 3 Variable Neighbourhood Search for Colour Image Quantisation 53 3.1 RelatedWork..................................... .. 55 3.1.1 The Genetic C-Means Heuristic (GCMH) . ..... 55 3.1.2 The Particle Swarm Optimisation (PSO) Heuristic . .......... 56 3.2 VNSMethodsfortheCIQProblem. ..... 57 3.3 ComputationalResults. ...... 59 3.4 Summary ......................................... 63 4 Variable Neighbourhood Decomposition Search for the 0-1 MIP Problem 67 4.1 TheVNDS-MIPAlgorithm .. .. .. .. .. .. .. .. .. .. .. .. ... 68 4.2 ComputationalResults. ...... 70 4.3 Summary ......................................... 81 5 Applications of VNDS to Some Specific 0-1 MIP Problems 87 5.1 The Multidimensional Knapsack Problem . ......... 87 5.1.1 RelatedWork................................... 89 5.1.2 VNDS-MIPwithPseudo-cuts . ... 92 5.1.3 ASecondLevelofDecompositioninVNDS . ..... 96 5.1.4 ComputationalResults. .... 98 5.1.5 Summary ..................................... 111 5.2 TheBargeContainerShipRoutingProblem . ......... 112 5.2.1 FormulationoftheProblem . .... 112 5.2.2 ComputationalResults. .... 118 5.2.3 Summary ..................................... 120 5.3 The Two-Stage Stochastic Mixed Integer Programming Problem .......... 120 5.3.1 Existing Solution Methodology for the Mixed Integer 2SSP ......... 123 5.3.2 VNDS Heuristic for the 0-1 Mixed Integer 2SSP . ....... 123 5.3.3 ComputationalResults. .... 125 5.3.4 Summary ..................................... 127 6 Variable Neighbourhood Search and 0-1 MIP Feasibility 129 6.1 RelatedWork: FeasibilityPump . ....... 130 6.1.1 BasicFeasibilityPump. .... 131 6.1.2 GeneralFeasibilityPump . .... 131 6.1.3 ObjectiveFeasibilityPump . ..... 132 6.1.4 Feasibility Pump and Local Branching . ....... 133 6.2 VariableNeighbourhoodPump . ...... 134 iii 6.3 Constructive Variable Neighbourhood Decomposition Search ............ 136 6.4 ComputationalResults. ...... 136 6.5 Summary ......................................... 144 7 Conclusions 147 Bibliography 151 Index 171 A Computational Complexity 175 A.1 DecisionProblemsandFormalLanguages . ......... 175 A.2 TuringMachines .................................. ... 176 A.3 Time Complexity Classes p and np ........................... 180 A.4 np-Completeness ..................................... 181 B Statistical Tests 183 B.1 FriedmanTest .................................... 183 B.2 Bonferroni-DunnTestandNemenyiTest . ......... 184 C Performance Profiles 185 iv Contents List of Figures 1.1 Structural classification of hybridisations between metaheuristics and exact methods. 10 2.1 Basiclocalsearch. ............................... ..... 22 2.2 Bestimprovementprocedure. ....... 23 2.3 Firstimprovementprocedure. ........ 23 2.4 Localsearch: stallinginalocaloptimum. ........... 24 2.5 Simulatedannealing. .. .. .. .. .. .. .. .. .. .. .. .. ...... 25 2.6 Geometric cooling scheme for Simulated annealing. .............. 26 2.7 Tabusearch. ..................................... .. 27 2.8 Greedyrandomisedadaptivesearch. ......... 29 2.9 GLS: escaping from local minimum by increasing the objective function value. 30 2.10 Guidedlocalsearch. .. .. .. .. .. .. .. .. .. .. .. .. ...... 31 2.11 Iteratedlocalsearch. ........ 32 2.12 The change of the used neighbourhood in some typical VNS solution trajectory. 34 2.13VNDpseudo-code. ................................ .... 35 2.14RVNSpseudo-code. ............................... .... 36 2.15 ThebasicVNSscheme. ............................. .... 37 2.16 TheBasicVNSpseudo-code. ...... 37 2.17GeneralVNS..................................... ... 38 2.18SkewedVNS. ..................................... .. 39 2.19 Variable neighbourhood decomposition search. .............. 40 2.20 VNSformulationspacesearch. ........ 41 2.21 Variable neighbourhood descent with memory. ............ 42 2.22 Localsearchinthe 0-1 MIP solutionspace. ........... 46 2.23 UpdatingtheneighbourhoodsizeinLB. .......... 47 2.24 NeighbourhoodupdateinVND-MIP. ........ 48 2.25 VNBshakingpseudo-code. ...... 48 2.26 VariableNeighbourhoodBranching. .......... 49 2.27 A standard local search-based metaheuristic. .............. 51 2.28 Hyper-reactivesearch. ........ 52 2.29 TheHyper-reactiveVNS. ...... 52 3.1 RVNSforCIQ. ..................................... 58 v vi List of Figures 3.2 VNDSforCIQ...................................... 59 3.3 Time performance of VNS quantisation methods . .......... 62 3.4 MSE/running time comparison between the M-Median and the M-Means solution. 63 3.5 “Lenna” quantised to 16 colours: (a) M-Median solution, (b) M-Means solution. 64 3.6 “Lenna” quantised to 64 colours: (a) M-Median solution, (b) M-Means solution. 64 3.7 “Lenna” quantised to 256 colours: (a) M-Median solution, (b) M-Means solution. 64 3.8 “Baboon” quantised to 16 colours: (a) M-Median solution, (b) M-Means solution. 65 3.9 “Baboon” quantised to 64 colours: (a) M-Median solution, (b) M-Means solution. 65 3.10 “Baboon” quantised to 256 colours: (a) M-Median solution, (b) M-Means solution. 65 3.11 “Peppers” quantised to 16 colours: (a) M-Median solution, (b) M-Means solution. 66 3.12 “Peppers” quantised to 64 colours: (a) M-Median solution, (b) M-Means solution. 66 3.13 “Peppers” quantised to 256 colours: (a) M-Median solution, (b) M-Means solution. 66 4.1 VNDSforMIPs. .................................... 69 4.2 VNDforMIPs...................................... 70 4.3 Relative gap average over all instances in test bed vs. computational time. 74 4.4 Relative gap average over demanding instances vs. computational time. 74 4.5 Relative gap average over non-demanding instances vs. computational time. 75 4.6 The change of relative gap with computational time for biella1instance. 78 4.7 Relative gap values (in %) for large-spread instances. ................ 80 4.8 Relative gap values (in %) for medium-spread instances. ............... 80 4.9 Relative gap values (in %) for small-spread instances. ............... 81 4.10 Relative gap values (in %) for very small-spread instances............... 81 4.11 Bonferroni-Dunn critical difference from the solution quality rank of VNDS-MIP . 85 4.12 Bonferroni-Dunn critical difference from the running time rank of VNDS-MIP . 85 5.1 Linearprogrammingbasedalgorithm. ......... 89 5.2 An iterativerelaxationbased heuristic. ............. 91 5.3 An iterative independent relaxation based heuristic. ................ 92 5.4 VNDS-MIPwithpseudo-cuts.. ...... 93 5.5 VNDS-MIP with upper and lower bounding and another orderingstrategy. 96 5.6 Two levels of decomposition with hyperplanes ordering. ............... 97 5.7 Flexibility for changing the hyperplanes. ............. 99 5.8 Performance profiles of all 7 algorithms over the OR librarydataset.. 110 5.9 Performance profiles of all 7 algorithms over the GK data set............. 111 5.10 Example itinerary of a barge container ship . ............ 114 5.11 VNDS-SIPpseudo-code. ...... 124 6.1 Thebasicfeasibilitypump. ....... 132 6.2 ConstructiveVND-MIP. ..... 135 6.3 The variable neighbourhood pump pseudo-code. ............ 135 6.4 Constructive VNDS for 0-1 MIP feasibility. ........... 139 List of Tables 3.1 The MSE of the VNS, GCMH and PSO heuristics . ..... 60 3.2 The MSE/CPU time of the RVNDS’, RVNDS, VNDS and RVNS algorithms . 61 3.3 The MSE/CPU time comparison between the M-Median and M-Means solutions. 62 4.1 Testbedinformation. ............................. ..... 72 4.2 VNDS1andVNDS2timeperformance. ..... 76 4.3 VNDS-MIP objective values for two different parameters settings .......... 77 4.4 Objective function values for all the 5 methods tested. ............... 78 4.5 Relative gap values (in %) for all the 5 methods tested. ............. 79 4.6 Running times (in seconds) for
Load more

Recommended publications
  • A Computational Study of a Solver System for Processing Two-Stage Stochastic Linear Programming Problems ∗
    A computational study of a solver system for processing two-stage stochastic linear programming problems ¤ Victor Zverovich Csaba F¶abi¶an Francis Ellison Gautam Mitra July 2, 2009 1 Introduction and background Formulation of stochastic optimisation problems and computational algo- rithms for their solution continue to make steady progress as can be seen from an analysis of many developments in this ¯eld. The edited volume by Wallace and Ziemba (2005) outlines both the SP modelling systems and many applications in diverse domains. More recently, Fabozzi et al. (2007) has considered the application of SP models to challenging ¯nancial engineering problems. The tightly knit yet highly focused group of researchers COSP: Committee on Stochastic Pro- gramming, their triennial international SP conference, and their active web- site points to the progressive acceptance of SP as a valuable decision tool. At the same time many of the major software vendors, namely, XPRESS, AIMMS, MAXIMAL, and GAMS have started o®ering SP extensions to their optimisation suites. Our analysis of the modelling and algorithmic solver requirements re- veals that (a) modelling support (b) scenario generation and (c) solution methods are three important aspects of a working SP system. Our research is focussed on all three aspects and we refer the readers to Valente et al. (2009) for modelling and Mitra et al. (2007) and Di Domenica et al. (2009) for scenario generation. In this paper we are concerned entirely with com- putational solution methods. Given the tremendous advance in LP solver ¤CARISMA, Brunel University, Uxbridge, Middlesex, United Kingdom 1 algorithms there is certain amount of complacency that by constructing a "deterministic equivalent" problems it is possible to process most realistic instances of SP problems.
    [Show full text]
  • A Computational Study of a Solver System for Processing Two-Stage Stochastic Linear Programming Problems
    A computational study of a solver system for processing two-stage stochastic linear programming problems Victor Zverovich ∗y Csaba I. F´abi´an zx Francis Ellison ∗y Gautam Mitra ∗y November 13, 2009 1 Introduction and background Formulation of stochastic optimisation problems and computational algo- rithms for their solution continue to make steady progress as can be seen from an analysis of many developments in this field. The edited volume by Wallace and Ziemba (2005) outlines both the SP modelling systems and many applications in diverse domains. More recently, Fabozzi et al. (2007) has considered the application of SP models to challenging financial engineering problems. The tightly knit yet highly focused group of researchers COSP: Committee on Stochastic Pro- gramming, their triennial international SP conference, and their active web- site points to the progressive acceptance of SP as a valuable decision tool. At the same time many of the major software vendors, namely, XPRESS, AIMMS, MAXIMAL, and GAMS have started offering SP extensions to their optimisation suites. Our analysis of the modelling and algorithmic solver requirements re- veals that (a) modelling support (b) scenario generation and (c) solution ∗CARISMA: The Centre for the Analysis of Risk and Optimisation Modelling Appli- cations, School of Information Systems, Computing and Mathematics, Brunel University, UK. yOptiRisk Systems, Uxbridge, Middlesex, UK. zInstitute of Informatics, Kecskem´etCollege, 10 Izs´aki´ut,Kecskem´et,6000, Hungary. E-mail: [email protected]. xDepartment of OR, Lor´andE¨otv¨osUniversity, Budapest, Hungary. 1 methods are three important aspects of a working SP system. Our research is focussed on all three aspects and we refer the readers to Valente et al.
    [Show full text]
  • Informs Joint International Conference Rome, Italy July 1-4, 2013
    EURO-INFORMS JOINT INTERNATIONAL CONFERENCE ROME, ITALY JULY 1-4, 2013 FINAL REPORT This report was prepared by Paolo Dell’Olmo and submitted to the EURO Executive Council May 2014 Rome, Italy 2 1. PLANNING AND ORGANIZATIONAL DECISION-MAKING In 2010, at the EURO XXIV Conference in Lisbon, the Italian Operational Research Society (AIRO) was formally appointeD to host the EURO – INFORMS XXVI European Conference on Operational Research 2013 in Rome, Italy. The Conference organization anD structuring was carrieD out according to the agreement (“Rome Protocol”) between EURO, INFORMS anD AIRO signed in 2011. 1.1 Programme Committee • Marc Sevaux (PC Co-Chair/EURO), Université De Bretagne-SuD, France • DaviD Simchi-Levi (PC Co-Chair/INFORMS), Massachusetts Institute of Technology, CambriDge, USA • Sally BrailsforD (EURO Vice President 1) University of Southampton, UniteD KingDom • Maria Antonia Carravilla, UniversiDaDe do Porto, Portugal • Marielle Christiansen, (EURO25 Chair) Norwegian University of Science anD Technology, TronDheim, Norway • Paolo Dell’Olmo (OC Chair), Sapienza University of Rome, Italy • Guillermo Gallego, Columbia University, New York, USA • Jérémie Gallien, LonDon Business School, United Kingdom • Michel GenDreau, Ecole Polytechnique, Montreal, CanaDa • Diego Klabjan, Northwestern University, Evanston, USA • Ana Meca Martinez, Universidade Miguel Hernàndez de Elche, Spain • Rolf Möhring, Tecnische Universitat Berlin, Germany • Michael PineDo, New York University, USA • DaviD Psinger (EURO27 Chair) University of Copenhagen,
    [Show full text]
  • Fortsp: a Stochastic Programming Solver Version 1.2
    FortSP: A Stochastic Programming Solver Version 1.2 January 31, 2014 http://www.optirisk-systems.com http://www.carisma.brunel.ac.uk Version: 1.2 Prepared by Victor Zverovich, Cristiano Arbex Valle, Francis Ellison and Gautam Mitra OptiRisk Systems Copyright c 2013 OptiRisk Systems DO NOT DUPLICATE WITHOUT PERMISSION All brand names, product names are trademarks or registered trademarks of their respective holders. The material presented in this manual is subject to change without prior notice and is intended for general information only. The views of the authors expressed in this document do not represent the views and/or opinions of OptiRisk Systems. OptiRisk Systems One Oxford Road Uxbridge, Middlesex, UB9 4DA United Kingdom www.optirisk-systems.com +44 (0) 1895 256484 2 Acknowledgements We would like to acknowledge the following persons for their contribution to the software: • Originally implemented by Dr Chandra Poojari and Dr Frank Ellison • Subsequent extensive reengineering and updates made by Dr Victor Zverovich We also acknowledge Dr Csaba I. F´abi´anfor his contribution in respect of Level Decompo- sition / Regularisation methods, and Dr Suvrajeet Sen. Finally, we thank Cristiano Arbex Valle who is now the custodian of Fortsp. Preface FortSP is a large scale stochastic programming (SP) solver, which processes linear and mixed integer scenario-based SP problems with recourse. It also supports scenario-based problems with chance constraints and integrated chance constraints. Several different SP algorithms are available for the solution, stochastic measures such as expected value of perfect information (EVPI) and value of the stochastic solution (VSS) may be calculated, and it can use CPLEX, FortMP or Gurobi as its embedded, underlying solver engine.
    [Show full text]
  • Implementing and Comparing Stochastic and Robust Programming
    Implementing and Comparing Stochastic and Robust Programming Université de Liège – Faculté des sciences appliquées Année académique 2014 - 2015 Mémoire de fin d’études réalisé en vue de l’obtention du grade de master ingénieur civil en informatique par Thibaut Cuvelier. ii Traditional optimisation tools focus on deterministic problems: scheduling airline flight crews (with as few employees as possible while still meeting legal constraints, such as maximum working time), finding the shortest path in a graph (used by navigation systems to give directions, usually based on GPS signals), etc. However, this deterministic hypothesis sometimes yields useless solutions: actual parameters cannot always be known to full precision, one reason being their randomness. For example, when scheduling trucks for freight transporta- tion, if there is unexpected congestion on the roads, the deadlines might not be met, the company might be required to financially compensate for this delay, but also for the following deliveries that could not be made on schedule. Two main approaches are developed in the literature to take into account this uncertainty: take decision based on probability distributions of the uncertain parameters (stochastic programming) or considering they lie in some set (robust programming). In general, the first one leads to a large increase in the size of the problems to solve (and thus requires algorithms to work around this dimensionality curse), while the second is more conservative but tends to change the nature of the programs (which can impose a new solver technology). Some authors [2] claim that those two mindsets are equivalent, meaning that the solutions they provide are equivalent when faced with the same uncer- tainty.
    [Show full text]
  • Advanced Acceleration Techniques for Nested Benders Decomposition in Stochastic Programming
    Dissertation Advanced acceleration techniques for Nested Benders decomposition in Stochastic Programming Christian Wolf, M.Sc. Schriftliche Arbeit zur Erlangung des akademischen Grades doctor rerum politicarum (dr. rer. pol.) im Fach Wirtschaftsinformatik eingereicht an der Fakultät für Wirtschaftswissenschaften der Universität Paderborn Gutachter: 1. Prof. Dr. Leena Suhl 2. Prof. Dr. Csaba I. Fábián Paderborn, im Oktober 2013 Acknowledgements This thesis is the result of working for nearly four years at the Decision Support & Opera- tions Research (DS&OR) Lab at the University of Paderborn. I would like to thank those people whose support helped me in writing my dissertation. First of all, I thank my supervisor Leena Suhl for giving me the possibility to pursue a thesis by offering me a job at her research group in the first place. Her support and guidance over the years helped me tremendously in finishing the dissertation. I also thank Achim Koberstein for introducing me to the field of Operations Research through the lecture “Operations Research A” and the opportunity to do research as a student in the field of stochastic programming. It is due to his insistence that I decided to write my computer science master’s thesis at the DS&OR Lab. I am very grateful to have Csaba Fábián as my second advisor. He not only gave valuable advice but also pointed me towards the on-demand accuracy concept. Collaborating with him has been both straightforward and effective. My present and past colleagues at the DS&OR Lab deserve a deep thank you for the enjoyable time I had at the research group in the last years.
    [Show full text]
  • Algorithmic Innovations and Software for the Dual Decomposition Method Applied to Stochastic Mixed-Integer Programs
    Noname manuscript No. (will be inserted by the editor) Algorithmic innovations and software for the dual decomposition method applied to stochastic mixed-integer programs Kibaek Kim · Victor M. Zavala Submitted: September 15, 2016 Abstract We develop algorithmic innovations for the dual decomposition method to address two-stage stochastic programs with mixed-integer recourse and provide a parallel software implementation that we call DSP. Our innova- tions include the incorporation of Benders-like cuts in a dual decomposition framework to tighten Lagrangian subproblems and aid the exclusion of infeasi- ble first-stage solutions for problems without (relative) complete recourse. We also use an interior-point cutting-plane method with new termination criteria for solving the Lagrangian master problem. We prove that the algorithm con- verges to an optimal solution of the Lagrangian dual problem in a finite number of iterations, and we prove that convergence can be achieved even if the master problem is solved suboptimally. DSP can solve instances specified in C code, SMPS files, and StochJump (a Julia-based and parallel algebraic modeling lan- guage). DSP also implements a standard Benders decomposition method and a dual decomposition method based on subgradient dual updates that we use to perform benchmarks. We present extensive numerical results using standard SIPLIB instances and a large unit commitment problem to demonstrate that This material is based upon work supported by the U.S. Department of Energy, Office of Science, under contract number DE-AC02-06CH11357. We gratefully acknowledge the com- puting resources provided on Blues, a high-performance computing cluster operated by the Laboratory Computing Resource Center at Argonne National Laboratory.
    [Show full text]