New Heuristic and Metaheuristic Approaches Applied to the Multiple-Choice Multidimensional Knapsack Problem
Total Page:16
File Type:pdf, Size:1020Kb
New Heuristic And Metaheuristic Approaches Applied To The Multiple-choice Multidimensional Knapsack Problem A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy by Chaitr S. Hiremath M.S., Wright State University, 2004 2008 Wright State University COPYRIGHT BY Chaitr S. Hiremath 2008 WRIGHT STATE UNIVERSITY SCHOOL OF GRADUATE STUDIES February 23, 2008 I HEREBY RECOMMEND THAT THE DISSERTATION PREPARED UNDER MY SUPERVISION BY Chaitr S. Hiremath ENTITLED New Heuristic And Metaheuristic Approaches Applied To The Multiple-choice Multidimensional Knapsack Problem BE AC- CEPTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DE- GREE OF Doctor of Philosophy. Raymond R. Hill, Ph.D. Dissertation Director Ramana Grandhi, Ph.D. Director, Engineering Ph.D. Program Joseph F. Thomas, Jr. , Ph.D. Dean, School of Graduate Studies Committee on Final Examination Raymond R. Hill, Ph.D. James T. Moore, Ph.D. Xinhui Zhang, Ph.D. Gary Kinney, Ph.D. Mateen Rizki, Ph.D. ABSTRACT Hiremath, Chaitr . Ph.D., Department of Biomedical, Industrial and Human Factors Engineer- ing, Wright State University, 2008. New Heuristic And Metaheuristic Approaches Applied To The Multiple-choice Multidimensional Knapsack Problem. The knapsack problem has been used to model various decision making processes. Industrial appli- cations find the need for satisfying additional constraints and these necessities lead to the variants and extensions of knapsack problems which are complex to solve. Heuristic algorithms have been developed by many researchers to solve the variants of knapsack problems. Empirical analysis has been done to compare the performance of these heuristics. Little research has been done to find out why certain algorithms perform well on certain test problems while not so well on other test problems. There has been little work done to gain knowledge of the test problem characteristics and their effects on algorithm performance. The research focuses on the Multiple-choice Multidimensional Knapsack Problem (MMKP), a complex variant of the knapsack problem. The objectives of the research are fourfold. The first objective is to show how empirical science can lead to theory. The research involves the empirical analysis of current heuristics with respect to problem structure especially constraint correlation and constraint slackness settings. The second objective is to consider the performance traits of heuristic procedures and develop a more diverse set of MMKP test problems considering problem charac- teristics like the number of variables, number of constraints, constraint correlation, and constraint right-hand side capacities. The third objective is the development of new heuristic approaches for solving the MMKP. This involves examining the existing heuristics against our new test set and using the analysis of the results to help in the development of new heuristic approaches. The fourth objective is to develop improved metaheuristic procedures for the MMKP using the improved heuristic approaches to initialize searches or to improve local search neighborhoods. iii Contents 1 Introduction 1 1.1 Discussion of Knapsack Problems .......................... 1 1.2 Overview of the Dissertation Research ........................ 2 1.3 Contributions of the Dissertation Research ...................... 3 2 Multi-Dimensional Knapsack Problems 5 2.1 Introduction ...................................... 5 2.2 Branch-and-Bound Approach ............................. 6 2.3 Dynamic Programming ................................ 10 2.4 Greedy Heuristics ................................... 13 2.5 Transformation Heuristics .............................. 16 2.6 Metaheuristic Approaches .............................. 18 2.6.1 Tabu Search (TS) ............................... 19 2.6.2 Genetic Algorithm (GA) ........................... 22 2.6.3 Simulated Annealing (SA) .......................... 28 3 Extensions and Variants of the Knapsack Problem involving the notion of sets 30 3.1 Introduction ...................................... 30 3.2 Multiple Knapsack Problems (MKP) ......................... 31 3.2.1 MKP Formulation .............................. 31 3.2.2 Heuristic Solution Approaches for MKP ................... 31 3.3 Multiple Choice Knapsack Problems (MCKP) .................... 33 3.3.1 MCKP Formulation ............................. 33 3.3.2 Heuristic Solution Approaches for MCKP .................. 34 3.4 Multiple-choice Multi-dimensional Knapsack Problems (MMKP) ......... 36 3.4.1 MMKP Formulation ............................. 36 3.4.2 Heuristic Solution Approaches for MMKP ................. 37 3.5 Applications and Formulations of the MMKP-type problems ............ 43 4 Legacy Heuristics and Test Problems Analysis 50 4.1 Introduction ...................................... 50 4.2 Legacy Heuristics ................................... 51 4.2.1 Legacy Heuristics for Multiple Knapsack Problems (MKP) ......... 51 4.2.2 Legacy Heuristics for Multiple Choice Knapsack Problems (MCKP) .... 55 iv 4.2.3 Legacy Heuristics for Multiple-choice Multi-dimensional Knapsack Prob- lems (MMKP) ................................ 58 4.3 Test Problem Analysis ................................ 69 4.3.1 Test Problem Analysis for Multiple Knapsack Problems (MKP) ...... 69 4.3.2 Test Problems for Multiple Choice Knapsack Problems (MCKP) ...... 71 4.3.3 Test Problems for Multiple-choice Multi-dimensional Knapsack Problems (MMKP) ................................... 74 4.4 Problem Structure Analysis of Test Problems .................... 76 4.4.1 Structure of MDKP Test Problems ...................... 76 4.4.2 Structure of MKP Test Problems ....................... 78 4.4.3 Structure of MCKP Test Problems ...................... 81 4.4.4 Structure of MMKP Test Problems ...................... 82 4.5 Summary ....................................... 84 5 Empirical Analyses of Legacy MMKP Heuristics and Test Problem Generation 90 5.1 Introduction ...................................... 90 5.2 Problem Generation and Problem Characteristics for MMKP ............ 91 5.2.1 Standard MMKP Test Problem Generation ................. 92 5.2.2 Analytical MMKP Test Problem Generation ................. 93 5.2.3 Competitive MMKP Test Problem Generation ................ 95 5.2.4 Analytical MMKP Test Sets Versus Available MMKP Test Set ....... 97 5.3 Empirical Analyses of MMKP Heuristics on Available Test Problems ....... 102 5.4 Empirical Analyses of MMKP Heuristics on New MMKP Test Problem Set .... 109 5.4.1 Analyses based on Constraint Right-Hand Side Setting ........... 109 5.4.2 Analyses based on Correlation Structure ................... 120 5.5 Summary ....................................... 125 6 New Greedy Heuristics for the MMKP 127 6.1 Introduction ...................................... 127 6.2 A TYPE-based Heuristic for the MMKP ....................... 127 6.3 New Greedy Heuristic Version 1 (CH1) ....................... 130 6.3.1 NG V3 Heuristic (Cho 2005) ......................... 130 6.3.2 CH1 Implementation ............................. 133 6.3.3 Empirical Tests for the CH1 Implementation ................ 134 6.4 New Greedy Heuristic Version 2 (CH2) ....................... 143 6.4.1 CH2 Implementation ............................. 143 6.4.2 Empirical Tests for the CH2 Implementation ................ 146 6.5 Summary ....................................... 152 7 Metaheuristic Solution Procedure for the MMKP 159 7.1 Introduction ...................................... 159 7.2 Concept of a Search Neighborhood .......................... 160 7.3 First-Level Tabu Search (FLTS) for the MMKP ................... 161 7.3.1 FLTS Implementation ............................ 161 7.3.2 Empirical Tests for the FLTS Implementation ................ 163 v 7.3.3 Extensions of the FLTS for the MMKP ................... 172 7.4 Sequential Fan Candidate List (FanTabu) for the MMKP .............. 172 7.4.1 FanTabu Implementation ........................... 175 7.4.2 Empirical Tests for the FanTabu Implementation .............. 177 7.5 CPCCP with Fan Candidate List (CCFT) for the MMKP .............. 188 7.5.1 CCFT Implementation ............................ 188 7.5.2 Empirical Tests for the CCFT Candidate List Implementation ....... 188 7.6 Comparison of TS approaches with Reactive Local Search Approach (RLS) .... 192 7.6.1 RLS Approach ................................ 192 7.6.2 Empirical Tests Comparing TS approaches with RLS Approach ...... 200 7.7 Summary ....................................... 201 8 Summary, Contributions, and Future Avenues 214 8.1 Summary and Contributions ............................. 214 8.1.1 Legacy Heuristics and Test Problem Analysis ................ 215 8.1.2 Insights on Heuristic Performance Based on Problem Structure ....... 215 8.1.3 Empirical Science Leading to Theory .................... 216 8.1.4 New Test Set Development .......................... 216 8.1.5 New Greedy Heuristics Development .................... 216 8.1.6 Metaheuristic Solution Procedure Development ............... 217 8.2 Future Avenues .................................... 217 Appendices 219 A Additional Results from Empirical Tests 219 B Details on Cho Generation Approach 225 Bibliography 228 vi List of Figures 2.1 Different types of crossover operators ((Zalzala and Fleming 1997) and (Renner and Ekart 2003)) ...................................... 24 2.2 Genetic Algorithm Flowchart (Renner and Ekart 2003) ............... 25 4.1 Range of Correlation Values Between Objective Function and Constraint