Wright State University CORE Scholar Browse all Theses and Dissertations Theses and Dissertations 2008 New Heuristic And Metaheuristic Approaches Applied To The Multiple-choice Multidimensional Knapsack Problem Chaitr Hiremath Wright State University Follow this and additional works at: https://corescholar.libraries.wright.edu/etd_all Part of the Engineering Commons Repository Citation Hiremath, Chaitr, "New Heuristic And Metaheuristic Approaches Applied To The Multiple-choice Multidimensional Knapsack Problem" (2008). Browse all Theses and Dissertations. 223. https://corescholar.libraries.wright.edu/etd_all/223 This Dissertation is brought to you for free and open access by the Theses and Dissertations at CORE Scholar. It has been accepted for inclusion in Browse all Theses and Dissertations by an authorized administrator of CORE Scholar. For more information, please contact [email protected]. New Heuristic And Metaheuristic Approaches Applied To The Multiple-choice Multidimensional Knapsack Problem A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy by Chaitr S. Hiremath M.S., Wright State University, 2004 2008 Wright State University COPYRIGHT BY Chaitr S. Hiremath 2008 WRIGHT STATE UNIVERSITY SCHOOL OF GRADUATE STUDIES February 23, 2008 I HEREBY RECOMMEND THAT THE DISSERTATION PREPARED UNDER MY SUPERVISION BY Chaitr S. Hiremath ENTITLED New Heuristic And Metaheuristic Approaches Applied To The Multiple-choice Multidimensional Knapsack Problem BE AC- CEPTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DE- GREE OF Doctor of Philosophy. Raymond R. Hill, Ph.D. Dissertation Director Ramana Grandhi, Ph.D. Director, Engineering Ph.D. Program Joseph F. Thomas, Jr. , Ph.D. Dean, School of Graduate Studies Committee on Final Examination Raymond R. Hill, Ph.D. James T. Moore, Ph.D. Xinhui Zhang, Ph.D. Gary Kinney, Ph.D. Mateen Rizki, Ph.D. ABSTRACT Hiremath, Chaitr . Ph.D., Department of Biomedical, Industrial and Human Factors Engineer- ing, Wright State University, 2008. New Heuristic And Metaheuristic Approaches Applied To The Multiple-choice Multidimensional Knapsack Problem. The knapsack problem has been used to model various decision making processes. Industrial appli- cations find the need for satisfying additional constraints and these necessities lead to the variants and extensions of knapsack problems which are complex to solve. Heuristic algorithms have been developed by many researchers to solve the variants of knapsack problems. Empirical analysis has been done to compare the performance of these heuristics. Little research has been done to find out why certain algorithms perform well on certain test problems while not so well on other test problems. There has been little work done to gain knowledge of the test problem characteristics and their effects on algorithm performance. The research focuses on the Multiple-choice Multidimensional Knapsack Problem (MMKP), a complex variant of the knapsack problem. The objectives of the research are fourfold. The first objective is to show how empirical science can lead to theory. The research involves the empirical analysis of current heuristics with respect to problem structure especially constraint correlation and constraint slackness settings. The second objective is to consider the performance traits of heuristic procedures and develop a more diverse set of MMKP test problems considering problem charac- teristics like the number of variables, number of constraints, constraint correlation, and constraint right-hand side capacities. The third objective is the development of new heuristic approaches for solving the MMKP. This involves examining the existing heuristics against our new test set and using the analysis of the results to help in the development of new heuristic approaches. The fourth objective is to develop improved metaheuristic procedures for the MMKP using the improved heuristic approaches to initialize searches or to improve local search neighborhoods. iii Contents 1 Introduction 1 1.1 Discussion of Knapsack Problems .......................... 1 1.2 Overview of the Dissertation Research ........................ 2 1.3 Contributions of the Dissertation Research ...................... 3 2 Multi-Dimensional Knapsack Problems 5 2.1 Introduction ...................................... 5 2.2 Branch-and-Bound Approach ............................. 6 2.3 Dynamic Programming ................................ 10 2.4 Greedy Heuristics ................................... 13 2.5 Transformation Heuristics .............................. 16 2.6 Metaheuristic Approaches .............................. 18 2.6.1 Tabu Search (TS) ............................... 19 2.6.2 Genetic Algorithm (GA) ........................... 22 2.6.3 Simulated Annealing (SA) .......................... 28 3 Extensions and Variants of the Knapsack Problem involving the notion of sets 30 3.1 Introduction ...................................... 30 3.2 Multiple Knapsack Problems (MKP) ......................... 31 3.2.1 MKP Formulation .............................. 31 3.2.2 Heuristic Solution Approaches for MKP ................... 31 3.3 Multiple Choice Knapsack Problems (MCKP) .................... 33 3.3.1 MCKP Formulation ............................. 33 3.3.2 Heuristic Solution Approaches for MCKP .................. 34 3.4 Multiple-choice Multi-dimensional Knapsack Problems (MMKP) ......... 36 3.4.1 MMKP Formulation ............................. 36 3.4.2 Heuristic Solution Approaches for MMKP ................. 37 3.5 Applications and Formulations of the MMKP-type problems ............ 43 4 Legacy Heuristics and Test Problems Analysis 50 4.1 Introduction ...................................... 50 4.2 Legacy Heuristics ................................... 51 4.2.1 Legacy Heuristics for Multiple Knapsack Problems (MKP) ......... 51 4.2.2 Legacy Heuristics for Multiple Choice Knapsack Problems (MCKP) .... 55 iv 4.2.3 Legacy Heuristics for Multiple-choice Multi-dimensional Knapsack Prob- lems (MMKP) ................................ 58 4.3 Test Problem Analysis ................................ 69 4.3.1 Test Problem Analysis for Multiple Knapsack Problems (MKP) ...... 69 4.3.2 Test Problems for Multiple Choice Knapsack Problems (MCKP) ...... 71 4.3.3 Test Problems for Multiple-choice Multi-dimensional Knapsack Problems (MMKP) ................................... 74 4.4 Problem Structure Analysis of Test Problems .................... 76 4.4.1 Structure of MDKP Test Problems ...................... 76 4.4.2 Structure of MKP Test Problems ....................... 78 4.4.3 Structure of MCKP Test Problems ...................... 81 4.4.4 Structure of MMKP Test Problems ...................... 82 4.5 Summary ....................................... 84 5 Empirical Analyses of Legacy MMKP Heuristics and Test Problem Generation 90 5.1 Introduction ...................................... 90 5.2 Problem Generation and Problem Characteristics for MMKP ............ 91 5.2.1 Standard MMKP Test Problem Generation ................. 92 5.2.2 Analytical MMKP Test Problem Generation ................. 93 5.2.3 Competitive MMKP Test Problem Generation ................ 95 5.2.4 Analytical MMKP Test Sets Versus Available MMKP Test Set ....... 97 5.3 Empirical Analyses of MMKP Heuristics on Available Test Problems ....... 102 5.4 Empirical Analyses of MMKP Heuristics on New MMKP Test Problem Set .... 109 5.4.1 Analyses based on Constraint Right-Hand Side Setting ........... 109 5.4.2 Analyses based on Correlation Structure ................... 120 5.5 Summary ....................................... 125 6 New Greedy Heuristics for the MMKP 127 6.1 Introduction ...................................... 127 6.2 A TYPE-based Heuristic for the MMKP ....................... 127 6.3 New Greedy Heuristic Version 1 (CH1) ....................... 130 6.3.1 NG V3 Heuristic (Cho 2005) ......................... 130 6.3.2 CH1 Implementation ............................. 133 6.3.3 Empirical Tests for the CH1 Implementation ................ 134 6.4 New Greedy Heuristic Version 2 (CH2) ....................... 143 6.4.1 CH2 Implementation ............................. 143 6.4.2 Empirical Tests for the CH2 Implementation ................ 146 6.5 Summary ....................................... 152 7 Metaheuristic Solution Procedure for the MMKP 159 7.1 Introduction ...................................... 159 7.2 Concept of a Search Neighborhood .......................... 160 7.3 First-Level Tabu Search (FLTS) for the MMKP ................... 161 7.3.1 FLTS Implementation ............................ 161 7.3.2 Empirical Tests for the FLTS Implementation ................ 163 v 7.3.3 Extensions of the FLTS for the MMKP ................... 172 7.4 Sequential Fan Candidate List (FanTabu) for the MMKP .............. 172 7.4.1 FanTabu Implementation ........................... 175 7.4.2 Empirical Tests for the FanTabu Implementation .............. 177 7.5 CPCCP with Fan Candidate List (CCFT) for the MMKP .............. 188 7.5.1 CCFT Implementation ............................ 188 7.5.2 Empirical Tests for the CCFT Candidate List Implementation ....... 188 7.6 Comparison of TS approaches with Reactive Local Search Approach (RLS) .... 192 7.6.1 RLS Approach ................................ 192 7.6.2 Empirical Tests Comparing TS approaches with RLS Approach ...... 200 7.7 Summary ....................................... 201 8 Summary, Contributions, and Future Avenues 214 8.1 Summary and Contributions ............................