RULE DRIVEN JOB-SHOP SCHEDULING DERIVED FROM NEURAL NETWORKS THROUGH EXTRACTION A thesis presented to the faculty of the Fritz J. and Dolores H. Russ College of Engineering and Technology of Ohio University In partial fulfillment of the requirements for the degree Master of Science Chandrasekhar V. Ganduri August 2004 This thesis entitled RULE DRIVEN JOB-SHOP SCHEDULING DERIVED FROM NEURAL NETWORKS THROUGH EXTRACTION BY CHANDRASEKHAR V. GANDURI has been approved for the Department of Industrial and Manufacturing Systems Engineering and the Russ College of Engineering and Technology by Gary R. Weckman Associate Professor of Industrial & Manufacturing Systems Engineering R. Dennis Irwin Dean, Fritz J. and Dolores H. Russ College of Engineering and Technology GANDURI, CHANDRASEKHAR V. M.S. August 2004. Industrial and Manufacturing Systems Engineering Rule Driven Job-Shop Scheduling Derived from Neural Networks through Extraction (122 pp.) Director of Thesis: Gary Weckman This thesis focuses on the development of a rule-based scheduler, based on production rules derived from an artificial neural network performing job shop scheduling. This study constructs a hybrid intelligent model utilizing genetic algorithms for optimization and neural networks as learning tools. Genetic algorithms are used for obtaining optimal schedules and the neural network is trained on these schedules. Knowledge is extracted from the trained network as production rules using two rule extraction procedures: Validity Interval Analysis and Decision Tree Induction. The performance of this extracted rule set is compared to the performance of genetic algorithm, attribute-oriented induction data mining method, ID3 algorithm and simple dispatching rules in scheduling a test set of 6x6 scheduling instances. The capability of the rule-based scheduler in providing near optimal solutions is discussed. Approved: Gary Weckman Associate Professor of Industrial and Manufacturing Systems Engineering 4 TABLE OF CONTENTS LIST OF TABLES.............................................................................................................. 8 LIST OF FIGURES ............................................................................................................ 9 CHAPTER 1. INTRODUCTION ................................................................................ 10 1.1 Manufacturing Scheduling................................................................................ 10 1.2 Job Shop Scheduling Problem .......................................................................... 11 1.3 Previous Research............................................................................................. 13 1.4 Current Research............................................................................................... 14 1.5 Thesis Structure................................................................................................ 15 CHAPTER 2. SOFT COMPUTING METHODOLOGIES......................................... 17 2.1 What is Soft Computing?.................................................................................. 17 2.2 Genetic Algorithms........................................................................................... 19 2.2.1 Methodology of Genetic Algorithms............................................................ 20 2.2.2 Components of a Genetic Algorithm ............................................................ 21 2.2.3 Simple Genetic Algorithm Outline ............................................................... 23 2.3 Machine Learning............................................................................................. 24 2.3.1 Decision Tree Induction................................................................................ 25 2.3.2 Attribute-Oriented Induction........................................................................ 30 5 2.4 Artificial Neural Networks ............................................................................... 32 2.4.1 Neural Computation...................................................................................... 32 2.4.2 The Multi-Layer Perceptron Classifier ......................................................... 36 2.4.3 Neural-Network Training..............................................................................39 2.4.4 Generalization Considerations...................................................................... 40 2.5 Rule Extraction in Neural Networks................................................................. 42 2.5.1 The Rule-Extraction Task............................................................................. 42 2.5.2 Approaches to Rule Extraction ..................................................................... 44 2.5.3 Validity Interval Analysis............................................................................. 47 2.5.4 Extraction of Decision Tree Representations ............................................... 49 CHAPTER 3. APPROACHES TO THE JOB-SHOP SCHEDULING PROBLEM ... 52 3.1 The Classical Job Shop Scheduling Problem (JSSP)........................................ 52 3.1.1 Problem Formulation .................................................................................... 52 3.1.2 Types of Schedules ....................................................................................... 56 3.2 Review of Approaches to solve JSSP ............................................................... 58 3.2.1 Heuristics-based Approaches........................................................................ 59 3.2.2 Local Search Methods and Meta-Heuristics................................................. 61 3.2.3 Artificial Intelligence Approaches................................................................ 64 3.2.4 Machine Learning Applications.................................................................... 68 CHAPTER 4. METHODOLOGY ............................................................................... 71 6 4.1 The Learning Task ............................................................................................ 71 4.1.1 Genetic Algorithm (GA) Solutions............................................................... 71 4.1.2 Setting up the Classification Problem........................................................... 73 4.1.3 Development of a Neural Network Model.................................................... 76 4.2 Knowledge Extraction from the Neural Network Model ................................. 79 4.2.1 Decision Tree Induction................................................................................ 80 4.2.2 Propositional Rules by Validity Interval Analysis........................................ 87 CHAPTER 5. RESULTS AND DISCUSSION........................................................... 91 5.1 Performance of the 12-12-10-6 MLP Classifier ............................................... 91 5.2 Efficacy of the Rule Extraction Task................................................................ 93 5.3 Schedule Generation and Comparison.............................................................. 95 5.3.1 Statistical Analysis........................................................................................ 98 CHAPTER 6. CONCLUSIONS AND FUTURE RESEARCH ................................ 102 6.1 Conclusions..................................................................................................... 102 6.2 Future Research.............................................................................................. 104 REFERENCES ............................................................................................................... 106 APPENDIX A EVALUATION OF NN CLASSIFIERS .............................................. 113 APPENDIX B NETWORK PARAMATERS ............................................................... 114 7 APPENDIX C DECISION TREE INDUCTION DATASETS..................................... 116 APPENDIX D NN DECISION TREE EXTRACTION................................................ 118 APPENDIX E ID3 DECISION TREE INDUCTION ................................................... 119 APPENDIX F TEST SCHEDULING SCENARIOS .................................................... 122 8 LIST OF TABLES Table 2.1 Binary representations of chromosomes........................................................... 21 Table 2.2 Training set for the PlayTennis concept ........................................................... 26 Table 3.1 A 3 x 3 job-shop problem ................................................................................. 53 Table 4.1 The ft06 instance devised by Fisher and Thomson [88]................................... 72 Table 4.2 ProcessTime and RemainingTime feature classes............................................ 74 Table 4.3 MachineLoad feature classification.................................................................. 75 Table 4.4 Assignment of class labels to target feature...................................................... 76 Table 4.5 Sample data for the classification task.............................................................. 77 Table 4.6 Training parameters for the 12-12-10-6 MLP classifier................................... 79 Table 4.7 The rule set containing 48 rules (NN-Rule set) ................................................ 86 Table 5.1 Confusion matrix of the 12-12-10-6 MLP classifier ........................................ 91 Table 5.2 Comparison of classifiers.................................................................................