The Pennsylvania State University The Graduate School Department of Industrial and Manufacturing Engineering AUGUMENTED SIMULTANEOUS PERTURBATION STOCHASTIC APPROXIMATION (ASPSA) FOR DISCRETE SUPPLY CHAIN INVENTORY OPTIMIZATION PROBLEMS A Thesis in Industrial Engineering and Operations Research by Liya Wang 2006 Liya Wang Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy December 2006 The thesis of Liya Wang was reviewed and approved* by the following: Vittal Prabhu Associate Professor of Industrial and Manufacturing Engineering Thesis Advisor Chair of Committee A. Ravi Ravindran Professor of Industrial Engineering and Affiliate Professor of IST Ling Rothrock Assistant Professor of Industrial and Manufacturing Engineering Hong Xu Professor of Management Science and Supply Chain Management Richard J. Koubek Professor of Industrial Engineering Head of the Department of Industrial and Manufacturing Engineering *Signatures are on file in the Graduate School iii ABSTRACT In recent years, simulation optimization has attracted a lot of attention because simulation can model the real systems in fidelity and capture the dynamics of the systems. Simultaneous Perturbation Stochastic Approximation (SPSA) is a simulation optimization algorithm that has attracted considerable attention because of its simplicity and efficiency. SPSA performs well for many problems but does not converge for some. This research proposes Augmented Spontaneous Perturbation Stochastic Approximation (ASPSA) algorithm in which SPSA is extended to include presearch, ordinal optimization, non-uniform gain, and line search. Extensive tests show that ASPSA achieves speedup and improves solution quality. ASPSA is also shown to converge. For unconstrained problems ASPSA uses random presearch whereas for constrained problems presearch search is used to find a feasible solution, thereby extending the gradient based approach. Performance of ASPSA is tested for supply chain inventory optimization problems including serial and fork-join supply chain without constraints and fork-join supply chain network with customer service level constraints. To evaluated performance of ASPSA, a naïve implementation of Genetic Algorithm is used to primarily test solution quality and indicate computation effort. Experiments show that ASPSA is comparable to Genetic Algorithms (GAs) in solution quality (worst case 16.67%) but is much more efficient computationally (12x faster). iv TABLE OF CONTENTS LIST OF FIGURES .....................................................................................................vii LIST OF TABLES.......................................................................................................ix ACKNOWLEDGEMENTS.........................................................................................xi Chapter 1 Introduction ................................................................................................1 Chapter 2 A Parallel Algorithm for Setting WIP Levels for Multi-product CONWIP Systems ................................................................................................9 2.1 Introduction.....................................................................................................9 2.2 Problem Formulation and Properties..............................................................12 2.3 Algorithms ......................................................................................................15 2.3.1 Sequential Algorithm............................................................................16 2.3.2 Parallel algorithm .................................................................................17 2.4 Numerical Results...........................................................................................19 2.5 Conclusion ......................................................................................................25 Chapter 3 Literature Review.......................................................................................27 3.1 Simulation Optimization.................................................................................27 3.1.1 Gradient Based Methods ......................................................................30 3.1.1.1 Stochastic Approximation (SA) .................................................30 3.1.1.1.1 Finite Differences.............................................................31 3.1.1.1.2 Simultaneous Perturbation Stochastic Approximation (SPSA) .....................................................................................33 3.1.1.2 Likelihood Ratios (LR) ..............................................................36 3.1.1.3 Perturbation Analysis (PA) ........................................................37 3.1.1.4 Frequency Domain Method (FDM) ...........................................37 3.1.2 Response Surface Methodology (RSM)...............................................38 3.1.3 Statistical Methods ...............................................................................40 3.1.3.1 Importance Sampling Methods ..................................................40 3.1.3.2 Ranking and Selection................................................................41 3.1.3.3 Sample Path Optimization..........................................................43 3.1.3.4 Ordinal Optimization (OO) ........................................................43 3.1.4 Heuristic Methods ................................................................................45 3.1.4.1 Random Search Method.............................................................45 3.1.4.2 Evolutionary Algorithms (EAs) .................................................46 3.1.4.3 Simulated Annealing..................................................................51 3.1.4.4 Tabu Search................................................................................52 3.1.5 A-Team.................................................................................................54 3.1.6 Optimization for Simulation Software .................................................54 v 3.2 Supply Chain Management.............................................................................55 3.2.1 Supply Chain Inventory Replenishment Policy ...................................56 3.2.2 Literature Review of SCM ...................................................................57 Chapter 4 Research Methodology...............................................................................61 4.1 SPSA...............................................................................................................62 4.1.1 Basic Algorithm....................................................................................62 4.1.2 Relationship of Gradient Estimate to True Gradient............................64 4.2 Augmented SPSA (ASPSA-I) For Unconstrained Optimization Problems ...66 4.2.1 Ordinal Optimization............................................................................67 4.2.2 Presearch...............................................................................................70 4.2.3 Non-uniform Step Size.........................................................................71 4.2.4 Line Search...........................................................................................72 4.2.5 Long Term Memory .............................................................................73 4.3 Convergence Proof of ASPSA-I Algorithm ...................................................75 Chapter 5 Apply ASPSA-I on Unconstrained Discrete Supply Chain Inventory Optimization Problems .........................................................................................77 5.1 Supply Chain Model.......................................................................................77 5.1.1 Model Assumptions..............................................................................78 5.1.2 Inventory Control Policy: a Base-stock Policy ....................................78 5.1.3 Measure of Performance.......................................................................79 5.1.4 Model Description................................................................................80 5.2 Multi-agent Based Supply Chain Simulation .................................................82 5.3 Calculation of Upper and Lower Bound for Base Levels...............................86 5.4 Experiment Design .........................................................................................87 5.4.1 Supply Chain Settings ..........................................................................87 5.4.2 Simulation Based Evaluation of TSCC for a Given Set of Base- stock Levels............................................................................................88 5.4.3 Performance Evaluation .......................................................................88 5.5 Solution Methodologies under Evaluation .....................................................90 5.5.1 ASPSA-I algorithm ..............................................................................90 5.5.2 SPSA algorithm....................................................................................91 5.5.3 Genetic Algorithm (GA).......................................................................91 5.6 Complete Enumeration (CE) ..........................................................................92 5.7 Results and Discussions..................................................................................92