JOHANNES KEPLER UNIVERSITAT¨ LINZ JKU Technisch-Naturwissenschaftliche Fakult¨at Adaptive Heuristic Approaches for Dynamic Vehicle Routing - Algorithmic and Practical Aspects DISSERTATION zur Erlangung des akademischen Grades Doktor im Doktoratsstudium der Technischen Wissenschaften Eingereicht von: Stefan Vonolfen, MSc Angefertigt am: Institut f¨ur Formale Modelle und Verifikation Beurteilung: Priv.-Doz. Dr. Michael Affenzeller (Betreuung) Univ.-Prof. Dr. Karl D¨orner Linz, Juni, 2014 Acknowledgments The work presented in this thesis would not have been possible without the many fruitful discussions with my colleagues from the research group Heuris- tic and Evolutionary Algorithms Laboratory (HEAL) and without the Heuris- ticLab optimization environment as a software infrastructure (the web pages of all members of HEAL as well as further information about HeuristicLab can be found at: http://www.heuristiclab.com). Particularly, I would like to thank Michael Affenzeller for his guidance concerning the algorithmic aspects of this thesis as my supervisor as well as providing a very supportive working environment as the research group head. I would also like to thank Stefan Wagner for triggering my research interest in metaheuristic algorithms during the supervision of my bachelor and master thesis. The discussions with my colleagues Andreas Beham and Erik Pitzer about vehicle routing, fitness landscape analysis, and algorithm selection led to many ideas presented in this thesis. Michael Kommenda provided important insights on genetic programming and Monika Kofler contributed knowledge about storage assignment. Stephan Hutterer was working on the generation of policies for smart grids and pointed out many links to related literature. I would also like to thank Prof. Karl D¨orner from the institute for pro- duction and logistics management at the JKU for giving me the possibility to present my work at the ORP3 workshop as well as during a seminar at his institute. The workshop provided the possibility to submit my work to a renowned international operations research journal (the article is currently in third revision and is based on topics presented in this thesis). The discus- sions at the seminar and the workshop as well as the review comments on the journal article provided many valuable insights about algorithmic extensions and modeling aspects. Last, but not least I would also like to acknowledge the received funding. The work described in this thesis was done within the Josef Ressel Centre for Heuristic Optimization supported by the Austrian Research Promotion Agency (FFG) as well as the Regio 13 program sponsored by the European Regional Development Fund and by Upper Austrian public funds. Eidesstattliche Erkl¨arung Ich erkl¨are an Eides statt, dass ich die vorliegende Dissertation selbstst¨andig und ohne fremde Hilfe verfasst, andere als die angegebenen Quellen und Hilfsmittel nicht benutzt bzw. die w¨ortlich oder sinngem¨aß entnommenen Stellen als solche kenntlich gemacht habe. Die vorliegende Dissertation ist mit dem elektronisch ubermittelten¨ Textdokument identisch. Zusammenfassung Die dynamische Tourenplanung gewinnt im Bereich der Logistikoptimierung mehr und mehr an Bedeutung. Die Entwicklung von immer effizienteren Opti- mierungalgorithmen sowie technologische Verbesserungen von Telematiksys- temen erm¨oglichen den Einsatz von praxisnahen Modellen in Unternehmen. Bei der Entwicklung von Optimierungsverfahren wird allerdings oftmals ein Kompromiss zwischen breiter Anwendbarkeit und Spezialisierung ge- macht. Hochspezialisierte L¨osungsstrategien funktionieren in gewissen Situa- tionen zwar besser, machen jedoch Abstriche im Hinblick auf Robustheit. Aus dieser Beobachtung heraus ergibt sich die in dieser Arbeit verfolgte Vi- sion eines adaptiven Entscheidungs-Unterstutzungssystems¨ fur¨ dynamische Tourenplanungsumgebungen mit sich ¨andernden Problemcharakteristiken. Eine laufende Anpassung der L¨osungsstrategien erfordert die Verlagerung der Algorithmenentwicklung auf eine h¨ohere Abstraktionsebene. Eine Meta- Betrachtungsweise von Algorithmen erlaubt eine semi-automatische Generie- rung von spezialisierten L¨osungsstrategien als auch eine adaptive Algorith- menauswahl auf Basis von Problemcharakteristiken. Auf diese Weise werden die St¨arken von verschiedenen spezialisierten Algorithmen in sich ¨andernden Problemumgebungen kombiniert. Aufbauend auf diesen Konzepten wird ein algorithmisches Rahmenwerk vorgestellt, welches aus drei grundlegenden Bausteinen besteht. Die simula- tionsbasierte Optimierung erlaubt die Erstellung von praxisnahen Modellen mit stochastischen Einflussgr¨oßen und komplexen dynamischen Interaktio- nen. Auf Basis dieser Modelle werden unter Anwendung von Evolution¨aren Algorithmen und best¨arkendem Lernen spezialisierte L¨osungsstrategien ge- sucht. Diese werden zu Algorithmenportfolios kombiniert, welche eine situa- tive Auswahl anhand der Problemcharakteristiken erm¨oglichen. Die bedeutendste Leistung dieser Arbeit ist die semi-automatische Gene- rierung und Adaption von algorithmischen Strategien, was durch die Verlage- rung der Algorithmenentwicklung auf eine h¨ohere Abstraktionsebene erreicht wird. Zukunftige¨ Entwicklungen, wie die Integration von maschinellem Ler- nen oder die Erforschung der L¨osungsraumcharakteristiken, stehen vor allem im Kontext von autonom agierenden adaptiven Tourenplanungssystemen. Das vorgestellte algorithmische Rahmenwerk bietet hierfur¨ eine Grundlage. Abstract Dynamic vehicle routing is an active field of research due to the practical relevance as well as the advances in operations research and telematics. Ad- vanced algorithmic approaches are being developed and at the same time, more and more realistic problem formulations are being investigated enabling to transfer the findings into practice. The main vision pursued in this thesis is a decision support system for dynamic vehicle routing problems that is adaptive in terms of problem char- acteristics and automatically changes its algorithmic strategies based on the environment. The motivation for such a system stems from the fact that there is a tradeoff between generalization and specialization in algorithm de- sign. On the one hand, research on algorithms for dynamic vehicle routing problems focused mainly on robust behavior over a large range of problem instances. On the other hand, it has been identified that highly special- ized policies have the potential to outperform these general strategies while non-robust behavior was observed for them as a trade-off. The methodological developments presented in this thesis aim to solve this dilemma by raising the abstraction level of algorithm design for dynamic vehicle routing problems to a meta-level to pursue the goal of self-adaptive algorithmic strategies. On the meta-level a semi-automatic generation as well as an adaptive selection of policies based on the problem characteristics is performed combining the strengths of several specialized policies in changing environments. Three essential building blocks of an adaptive algorithmic framework for dynamic vehicle routing are identified. Simulation optimization allows modeling practical variants with rich side constraints. Three practical case- studies from production and logistics are investigated while highlighting the transfer of findings into practice. Based on a simulation model, specialized routing policies are generated by means of direct policy search and rein- forcement learning. Routing policies for three different variants are evolved. Human-designed as well as generated routing policies are combined to an algorithm portfolio allowing a dynamic situational policy selection. A case- study is presented illustrating the potential of the methodology. The main achievement of this thesis is raising the abstraction level for algorithm design in the context of dynamic vehicle routing by means of the proposed algorithmic framework. While the semi-automated adaption of the algorithmic strategies has been reached, future research should focus on a fully autonomous system that learns in a changing environment. Such a system requires the incorporation of machine learning and a fundamental understanding of problem characteristics linked to algorithm performance. Contents 1 Introduction 1 1.1 MotivationandResearchQuestions . 1 1.2 Synopsis.............................. 3 1.3 ChapterOverview......................... 5 2 Dynamic Vehicle Routing 7 2.1 Foundations............................ 7 2.1.1 TheVehicleRoutingProblem . 7 2.1.2 From Static to Dynamic Vehicle Routing . 18 2.1.3 DynamicallyArrivingInformation . 20 2.1.4 Objectives and Performance Evaluation . 24 2.1.5 Categorization and Application Areas . 26 2.2 SolutionMethods ......................... 28 2.2.1 DynamicPolicies . 29 2.2.2 AdaptionofStaticAlgorithms . 31 2.2.3 ConsideringFutureEvents . 33 2.2.4 Parallelization. 35 2.3 Technical Requirements and Practical Implementation . ... 37 2.3.1 TechnicalComponents . 37 2.3.2 DecisionSupportSystems . 38 2.4 ResearchDirections. 39 3 Simulation-Based Optimization of Production and Logistic Scenarios 41 3.1 SimulationOptimizationEnvironment . 41 3.1.1 Generic Simulation Optimization Core . 43 3.1.2 Specializations . 46 3.2 Optimization of Transport Activities in Steel Production ... 50 3.2.1 ContextandMotivation . 50 3.2.2 SimulationModel . 53 3.2.3 OptimizationApproach. 55 3.2.4 Conclusions ........................ 58 3.3 Integrated Warehousing and In-House Transport ............................. 60 3.3.1 ContextandMotivation