Anticipation vs. Reoptimization for Dynamic Vehicle Routing with Stochastic Requests Marlin W. Ulmer Abstract Due to new business models and technological advances, dynamic vehicle routing is gaining increasing interest. Solving dynamic vehicle routing problems is challenging, since it requires optimization in two directions. First, as a reaction to newly revealed information, current routing plans need to be reoptimized. Second, uncertain future changes of information need to be anticipated by explicitly quantifying the impact of future information and routing developments. Since customers or drivers usually wait for response, decisions need to be derived in real-time. This limited time often prohibits extensive optimization in both directions and the question arises how to utilize the limited calculation time effectively. In this paper, we compare the merit of route reoptimization and anticipation for a dynamic vehicle routing problem with stochastic requests. To this end, we present a policy allowing for a tunable combination of two existing approaches, each one aiming on optimization in one direction. We show that anticipation is beneficial in every case. We further reveal how the optimization direction is strongly connected to the degree of dynamism, the percentage of unknown requests. Keywords: Dynamic Vehicle Routing, Stochastic Requests, Degree of Dy- namism, Reoptimization, Anticipation, Mixed-Integer Programming, Ap- proximate Dynamic Programming 1 Introduction In a dynamic vehicle routing problem (DVRP), information is uncertain and re- vealed over time forcing subsequent adaptions of routing plans. Dynamic routing problems have become a center of attention in the research community driven by 1 technological advances and new business models (Ulmer 2016). Recent reviews are given by Pillac et al. (2013), Ritzinger et al. (2015), and Psaraftis et al. (2015). Dynamic vehicle routing problems are expected to be one main research field in city logistics (Savelsbergh and Van Woensel 2016). The literature reviews identify stochastic customer requests as the main driver of uncertainty in practice, e.g., for emergency vehicles, technical, healthcare, and courier services as well as parcel and passenger transportation in the growing fields of same-day delivery, shared mobility, or demand responsive transportation. Other examples of uncertainty are stochastic service times (Zhang et al. 2015), demands (Goodson et al. 2015), or travel times (Schilde et al. 2014). Even though routing applications with uncertain requests differ in their objec- tives and constraints, they all share the requirement of assigning new requests to vehicles and of determining efficient routing plans for the fleet. On the operational level, they often are limited in their accessible resources such as number of ve- hicles, working hours of drivers, inventory, vehicle’s capacities, etc. Most of the approaches aim at utilizing these limited resources to maximize revenue or the number of served customers. Since requesting customers often wait for responses, decision making needs to be conducted in real-time and the time for computational calculations is highly limited. 1.1 The Degree of Dynamism The aforementioned applications differ in their level of uncertainty, in particular, the question of how many customers are initially known and how many customers may stochastically request over the time horizon. Larsen et al. (2002) denotes the expected percentage of uncertain requests as the degree of dynamism (DOD). The DOD is defined by the ratio between expected number of initial customers EjC0j and new requests EjC+j: jC j DOD = E + : (1) EjC0j + EjC+j The DOD is one main dimension to classify DVRPs with stochastic requests. A moderate DODs may be experienced for applications such as oil distribution, patient transports, or grocery delivery (Ritzinger et al. 2014, Ehmke and Campbell 2014). The range of applications with high DODs comprises emergency vehicles or courier services (Maxwell et al. 2010, Thomas 2007) and can especially be found in emerging applications such as same-day delivery, demand-responsive transportation, or shared mobility (Voccia et al. 2016, Hall¨ et al. 2012, Brinkmann 2 et al. 2015). For a detailed classification of DVRP-applications, the interested reader is referred to Ulmer (2016). 1.2 Reoptimization vs. Anticipation As the literature reviews reveal, research on DVRPs usually focus on either reop- timization of the current routing plan or anticipation of future information and routing evolution. We describe these two directions in the following in detail. To this end, we draw on the Bellman Equation. First, we recall a solution for a dynamic decision problem as a decision policy π assigning a routing plan to every potential state of the problem. The optimal policy π∗ can be derived by following the Bellman Equation: K X π∗ max R(Sk; x) + E R(Si;X (Si))j(Sk; x) : (2) x2X (Sk) i=k+1 The Bellman Equation contains two parts. The first addend represents the im- mediate rewards or costs resulting from the current decision induced by the routing plan. The second addend represents the expected sum of future rewards or costs based on the current decision (and following decision policy π∗). Since the Curses of Dimensionality in DVRPs are usually tremendous, an optimal policy cannot be achieved for problems of practical interest and research draws on heuristic methods (Powell 2011). These heuristics either focus on reoptimization or anticipation and can be classified based on the Bellman Equation. Reoptimization addresses the first addend by updating the routing plan to maximize immediate reward or minimize immediate costs (Gendreau et al. 1999). Anticipation addresses the second addend by explicitly quantifying the expected future costs or rewards (Klapp et al. 2016). The two directions are depicted in Figure 1. The current state and routing plan is shown in the lower left. The black circle indicates the vehicle’s current location. On the y-axis, the improvement of the current routing plan with respect to reoptimization is shown representing the first term of the Bellman Equation. The efficiency of the routing plan increases from the coarse plan in the lower left to the most efficient in the upper left of Figure 1. On the x-axis, the potential temporal evolution of routing, information, and route updates is depicted in a simplified way. The development shows how the vehicle executes the route which may be frequently updated according to newly revealed information. Anticipation is now the explicit quantification of the x-axis in the determination of an effective routing plan represented by the second term of the Bellman Equation. The calculation 3 Ca lcu la tio n Lim it Reoptimization of Routing Plans Anticipation of Information and Routing Plan Evolution Figure 1: Reoptimization and Anticipation limit is indicated by the diagonal line. As aforementioned, extensive real-time calculations in both dimensions are prohibited. Due to the comprehensive knowledge based on the static and deterministic routing literature, many solution approaches focus on the y-axis of Figure 1 re- optimizing routing plans on a rolling horizon based on the currently available information. Typically, the x-axis is neglected. Still, with advances in informa- tion and communication technology stochastic information about potential future developments becomes accessible. Gendreau et al. (2016) sees the informational process as “an important dimension to consider” in the optimization decisions. As Speranza (2016) and Savelsbergh and Van Woensel (2016) state, anticipation, hence, the consideration of the x-axis by integrating both stochastic information and future adaptions, is a major challenge in DVRP-research. Anticipatory solution approaches often draw on simulation of future information and routing develop- ment to evaluate the current state’s decision. Still, due to the limited calculation 4 time, they usually base on routing heuristics and typically neglect the y-axis, hence, routing reoptimization. Both directions have therefore advantages and shortcom- ings. Reoptimization may free valuable resources allowing for more efficient plans while anticipation may incorporate important future developments in current decision making allowing for more effective routing and assignment decisions. 1.3 Contributions The main purpose of this paper is the analysis from which direction DVRPs may be approached with respect to their DODs. To this end, we present an approach combining two existing solution approaches from each direction: reoptimization by means of mixed-integer programming and anticipation based on approximate dynamic programming. This new approach is tunable to allow shifting the focus between the two directions along the line of computational limitation depicted in Figure 1. To analyze the merits of both research directions, we run a computational evaluation on the vehicle routing problem with stochastic requests (VRPSR) and a variety of instance settings mimicking the conditions of different practical applica- tions represented by varying DODs. As we show, there is no dominating research direction. Still, the optimization focus should depend on the degree of uncertainty. A low DOD demands for an emphasis on reoptimization while moderate and high DODs require significant effort in anticipation. We not only show that both direc- tions are valuable and a combination is essential to achieve effective and efficient decision policies. We also show how they should be combined under