This article was downloaded by: [18.172.5.91] On: 05 August 2016, At: 07:10 Publisher: Institute for Operations Research and the Management Sciences (INFORMS) INFORMS is located in Maryland, USA
Transportation Science
Publication details, including instructions for authors and subscription information: http://pubsonline.informs.org Revenue Management in a Dynamic Network Environment
Dimitris Bertsimas, Ioana Popescu,
To cite this article: Dimitris Bertsimas, Ioana Popescu, (2003) Revenue Management in a Dynamic Network Environment. Transportation Science 37(3):257-277. http://dx.doi.org/10.1287/trsc.37.3.257.16047
Full terms and conditions of use: http://pubsonline.informs.org/page/terms-and-conditions
This article may be used only for the purposes of research, teaching, and/or private study. Commercial use or systematic downloading (by robots or other automatic processes) is prohibited without explicit Publisher approval, unless otherwise noted. For more information, contact [email protected]. The Publisher does not warrant or guarantee the article’s accuracy, completeness, merchantability, fitness for a particular purpose, or non-infringement. Descriptions of, or references to, products or publications, or inclusion of an advertisement in this article, neither constitutes nor implies a guarantee, endorsement, or support of claims made of that product, publication, or service. © 2003 INFORMS
Please scroll down for article—it is on subsequent pages
INFORMS is the largest professional society in the world for professionals in the fields of operations research, management science, and analytics. For more information on INFORMS, its publications, membership, or meetings visit http://www.informs.org Revenue Management in a Dynamic Network Environment
Dimitris Bertsimas • Ioana Popescu Sloan School of Management, MIT, Cambridge, Massachusetts 02139 INSEAD, Fontainebleau Cedex 77305, France [email protected] • [email protected]
e investigate dynamic policies for allocating scarce inventory to stochastic demand for Wmultiple fare classes,in a network environment so as to maximize total expected rev- enues. Typical applications include sequential reservations for an airline network,hotel,or car rental service. We propose and analyze a new algorithm based on approximate dynamic programming,both theoretically and computationally. This algorithm uses adaptive,nonad- ditive bid prices from a linear programming relaxation. We provide computational results that give insight into the performance of the new algorithm and the widely used bid-price control,for several networks and demand scenarios. We extend the proposed algorithm to handle cancellations and no-shows by incorporating oversales decisions in the underlying linear programming formulation. We report encouraging computational results that show that the new algorithm leads to higher revenues and more robust performance than bid-price control.
Introduction policies for allocating inventory to correlated,stochas- Capacity constrained service industries,such as trans- tic demand for multiple classes,in a network environ- portation,tourism,entertainment,media,and internet ment. Specifically,we design a decision support tool, providers are constantly faced with the problem of based on stochastic and dynamic optimization tech- intelligently allocating their limited,perishable inven- niques,that at each point in time accepts or rejects a tories to demand from different market segments, reservation request,based on the currently available with the objective of maximizing total revenues. Rev- inventory,past sales and future potential demand,so enue management is concerned with the theory and as to maximize total expected revenues. practice underlying this type of problem. Following airline deregulation,revenue management techniques Problem Definition have had an important impact on the development of the industry,providing up to 4%–10% increases The main problem we address in this paper is as fol- in company revenues (Fuchs 1987). For example,in lows. We are given an airline (hotel,car rental) net- 1997,American Airlines collected one billion dollars work composed of l legs (pairs of consecutive days for hotels,car rentals),which are used to serve a total
Downloaded from informs.org by [18.172.5.91] on 05 August 2016, at 07:10 . For personal use only, all rights reserved. by implementing revenue management,representing most of the company’s profit (Cook 1998). of m demand classes. The initial inventory is given by
Optimization techniques have been essential in the a vector N = N1 Nl of leg capacities. The net- development of revenue management tools,particu- work is described by a l ×m matrix A and a m-vector
larly for seat allocation models. In this research,we R = R1 Rm: Rj is the fare category of class j,
are interested in investigating the design of dynamic which utilizes aij units of resource (leg) i. In this way,
0041-1655/03/3703/0257$05.00 Transportation Science © 2003 INFORMS 1526-5447 electronic ISSN Vol. 37,No. 3,August 2003,pp. 257–277 BERTSIMAS AND POPESCU Revenue Management in a Dynamic Network Environment
a demand class j is defined by its itinerary Aj (a col- does not matter). Usually,we have partial information
umn of matrix A) and its fare category Rj . In an airline about the demand process,which might consist of the network without group discounts, A is a 0–1 matrix expected demand to come Dt = Et,and possibly which may contain repeated columns for each fare other type of information available from forecasting class on a given itinerary. To account for special group tools,such as cancellation or no-show probabilities. fares,integer multiples of the itinerary-incidence vec- The state of the system S is given by the time t tor are allowed. (t periods to departure) and the sales-to-date record For example,consider a very simple network cor- s = s1 sm for each demand class. If cancellations responding to a weekend in a hotel: There are three and no-shows are not allowed,then it is sufficient to nodes Fri, Sat, Sun and l = 2 legs (1) Fri–Sat and define the state based on the remaining inventory n = (2) Sat–Sun with total capacities N = N N . Sup- 1 2 n n. A common practice is to use the latter pose there is demand for all types of stays,i.e., 1 l model and account for cancellations and no shows by “itineraries:” (1) Fri–Sat,(2) Sat–Sun,and (3) Fri–Sat– incorporating a virtual increase,called overbooking Sun,with two ( high and low) fare classes for each type. pads,in the initial capacity definition. Moreover,suppose there are discounts for groups of 10 The general stochastic,dynamic inventory control size k1 = 10 for (1) Fri–Sat night stays,at a rate of R1 per group,that is a total of m = 7 classes. The leg- problem for network revenue management (NRM) class incidence matrix,together with the correspond- can thus be stated as follows: At time t,and given ing fare structure R,is given by: that the state of the network is S,should we accept or reject a new class j request? The overall objective 10 Rh Rl Rh Rl Rh Rl R is to maximize total expected revenues. 1 1 2 2 3 3 1 R = 110011 10 The decision to accept or reject determines an A admission control policy,which is in general a func- 001111 0 tion of the current network configuration (s or n),
We assume a finite booking horizon of length T , the time-to-go t,the currently requested fare Rj ,as with the time line sufficiently discretized so as to well as partial information from demand forecasts. If allow at most one request (reservation or cancellation) we accept the request,the new state becomes s +
per time period almost surely (a.s.). Time is counted kj · ej t− 1,where kj is the size of the class j group
backwards: Time t = T is the beginning of the book- request,and ej is the jth unit vector; when can- ing horizon and time t = 0 is the end of the reserva- cellations are not allowed,the state of the network tion period,where the no-shows are being counted. nt becomes simply n − Aj t. If the request is Customers who do not consume their reservations get rejected,only the time component of the state vector full,partial,or no refund,depending on their fare is changed to t − 1. class. If at the end of the horizon,the inventory is In reality,the control policy has an impact on oversold,at time t =−1 redistribution decisions are the demand process,because customer choice may being made. These can be class upgrades or customer depend on the opportunity set. Capturing and quan- bumping,in which case companies pay overbooking tifying this type of feedback phenomenon is a sub- penalties. For the case of hotels and car rentals,an tle task that goes beyond the scope of this work. We infinite time horizon could be more appropriate; how- will thus make the simplifying assumption that the ever,one can decompose the problem in fixed length demand process is independent of the control policy.
Downloaded from informs.org by [18.172.5.91] on 05 August 2016, at 07:10 . For personal use only, all rights reserved. time periods (one month,one year,etc.). The demand (to come) process at time t is denoted by t,and t represents the corresponding ran- Notation t We will use the following notation throughout the dom vector of cumulative demands. That is, j is a random variable representing the number of class j paper. Vectors will be denoted in bold,and random requests to come from time t until departure (order variables and processes in calygrahic style.
258 Transportation Science/Vol. 37,No. 3,August 2003 BERTSIMAS AND POPESCU Revenue Management in a Dynamic Network Environment
Time: NRM problem. To compare these different policies, T = length of time horizon (number of time peri- we provide structural and computational results. ods); The most popular technique developed in the cur- t = time periods left until departure (count-down). rent literature is an additive bid-pricing approach. Network: These are mechanisms whereby the opportunity cost l = number of legs in the network; of each itinerary is estimated as the sum of the m = number of classes (itineraries with fare cate- shadow prices of the incident legs,obtained from a gories); linear programming formulation of the problem (see N = total initial network capacity (l-vector); Formulation (2) in §2.2). However,there are two obvi- A = leg-class incidence l × m-matrix; ous drawbacks to additive bid prices: aij = quantity of resource i utilized by bundle j. (a) They are not well defined if there are multiple Sales and inventory: dual solutions. n = remaining inventory (l-vector); (b) They are restrictive in their way of taking into s = sales to date vector (m-vector); account bundles by their predefined additive struc- t sj = the number of itineraries sold to class j until ture. In particular,they do not account for changes of time t; a dual basis in response to accepting large-group and so = the number of class j itineraries overbooked at j multileg itinerary requests. departure. The contributions of this paper are as follows: Fares,refunds,and penalties: (1) We propose an efficient control policy,that is R = revenue collected for one class j sold; j well defined if there are multiple dual solutions, Rc = the refund per class j cancellation; j and does not have an additive structure. The pro- Rns = the refund per class j no-show; j posed control policy,which we call certainty equiva- C = the overbooking penalty per demand class j. j lent control (CEC),belongs in the class of approximate Demand: dynamic programming mechanisms (see Bertsekas pt = probability of request for class j at time t; j and Tsitsiklis 1998),in which the cost-to-go function is pct = the probability of a class j cancellation occur- j approximated by the value of a linear programming ring at time t; (LP) relaxation. We remark that this LP is equivalent pt = the probability of no request (reservation or 0 to a network flow problem in the case of origin- cancellation) at time t; destination (OD) demands or linear networks (where pns = the probability that a class j reservation will j nodes can be ordered so that the arcs are pairs of con- not show up; secutive nodes ii + 1 i = 1 l). t = demand (to come) process (m-dimensional); t = aggregate demand (to come) distribution (2) We provide structural properties that com- (m-dimensional); pare the behavior of the proposed CEC policy with Dt = Et = expected aggregate demand to come the additive bid-price approach. These results offer (m-dimensional); insight into the behavior of both methods. = the cancellation process,adapted to the sales (3) We propose several algorithmic improvements history (not a decision); of the CEC policy based on approximate dynamic = the no-show distribution (adapted to final programming. sales,adjusted for cancellations). (4) We provide computational results that give We will use the operator x+ = maxx0,for insight into the performance of these algorithms and + Downloaded from informs.org by [18.172.5.91] on 05 August 2016, at 07:10 . For personal use only, all rights reserved. x ∈ R,which naturally extends for vectors: x = several variations,for different networks and demand + + n scenarios. We observe that the CEC algorithm per- x1 xn for x = x1 xn ∈ R . forms very well in practice,giving results that are Contributions very close to optimum. For high load factors,we In this paper,we evaluate from different perspectives observe an average 5%–10% improvement over exist- several policies for solving the dynamic and stochastic ing policies (additive bid pricing). We describe and
Transportation Science/Vol. 37,No. 3,August 2003 259 BERTSIMAS AND POPESCU Revenue Management in a Dynamic Network Environment
simulate extensions of this algorithm that result in sig- and Stone (1991). An analogous discrete time solu- nificantly higher improvements (up to 20%). Interest- tion is provided by Lee and Hersh (1993),who ingly,the CEC policy appears to be significantly more also provide a method for estimating arrival rates. robust to noise and bias in the demand forecast. Subramanian et al. (1999) propose a dynamic pro- (5) We extend these algorithms to handle cancel- gramming model that handles cancellations and over- lations and no-shows by incorporating overbooking booking,by analogy to a problem in the optimal control in the underlying mathematical programming control of admission in a queueing system. formulation. This extension preserves several struc- A natural,but much harder question is to deter- tural properties. We report computational results that mine which of the single leg results can be extended show that the proposed algorithm improves upon the to network (multiproduct) settings,and how. A major performance of the bid-price control policy. conceptual advance in the study and practice of net- work revenue management was introduced by bid- Structure price control. These are additive,leg-based shadow The remainder of the paper is organized as follows: In prices used to approximate the opportunity cost of the next section,we present an overview of the litera- itinerary capacity. The concept was proposed by ture. Section 2 describes several (dynamic,stochastic, Simpson (1989),and further analyzed by Williamson linear,and network flow) models and formulations (1992) in extensive simulation studies. A deficiency of for the NRM problem and evaluates the relationships such mathematical programming based models is that between them. Section 3 presents efficient algorithms they do not account for nesting of the fare classes. for the NRM problem based on ideas from approxi- To overcome this problem,Curry (1992) proposes a mate dynamic programming. In §4,we present struc- virtual nesting method. In all cases,however,the allo- tural properties for the proposed CEC policy and cation of capacity to itinerary demand is decided by contrast them with additive bid pricing. In §5,we extend our model and algorithms to handle cancella- a one-time,static rule (one fixed set of bid prices). tions,no-shows,and overbooking. Finally,in §6,we The most realistic and relevant,yet least investi- present computational results. The last section sum- gated model for NRM is the dynamic network model. marizes our conclusions. Talluri and van Ryzin (1998) study a dynamic net- work model using bid-price control mechanisms and argue why bid-price policies are not optimal in gen- 1.Literature Review eral. They provide an asymptotic regime when certain and Positioning bid-price controls,based on a probabilistic program- ming formulation of the problem,are asymptoti- The NRM problem can be viewed as a particular cally optimal. In the context of hotels,Bitran and instance of the general class of perishable asset rev- enue management problems (PARM). Initial devel- Mondschein (1995) propose a dynamic policy that opments in static single leg revenue management extends to multiple-night stays,but do not give any are due to Littlewood (1972),followed by Simpson further analysis. (1989) and Belobaba (1987),who proposed a subop- Chen et al. (2000) formulate the problem as a timal policy for computing protection levels based Markov decision problem,and use linear program- on expected marginal seat revenues (EMSR). Curry ming and regression splines to approximate the value function. Gunther et al. (2000) introduce a new
Downloaded from informs.org by [18.172.5.91] on 05 August 2016, at 07:10 . For personal use only, all rights reserved. (1989),Wollmer (1992),and Brumelle and McGill (1993),derive the optimal solution for the single leg method to compute bid price for single hub airline static model. Robinson (1995) proposes an extension networks. Both studies report encouraging simulation that handles nonmonotonic fare classes. results. None of these take into account cancellations. A characterization of the optimal dynamic policy For further pointers to related literature,we refer based on a threshold time property is due to Diamond the interested reader the survey of McGill and
260 Transportation Science/Vol. 37,No. 3,August 2003 BERTSIMAS AND POPESCU Revenue Management in a Dynamic Network Environment
van Ryzin (1999),and for a schematic summary the demands,this can be computed via the Bellman equa- Ph.D. thesis of Popescu (1999). tion as follows:
Relative Positioning. Our approach can be viewed DPnt as extending the single-leg models investigated by m t j Lee and Hersh (1993) and Bitran and Mondschein = pj maxDPnt− 1 Rj + DPn − A t− 1 (1995),who actually propose a similar LP-based j=1 heuristic for the multiple-night booking problem in t + p0DPnt− 1 a hotel,but without any further analysis. Our over- m t booking model is similar to the early single-leg DP = DPnt− 1 + pj Rj − DPnt− 1 formulation of Rothstein (1971,1974). As far as static j=1 network models are concerned,our LP formulation is + DPn − Aj t− 1 + (1) similar to the one proposed by Williamson (1992),but we handle multiple classes and group bookings. The for all n ≤ N , t ≤ T ,with the boundary conditions: network flow formulation we provide for linear net- DPnt=− if n < 0 for some i and works and origin-destination fare (ODF) demands,is i the same as those proposed by Glover et al. (1982) for DPn 0 = 0 for n ≥ 0 airlines and Chen (1998) for hotels.
We define the opportunity cost OCj nt to be
OC nt= DPnt− 1 − DPn − Aj t− 1 2.General Models j and Relationships Then,the optimal policy accepts a request if and only
The problem of dynamic inventory control for NRM if the corresponding fare Rj exceeds its current oppor- belongs in the class of finite horizon decision prob- tunity cost. The value function can thus be expressed lems under uncertainty. In this section,we present as follows: several optimization models for addressing the NRM m t + problem,and discuss various relationships between DPnt= DPnt− 1 + pj Rj − OCj nt them. Again,we assume that the control policy does j=1 not feed back into the evolution of the demand pro- One can easily show that the value function is non- cess. We restrict to the case when cancellations and decreasing in n and t. Moreover,for the single leg overbooking are not permitted; this case is discussed case without batch arrivals,the value function is con- in detail in §5. cave and opportunity costs decrease with t and n (see Diamond and Stone 1991). Based on these properties, 2.1. The Dynamic Programming Model one can show that the optimal policy is characterized The stochastic dynamic programming model provides by threshold times. Threshold times are points in time the optimal policy for the NRM problem,by evaluat- during the booking horizon before which requests are ing the whole tree of possibilities and making at each rejected,and after which requests are accepted. In point in time the decision (to sell or not to sell) that the general network case,however,this is not true,as would imply higher future expected revenues. we will show in an example in §4.1. The states S = nt are defined by the current Downloaded from informs.org by [18.172.5.91] on 05 August 2016, at 07:10 . For personal use only, all rights reserved. available capacity vector n when the remaining time 2.2. The Integer and Linear Programming is t. The stochasticity is given by the demand process Model (IP, LP) to come t for the remaining t periods. We define The most frequently utilized formulations for the DPnt to be the maximum expected revenue to NRM problem are static models. These are determin- be collected from state nt. Assuming independent istic analogues of the stochastic dynamic problem,
Transportation Science/Vol. 37,No. 3,August 2003 261 BERTSIMAS AND POPESCU Revenue Management in a Dynamic Network Environment
that use only expected demand information,and are Proposition 1. DPnt≤ LPn Dt. usually much simpler to solve. For a full proof see Popescu (1999). The idea is Suppose that all the available demand information that for each pathwise realization of demand,the rev- consists of (unbiased) forecasts of the expected aggre- enue collected by the DP-policy along that path is gate demand to come Dt = Et over the remaining upper bounded by the value of a “perfect hindsight” t periods. The integer programming model computes stochastic program. The perfect hindsight model, the optimal allocation y∗ of available inventory n denoted PI,determines the optimal allocation of to the expected itinerary demand Dt,by maximiz- inventory for each particular realization of demand, ing total revenues subject to capacity and itinerary and then computes the expected reward over all demand constraints. For all n ≤ N and t ≤ T ,we have scenarios: IPn Dt = max R · y DPnt ≤ PIn t = E LPn t s.t. A · y ≤ n t t t 0 ≤ y ≤ D ≤ LPnE = LPn D
y integer t Here, is the cumulative demand corresponding The linear programming relaxation of this problem to scenario . Since the LP value is concave in the provides an efficient way to compute the best possible demand vector,the second inequality follows from “fractional” allocation of inventory,and is defined by Jensen’s inequality. Furthermore,the two values con- simply relaxing the integrality constraint: verge as demand becomes very large (see Popescu 1999). LPn Dt = max R · y s.t. A · y ≤ n 2.3. Origin-Destination RM and t 0 ≤ y ≤ D (2) Linear Networks Consider a particular case of the NRM problem, Clearly,we have that IP n Dt ≤ LPn Dt. This where demand is by ODF,as opposed to itinerary inequality can be strict,but there are particular specific. This is by default the case for linear net- instances when equality holds,one of which (linear works,where there is no question of routing,such as networks) we describe in the next section. One can in hotel or car rental revenue management (where the obtain further insight into the optimal LP-allocation nodes are days). In the case of airline revenue man- by looking at the dual problem: agement this situation occurs when there are perfectly LPn Dt = min v · n + u · Dt substitutable routes (same price,same travel time,and s.t. v · A + u ≥ R so on). In these cases,the Model (2) can be repre- u v ≥ 0 sented as a network flow problem. The advantage of this formulation is that it is very easy to solve in prac- This formulation can be equivalently written as: tice and reoptimization is very fast. For airlines,this model provides (for the company) optimal routing of LPn Dt = min v · n + R − v · A+ · Dt v≥0 path-indifferent customers. t The network representation is as follows: The nodes = min vk · n + uk · D k∈K are the origins and destinations (days for hotels,{air- where K denotes the index set of extreme points port,time}-pairs for airlines). There are multiple for-
Downloaded from informs.org by [18.172.5.91] on 05 August 2016, at 07:10 . For personal use only, all rights reserved. vk uk of the dual polyhedron. Thus,the objective ward arcs odf representing the flow from origin o value of Model (2) is a piecewise linear, concave, and to destination d from fare-class f . The capacity of each nondecreasing function of the expected demand to forward arc is the corresponding aggregate demand t f f come D and available capacity n. We next relate the Dod. The revenue collected along arc odf is Rod. For objective values of the dynamic and static formula- each “leg” ij (flight leg,respectively,pair of consec- tions at any state nt. utive days) there is a backward arc of the type ji,
262 Transportation Science/Vol. 37,No. 3,August 2003 BERTSIMAS AND POPESCU Revenue Management in a Dynamic Network Environment
capacitated by nij ,the available inventory of leg ij. prices v from the linear programming formulation (2). The revenue collected along backward arcs is zero. Then itinerary bid prices are computed additively,at This model has been proposed by Glover et al. (1982) each state S = nt,and for each class j request as in the context of airlines,and for hotel revenue man- S j agement by Chen (1998),who proves that it reduces BPj S = v · A (3) to solving a network flow problem. Notice that bid prices depend on the choice of Proposition 2 (Integrality of Solution). For optimal dual variables vS. This technique was ini- ODF models with integer data, the optimal LP-solution is tially proposed by Simpson (1989),then studied by t t integral, that is IPn D = LPn D . Williamson (1992) in her Ph.D. thesis. Several proba- bilistic models have been investigated by Glover et al. 3.Approximate Dynamic (1982),Williamson (1992),and Talluri and van Ryzin Programming Algorithms (1998). It has been observed (Williamson 1992) that the LP model achieves a better performance,and is In this section,we present several algorithms for the more efficient. For recent work on bid-price control NRM problem. Most of these algorithms belong in the see Talluri and van Ryzin (1998) and Gunther et al. generic class of approximate dynamic programming (2000). methods (see Bertsekas and Tsitsiklis 1998),in which an approximate value to the exact value function is 3.2. Certainty Equivalent Control used in the Bellman equation. The main disadvantages that are apparent from the Given a certain efficient mathematical program- definition of leg-based additive bid prices is that ming formulation (MP) of the NRM problem,a (a) they are not uniquely defined (several sets of generic approximate DP algorithm for the NRM prob- lem has the following structure: shadow prices may be optimal),and (b) they provide an additive approximation of the opportunity costs, Generic MP-Policy. Identify an efficient formula- which are not necessarily additive due to “bundle tion MP of the NRM problem. effects” (group or multileg itinerary requests may At any current state S = nt, determine basis changes in the dual LP). (1) For a class j request,compute an MP-based esti- We provide a different approximation scheme for mate of the opportunity cost OCMPS. j opportunity costs,based on certainty equivalent (2) Sell to class j if and only if its fare R exceeds j adaptive control. The idea is to approximate the value its opportunity cost estimate,i.e., function of the dynamic program DPnt defined in MP Equation (1) by the value of the linear programming Rj ≥ OCj S problem LPn Dt defined in §2.2. Thus,in Step 1 of (3) Go to Step 1 and ITERATE. the generic algorithm,a request for a given class j We will denote the expected value of this policy at will be accepted if and only if its price Rj exceeds its an initial state S as MPS. The difference between current opportunity cost estimate given by: various algorithms comes from the approximating MP and the MP-based opportunity cost measure,that is LP t−1 j t−1 OCj nt= LPn D − LPn − A D from Step 1. Because the opportunity cost estimate is calculated 3.1. Bid-Price Control in terms of LP objectives,it is uniquely defined,in
Downloaded from informs.org by [18.172.5.91] on 05 August 2016, at 07:10 . For personal use only, all rights reserved. Bid-price control is a popular method in NRM, that its value will not depend on the choice of (dual) whereby the opportunity cost (shadow price,or bid solution. Thereby,the first drawback of bid prices is price) of an itinerary is approximated by the sum of resolved. opportunity costs of the legs along that itinerary. First, This is the certainty equivalent control (CEC) pol- opportunity cost estimates (bid prices) are determined icy,and we denote its expected value for an initial
for each leg in the network,usually as the leg-shadow state S as LPS = CECS. For more on certainty
Transportation Science/Vol. 37,No. 3,August 2003 263 BERTSIMAS AND POPESCU Revenue Management in a Dynamic Network Environment
equivalence,see Bertsekas (1995). A similar policy is Simulations Using Monte Carlo Demand Esti- proposed by Bitran and Mondschein (1995) in the con- mation. One problem with the certainty equivalent text of hotel revenue management,but no analysis is policy is that it only considers expected demand infor- provided. Notice that in the case of linear networks, mation,and uses a deterministic approach to a highly it is actually desirable to use the equivalent network stochastic problem. We propose a variation on the flow formulation as described in §2.3 instead of the certainty equivalent policy that uses Monte Carlo usual LP model because reoptimization at each stage demand estimation to capture demand variability. becomes a much simpler task. Suppose we have information that the cumula- More generally,one can use this technique with vir- tive demand to come t−1 follows a certain distribu- tually any mathematical programming (MP) model tion. We generate r samples from this distribution: that provides an approximation of the value func- t−1 t−1 D1 Dr . In Step 1 of the generic algorithm,we tion. The corresponding OC estimate for an itinerary j calculate the opportunity cost estimate of itinerary j MP request at the current state S is defined as OCj S = as a weighted average, MPS − MPS ,where S and S are the rejj accj accj rejj r states corresponding to the accept,and respectively MC LP t−1 OCj nt= $i · OCj n Di reject decision for the current request j. For exam- i=1 ple,for the static LP model without cancellations and where $ = Pt−1 = D t−1 t−1 ∈ &D t−1 D t−1' overbooking,if S = nt,then S = n − Aj t− 1 i i 1 r accj and OCLPn D t−1 = LPn D t−1 − LPn − Aj D t−1. and S = nt− 1. j i i i rejj One difficulty with implementing this procedure is 3.3. Extensions that we might not have enough information about the In this section,we describe several extensions that aggregate demand and/or it may be too expensive to provide improvements in the efficiency or accuracy of compute the actual value of the conditional probabil- the adaptive policies described above. ities $i. For this reason,we run a simplified version of this policy,that assigns the same weights to all the Rollout Policy. The expected value H S of any trials: OCMCnt= 1/r · r OCLPn D t−1. heuristic H started in state S provides an approxi- j i=1 j i mation of the DPS value. Therefore,the expected heuristic values can in turn be used to provide esti- 4.Structural Properties mates of opportunity costs,determined as follows: In this section,we derive several structural properties of the new approximate dynamic programming algo- OCRHnt= nt− 1 − n − Aj t− 1 j H H rithm (CEC) and compare it with additive bid-price This opportunity cost estimation mechanism leads control algorithms (BPC) developed in the literature. to a new approximate dynamic programming (ADP) Given that the NRM problem requires a real- heuristic,called the rollout of H,and denoted RH. time response,it is desirable to use computationally This method is simply a form of policy iteration and inexpensive models to construct approximations of is described in detail in Bertsekas and Tsitsiklis (1998). the opportunity cost. This motivates the choice for It has been observed in the dynamic programming using the LP formulation described in §2.2. We are literature that this procedure systematically improves interested in a comparative structural assessment of heuristic performance (see also Bertsimas et al. 1999, the two LP-based policies described previously,BPC Bertsekas et al. 1997,Bertsekas and Castanon 1998). and CEC,and their corresponding opportunity cost
Downloaded from informs.org by [18.172.5.91] on 05 August 2016, at 07:10 . For personal use only, all rights reserved. For practical purposes,we suggest using Monte approximations.
Carlo simulation for evaluating the policy value H From an asymptotic point of view,it should be for a subset of states,and then interpolating these in noted that the CEC policy is asymptotically optimal an online fashion. An interesting research idea is to in the fluid scaling regime proposed by Talluri and investigate what types of preprocessing simulations van Ryzin (1998),whereby demand and capacities would provide an insightful information database. are simultaneously increased in a way that preserves
264 Transportation Science/Vol. 37,No. 3,August 2003 BERTSIMAS AND POPESCU Revenue Management in a Dynamic Network Environment
their relative values constant. They prove that in this Table 1 The Behavior of the BPC and CEC Policies as a Function of an regime,the additive bid-pricing policy converges to Optimal Primal Solution y∗
the optimum,as bid prices are being held fixed. By ∗ ∗ ∗ yj ≥ minDj 1yj < minDj 1 in all y , imitating their proof,one can show that the same ∗ ∗ ∗ ∗ yj in some y but = 0 in some yj = 0inally property holds for the CEC policy,with determinis- CEC accept reject reject tic prices,as OC estimates are being held fixed (see BPC accept accept reject Popescu 1999). The next result compares the opportunity cost policy will also accept,but not vice versa. This is approximations of a class j request at any given state. because the following situation may occur: BPj nt≤ Proposition 3. In any state nt, for any bid prices LP Rj < OCj nt,(see §4.1). j BPj ntBPj n − A t, the following inequalities hold: Table 1 provides a comparative characterization of the bid-pricing policy (BPC) versus the CEC policy,in BP nt≤ OCLPnt≤ BP n − Aj t (4) j j j terms of the structure of the primal optimal solutions y∗ of the LP model (2). We assume the BPC policy Inequalities are strict if accepting class j must incur a is well defined,in that the dual optimal solution is change of basis in the LP dual. unique. We also assume that the dual basis is not the Proof. Recall that we have defined: same for LPn Dt−1 and LPn −Aj Dt−1; if the dual basis does not change,then the policies are identical. OCLPnt = LPn Dt−1 − LPn − Aj Dt−1 j The following two propositions state and prove these = vnt · n + unt · Dt−1 results formally. − vn−Aj t · n − Aj − un−Aj t · Dt−1 Proposition 4 (Structural Properties of the BPC Policy). At any state nt,ifLPn Dt−1 has where vnt unt and vn−Aj t un−Aj t are optimal a unique dual optimal solution, then the corresponding dual solutions of LPn Dt−1 and LPn − Aj Dt−1, ∗ bid-price policy accepts only classes j for which yj > 0 in corresponding to the given bid prices: BPj nt = some primal optimal solution. vnt · Aj and BP n − Aj t = vn−Aj t · Aj ,respec- j Proof. By analyzing the primal and dual LP,one tively. Because both solutions are feasible for both can distinguish the following situations: programs,we obtain the following upper bounds by • In all optimal LP-solutions y∗ = 0,then unt = 0 evaluating each LP at the optimal solution of the j j from complementary slackness. By strict complemen- other: tary slackness (see Bertsimas and Tsitsiklis 1997, nt t−1 n−Aj t n−Aj t t−1 nt j LPn D ≤ v · n + u · D p. 192),we have v · A + uj >Rj ,i.e.,BP j nt= vnt · Aj >R,in which case the bid-price policy j t−1 nt j nt t−1 j LPn − A D ≤ v · n − A + u · D rejects class j. • In all optimal LP-solutions y∗ = Dt−1,in which The upper bound in Equation (4) follows by applying j j case unt = R − vnt · Aj + > 0,so the bid-price pol- the first of these inequalities in the OC formula,and j j icy accepts class j. the lower bound by the second inequality, • There is some optimal LP-solution such that 0 < nt j LP n−Aj t j y∗ Transportation Science/Vol. 37,No. 3,August 2003 265 BERTSIMAS AND POPESCU Revenue Management in a Dynamic Network Environment ∗ t−1 (a) CEC accepts class j if yj ≥ minDj 1 in some In general,if at a given state the CEC policy accepts optimal solution of LPn Dt−1. a request from itinerary j,then at the same state the ∗ t−1 (b) CEC rejects class j if yj < minDj 1 in all opti- bid-pricing control policy will also accept,but not vice mal solutions of LPn Dt−1. versa (see Proposition 3). The following situation may occur: BPj ≤ R < OCLPj,and so we will accept Proof. If in some optimal solution y∗ of LPn j under the BPC policy but not under the CEC policy. Dt−1 we have that y∗ ≥ minDt−1 1,then y∗ − j j The following is an example of such behavior,that minDt−1 1·e ≥ 0 is a feasible solution of LPn−Aj j j provides insight into the structural properties of the Dt−1,where Dt−1 = Dt−1 −minDt−1 1·e ,and hence, j j j j two policies. t−1 ∗ t−1 j t−1 Consider a network with four nodes: a hub h,two LPnD = R ·y ≤minDj 1Rj +LPn−A Dj origin nodes o o ,and a destination node d. The legs j t−1 1 2 ≤ Rj +LPn−A D of the network are (1) o1h,(2) o2h,(3) hd. One where the last inequality holds because any opti- can think of this as part of a bigger network where mal primal solution of LPn − Aj Dt−1 is feasible the other (connecting) flights have been sold out. Sup- j pose there is demand from the origin nodes o o to to LPn − Aj Dt−1. So,OC LP n Dt = LPn Dt−1 − 1 2 j the hub node h and to the destination node d,on the LPn − Aj Dt−1 ≤ R . This proves Part (a). j itineraries: (1) o h,(2) o h,(13) o hd,(23) o hd. Sup- For Part (b),it follows that 0 ≤ y∗ Downloaded from informs.org by [18.172.5.91] on 05 August 2016, at 07:10 . For personal use only, all rights reserved. a given request in the same state; Suppose that there is one seat left on each leg,so n = • An instance when BPC is suboptimal; 1 1 1,and D1 > 1 and D23 > 1,so that the demand • An instance when CEC is suboptimal; for the high-paying mix is large enough for the corre- • A counterexample of a cross-concavity property sponding constraints to be nonbinding in an optimal ∗ ∗ (“decreasing differences”) of the LP and DP-value solution. Then the optimal LP solution is y1 = y23 = 1, ∗ ∗ functions for the NRM problem. y2 = y13 = 0,and the value of the LP is R1 + R23. 266 Transportation Science/Vol. 37,No. 3,August 2003 BERTSIMAS AND POPESCU Revenue Management in a Dynamic Network Environment Since the demand constraints are nonbinding,we Stone 1991),and would provide a sufficient condition must have that u = 0,and hence v1 ≥ R1v2 ≥ R2v3 ≥ for the property to extend to the network case. max0R13 − v1R23 − v2. Therefore,in an optimal However,the decreasing differences property is ∗ ∗ solution,the shadow prices are equal to v1 = R1v2 = violated in our example: ∗ R2v3 = R23 − R2. We can compute the bid prices and opportunity costs as follows: LP1 1 1 − LP0 1 1 LP = maxR1 + R23R2 + R13 − R23 Downloaded from informs.org by [18.172.5.91] on 05 August 2016, at 07:10 . For personal use only, all rights reserved. and s + 0iei + 0j ej ∈ S,the following relation holds: perform aircraft changes. fs + 0iei + 0j ej − fs + 0iei ≤ fs + 0j ej − fs. The typical overbooking method practiced by air- This cross-concavity property reduces in the uni- lines,hotels,and so on,is to decide an initial alloca- variate case to concavity. This is the key observation tion of overbooking pads,which are virtual increases underlying the proof of the threshold times property in leg-capacity. This is usually performed,in prac- for the optimal single-leg policy (see Diamond and tice as well as in the literature,as a static,one-time Transportation Science/Vol. 37,No. 3,August 2003 267 BERTSIMAS AND POPESCU Revenue Management in a Dynamic Network Environment decision made at the beginning of the booking hori- solved to decide which passengers should be refused zon. In the following,we propose a new method boarding so as to incur least penalties. For example, for dynamic overbooking control,where oversales in a two-leg network which is oversold by one seat decisions are dynamic and implicit in the admission on each leg,it is better to bump a connecting passen- control mechanism. ger rather than two different passengers on each leg whenever overbooking penalties are leg-subadditive. 5.1. General Models and Relationships This model does not incorporate secondary effects First,we extend the various models described in §2 to associated with overbooking (e.g.,image damage and incorporate cancellations,no shows and overbooking. loss of goodwill,customer value,demand shifting, and so on). A Dynamic Programming Model. In the case of perfect state information,this model provides the opti- The Integer and Linear Programming Models. In mal control policy. By allowing cancellations,the state the case of the NRM problem with cancellations and space of the DP-model becomes much larger because no shows,we observe that both the final revenue it is necessary to keep track of the past sales record s. gained from,and capacity occupied by a class j reser- The random quantities involved are the demand,can- vation are not deterministic quantities because they cellations,and no-show processes. Given the initial depend on cancellations and no-shows which are ran- network inventory N,the maximum expected rev- dom processes. To define a model that is consistent j enue (less refunds and penalties) to be collected from with these observations,we define Rj and A to be the state st onward (cost-to-go),is given by expected revenue gained from,and expected capacity occupied by a class j reservation,before the overbook- o t o DPNst = pj · max DPNst− 1 c ns ing period. Let pj and pj denote the probability that a j given class j reservation is cancelled at some point in o Rj + DPNs + ej t− 1 the booking period,and respectively does not show + pct · DPo s − e t− 1 − Rct up for the flight. We assume that these quantities are j N j j independent on the time the reservation was made, j sj ≥1 and so is the cancellation penalty. We denote the rev- + pt · DPo st− 1 0 N enue expected (or “adjusted”) from booking a class o j customer as R = R − 1 − pns · pc · Rc − 1 − pc · pns · DPNs 0 = Ps˜ bookings out of s materialize j j j j j j j s˜ ns Rj ,which accounts for potential cancellation and no- o shows events and respective refunds RcRns (but not · DPNs˜ −1 j j overbooking penalties). Let A j = 1 − pc · 1 − pns · Aj DPo s −1 =−min C · so j j N denote the expected capacity occupied by a class j s.t. A · s − so ≤ N reservation at the end of the horizon. Finally C = o (6) j 0 ≤ s ≤ s c ns 1 − pj · 1 − pj · Cj is the average overbooking cost The difference from the basic model is that cancel- of one class j reservation. lation events are incorporated in the Bellman Equa- With these notations,we can formulate the fol- tion (6),and the boundary conditions are changed to lowing integer programming approximation model, account for no-shows (time t = 0) and final bumping that maximizes expected revenues subject to expected decisions (time t =−1). The final bumping decision is capacity constraints: made so as to minimize total penalties,while keeping o o IPNst = max R · y − C · z Downloaded from informs.org by [18.172.5.91] on 05 August 2016, at 07:10 . For personal use only, all rights reserved. the actual capacity restrictions satisfied. s.t. 0 ≤ A · y + s − A · zo ≤ N If customer-walking penalties are payed per leg 0 ≤ y ≤ Dt c = c c,rather than per itinerary,the bound- 1 l y zo integer ary condition at t =−1 is simply c As − N+. When bumping penalties are itinerary specific (not leg- The vector y decides how many requests to be additive),then an optimization problem needs to be accepted in the future,whereas zo determines which 268 Transportation Science/Vol. 37,No. 3,August 2003 BERTSIMAS AND POPESCU Revenue Management in a Dynamic Network Environment customers (itinerary requests) should be bumped at 6.Computational Results the end of the horizon,if necessary. The “virtual” In this section,we present computational results that o overbooking pad for each leg is A · z . The capacity illustrate the relative practical performance of the constraint requires that cancellation-adjusted past and previously described admission control policies for future sales,less oversales,should not exceed the ini- the NRM problem,under different demand regimes. tial network capacity. Our objective is to evaluate the different models and With the change of variable so = zo/1−pc·1−pns, j j j j policies proposed,in terms of the following criteria: j = 1 n,the corresponding linear programming (1) running time,(2) quality of approximation,and relaxation,and its dual,can be written as follows: (3) robustness. In addition,we would like to fur- o o ther investigate the effectiveness of several extensions LPNst = max R · y − C · s s.t. A · s + y − so ≤ N and improvements. In §6.1,we report computational 0 ≤ y ≤ Dt results without cancellations and overbooking,while 0 ≤ so ≤ y + s in §6.2,we allow cancellations and overbooking. = min v · N − u · s + R − u + · Dt s.t. u = min C v · A 6.1. Computational Performance Without v ≥ 0 Cancellations and Overbooking We performed expected value calculations for two-leg 5.2. Adjusted Policies instances and simulation runs for larger networks. To Following the spirit of the basic adaptive bid pricing have a consistent and fairly accurate base for compar- and CEC policies,we adjust these to incorporate can- ing various policies,these were simulated simultane- cellations and overbooking. ously on the same realizations of the demand process. Adjusted BPC Policy. We modify the DP model so The computations were performed in MATLAB 4.0 on that at any given state S a class j request is accepted if an Intel Pentium II Celeron 450 MHz (128 MB RAM, WinNT 4.0). We restricted our attention to smaller and only if its “adjusted” fare Rj is higher than either its adjusted bid price,or the adjusted overbooking instances with the explicit objective to obtain insights penalty Cj ,i.e., into the behavior of both BPC and CEC. o Rj ≥ minCj BPj S 6.1.1. Models and Assumptions. We considered the following types of networks: o S j Here,bid prices are computed as BP j S = v · A , N2. A two-leg network with nodes o h d,legs S where v represent shadow prices for the leg-capacity (1) oh (2) hd and capacities N = N N . The available o 1 2 constraints in LP S. itineraries are (1) oh,(2) hd,and (3) ohd,with one class Adjusted CEC Policy. Similarly,we can adapt the per itinerary. The leg-class incidence matrix,together CEC policy to accept a class j request if and only if with the fare structure R,is: its “adjusted” revenue exceeds its “adjusted” oppor- o o o R R R tunity cost. Rj ≥OCj S = LPNst−1−LPNs+ej t− 1 2 3 R 1. Again,adjustment accounts for the fact that capac- = 101 ity and revenue might not be realized or may be A Downloaded from informs.org by [18.172.5.91] on 05 August 2016, at 07:10 . For personal use only, all rights reserved. overbooked. 011 The same type of arguments can be used to extend Proposition 1 for the case of cancellations,no shows N3. The three-leg network described in the exam- and overbooking. Moreover,the structural proper- ple of §4.1,with four nodes: a hub h,two origin nodes ties proved in §4 are preserved in the overbooking- o1o2,and a destination node d,and the legs (1) o1h, adjusted policies. (2) o2h,(3) hd. There is demand from the origin nodes Transportation Science/Vol. 37,No. 3,August 2003 269 BERTSIMAS AND POPESCU Revenue Management in a Dynamic Network Environment o1o2 to the hub node h and to the destination node d, We used the following alternative scenarios to model the arrival process: on the itineraries: (1) o1h,(2) o2h,(13) o1hd,(23) o2hd. Suppose that there is only one fare class per itinerary, HP. Homogeneous Poisson arrivals with arrival and there is no demand from h to d. The leg-class inci- rates given by a constant vector p (i.i.d. Bernoulli trials). dence matrix,together with the fare structure R,is: NHL. Nonhomogeneous Poisson with high-low demand. We assume arrival rates increase for high- R R R R 1 2 13 23 fare classes (pt = p/ log a + t · ) and decrease (pt = a p/ log a + T − t · ) for low-fare classes,as we R 10 1 0 a = approach departure. A 01 0 1 NLH. Nonhomogeneous Poisson with low-high 00 1 1 demand. We assume arrival rates decrease for high- t fare classes (p = p/ logaa + T − t· ) and increase (pt = p/ log a + t · ) for low-fare classes,as we N4. A four-leg network,with two origins o o , a 1 2 approach departure. two destinations d d and a hub h. The legs in the 1 2 The log-factors are given by ,and a is a constant, network are (1) o1h,(2) o2h,(3) hd1,(4) hd2,with capac- equal to the base of the logarithm (we take a = 2). ities N = N N N N . There is demand for all 1 2 3 4 6.1.2. Running Time. Exact calculations of the the eight itineraries,with one fare class per itinerary. optimal expected revenue (DP),and expected values The leg-class incidence matrix,together with the fare of the proposed policies (CEC,BPC) are practically structure R,is: impossible. Already for two-leg networks (N 2) with one class per itinerary,50 seats per leg initial capac- R1 R2 R3 R4 R13 R14 R23 R24 ity and T = 300 time periods the computation takes about two hours. 10001100 A tractable approach for measuring performance of R = 01000011 the proposed policies,however,is provided by simu- A lation. A simulation run of the CEC or BPC policies 00101010 for a two-leg network takes less than a minute (less 00010101 than a second per iteration). The largest network we simulated was a four-leg network N42 with two demand classes per itinerary. When the initial capaci- N 42. The same as N4,except there are two fares ties are in the order of N = 50–100 and the time hori- per itinerary,so 16 demand classes in all. The leg- zon has T = 300 time periods,a full simulation run class incidence matrix,together with a high-low fare for such an instance takes a few minutes (i.e.,a few h l structure R = R R ,is: seconds per iteration). l h l h l h l h l h l h l h l h R1 R1 R2 R2 R3 R3 R4 R4 R13 R13 R14 R14 R23 R23 R24 R24 11000000 1 1 1 1 0 0 0 0 R Downloaded from informs.org by [18.172.5.91] on 05 August 2016, at 07:10 . For personal use only, all rights reserved. = 00110000 0 0 0 0 1 1 1 1 A 00001100 1 1 0 0 1 1 0 0 00000011 0 0 1 1 0 0 1 1 270 Transportation Science/Vol. 37,No. 3,August 2003 BERTSIMAS AND POPESCU Revenue Management in a Dynamic Network Environment Table 2 Expected Value Calculation for N2 with N = 50 50;R= 25 20 35;p= 0 4 0 3 0 1; T = 10−200 (HP) (NLH) (NHL) T LP DP CEC BPC LP DP CEC BPC LP DP CEC BPC 10 195 195 195 195 201 4 201 4 201 4 201 4 249 7 249 7 249 7 249 7 20 390 390 390 390 413 5 413 5 413 5 413 5 498 8 498 8 498 8 498 8 30 585 585 585 585 632 9 632 9 632 9 632 9 747 3 747 3 747 3 747 3 40 780 780 780 780 857 6 857 6 857 6 857 6 995 1 995 1 995 1 995 1 50 975 975 975 975 1086 51086 51086 51086 51242 11242 11242 11242 1 60 1170 1170 1170 1170 1318 61318 61318 61318 61488 31488 11488 11488 70 1365 1365 1365 1365 1553 41552 21552 11552 21733 61703 1702 81690 4 80 1560 1559 51559 51559 51790 41755 31754 51754 91834 31800 91799 61753 4 90 1755 1745 71745 41745 71885 31863 31860 81840 91846 11831 41829 51791 1 100 1950 1897 51896 41897 11925 11904 71901 41852 11858 41845 81843 11809 6 110 2020 1998 91996 61993 41937 31922 1917 1863 31871 11858 71855 1822 5 120 2090 2064 62061 62031 61949 11934 31927 11876 91884 41872 11867 11837 130 2140 2110 62107 32021 91960 61945 91936 71889 51898 31886 21879 81851 5 140 2170 2145 52140 52018 31971 71957 11946 21899 51913 1901 1893 21864 150 2200 2175 72167 62018 91982 51968 11955 61907 21928 71916 71907 61875 8 160 2230 2202 42192 92019 31993 11978 71964 91913 11945 51933 61923 21887 8 170 2250 2222 82216 92019 42003 41989 11974 1918 11963 81952 1940 21900 8 180 2250 2236 22233 2019 42013 51999 31983 1922 91984 21972 51959 31915 5 190 2250 2243 82242 32019 42023 52009 31991 81928 62007 41995 81981 41932 4 200 2250 2247 52246 92019 42033 22019 2000 51934 72035 32023 72008 11952 8 Note. The first section considers homogeneous arrivals, and the next two are nonhomogeneous arrivals with = 50 50 30 and a = 2. For more realistic airline networks [20 legs (10 of the dependence of the policy on the choice of dual origins,10 destinations,and one hub),600 classes solutions (not enforced in our program). (100 itineraries,6 fare classes)] the time per iteration Simulation Results. For two-leg networks we pro- is still in the order of a few minutes. This supports vide in Table 3 a comparison between simulation the idea that these dynamic policies can be used in an results and exact calculations for the DP value func- online fashion to provide an adaptive control mech- tion and expected values of the CEC and BPC poli- anism where opportunity cost estimates are rapidly recalculated at each iteration. cies. The high dimensionality of the problem does not allow for exact expected value calculations for 6.1.3. Quality of Approximation larger networks. In Table 4,we present results from Expected Value Computations for Two-Leg Net- works. The results in Table 2 show expected value Table 3 Simulations and Expected Value Calculation for Two-Leg calculations for the case of the two-leg network, Network with different initial capacities,fare structures,and CEC BPC demand regimes. In all cases,we observe that when DP the time horizon is large,the value of the CEC pol- T LP EXP EXP SIM ( ) EXP SIM ( ) icy is near optimal,whereas the bid-pricing policy N2 Downloaded from informs.org by [18.172.5.91] on 05 August 2016, at 07:10 . For personal use only, all rights reserved. appears to be off by a constant. When the time hori- 50 975 975 975 973.3 (71) 975 973.2 (71) zon is small,both policies perform optimally. For 100 1950 1897 51896 4 1,904.05 (83) 1897 4 1,903.75 (82) intermediate stages when both some demand and 150 2200 2175 52167 6 2,174.9 (45) 2018 8 2,021.2 (44) capacity constraints become binding,it is not clear 200 2250 2247 52246 9 2,246.4 (18) 2019 4 2,023.3 (17) which policy is better. Note that the BPC value is non- 300 2250 2250 2250 2,250 (0) 2200 2,200 (0) monotonic over time. We believe that this is because Note. N2: N = 50 50; R = 25 20 35; p = 0 4 0 3 0 1. Transportation Science/Vol. 37,No. 3,August 2003 271 BERTSIMAS AND POPESCU Revenue Management in a Dynamic Network Environment Table 4 Simulations for Larger Networks (Top- noise in demand forecasts,we compare LP n D and Down) ELPn D + . From the dual formulation we have T LP CEC ( ) BPC ( ) that N3 ELPn D + − LPn D 100 2,300 2,273 (140) 2,268 (140) + 200 3,200 3,155 (184) 3,148 (181) = EOCn D ≤ R − v · A · E 300 3,600 3,508 (178) 3,476 (177) where v are the leg-shadow prices of LPn D.So N4 1 there is a sublinear effect on the value function,asso- 100 3,500 3,462 (150) 3,460 (154) 300 5,800 5,615 (78) 5,635 (74) ciated with the forecasting bias of the demand for 600 6,050 5,951 (85) 5,842 (94) those classes whose fares exceed their bid prices. N4 2 Furthermore,computational experiments show that 100 4,070 4,087 (213) 4,085 (216) the CEC policy is surprisingly robust to certain forms 200 6,125 5,914 (241) 5,856 (233) of noise and bias in the demand data,much more so 300 11,110 10,859 (302) 10,846 (309) than BPC. Note. N3: N = 60 60 100; R = 10 20 50 30 Correlated Random Noise. At each point in time and p = 0 2 0 3 0 4; N4 1: N = 50, R = 20, i t we generate a multivariate random variable ∼ 20 50 30 60 40 45 55, p = 0 2 0 25 0 05 0 05 0 1, l Nb ,to introduce noise into the arrival forecast. 0 15 0 1 0 1; N4 2: Ni = 50, R = 20 20 50 30, 60 40 45 55, with Rh = 2 · Rl and p = 0 15 0 02 0 2, We propose two alternatives for adding noise to the 0 01 0 05 0 015 0 05 0 01 0 1 0 01 0 15 0 015 0 1, forecast: 0 01 0 1 0 01. • Constant rate noise: Generate and add it to the arrival rate at each time; simulating various admission control policies for • Log-rate noise: Generate and add t = loga + larger networks. For simulation runs,we perform · t − a/T · to the arrival rate. NT = 100 trials and average the results to compute an From the results in Table 5,we observe that CEC expected value estimate. We also compute the stan- is constantly more robust than BPC to noise and bias. dard deviation of the estimation. In all cases,we The columns LP compute the LP-values with noisy t t t observe that our CEC policy performs better than the forecasts: LPn D = LPn D + E . BPC policy,by up to 2%. Uncorrelated Random Noise. The goal of this com- 6.1.4. Robustness in Estimation. So far,we have putational exercise is to illustrate and compare the assumed that the CEC and BPC policies use cor- robustness of the BPC and CEC policies,to increasing rect demand information,in the sense that the mean- bias in the demand forecast. We start with an initial demand forecast is unbiased. While this provides a (uncorrelated) bias per arrival b,and in each consecu- reasonable standard for comparing policies,in reality, tive experiment subscripted $b,we increase the bias the (expected demand) measurements obtained from by a factor of $ = 1 2 4. In the last experiment (), forecasting tools are seldom exact. We feel that it is we introduce correlated noise ( same as above) in of practical value to assess the robustness of the CEC the arrival rate estimation. We observe from Table 6 and BPC policies to noise and bias in the demand that CEC is constantly more robust than BPC to noise forecast. and bias. Downloaded from informs.org by [18.172.5.91] on 05 August 2016, at 07:10 . For personal use only, all rights reserved. To obtain a qualitative measure of how bias and 6.1.5. Extensions noise in the demand forecast affect these policies, we assess the impact on the underlying LP-value. Rollout Heuristics. We examine how previously Let be an m-variate random variable represent- computed values of the CEC and BPC policies can be ing the noise in demand forecast for each class. To used to provide estimates of opportunity costs,lead- measure the robustness of the LP-value to bias and ing to the roll-out heuristics R(CEC) and R(BPC). We 272 Transportation Science/Vol. 37,No. 3,August 2003 BERTSIMAS AND POPESCU Revenue Management in a Dynamic Network Environment Table 5 Computation with Noisy Forecasts Constant Rate Log Rate T LP LP DP CEC BPC LP LP DP CEC BPC 10 195 213 7 195 195 195 195 221 1 195 195 195 20 390 416 7 390 390 390 390 408 7 390 390 390 30 585 611 9 585 585 585 585 623 3 585 585 585 40 780 784 6 780 780 780 780 765 5 780 780 780 50 975 975 1 975 975 975 975 957 2 975 975 975 60 1170 1175 51170 1170 1170 1170 1123 51170 1170 1170 70 1365 1366 41365 1365 1365 1365 1315 31365 1365 1365 80 1560 1605 61559 51559 51559 51560 1569 1559 51559 51559 5 90 1755 1770 31745 71745 21745 31755 1685 51745 71745 21745 1 100 1950 1977 81897 51895 81895 41950 1878 51897 51895 81894 6 110 2020 2086 11998 91996 11961 42020 2008 51998 91996 1974 7 120 2090 2115 22064 62061 71971 72090 2069 22064 62061 61967 2 130 2140 2139 42110 62107 11980 12140 2105 52110 62106 31981 2 140 2170 2172 2145 52139 61987 72170 2147 12145 52137 41987 9 150 2200 2197 82175 72166 91994 52200 2168 2175 72162 51993 5 160 2230 2227 82202 42192 81999 52230 2178 82202 42185 52030 2 170 2250 2250 2222 82216 82000 22250 2203 22222 82208 42151 1 180 2250 2250 2236 22233 2000 22250 2240 52236 22227 32208 9 190 2250 2250 2243 82242 32130 62250 2235 22243 82240 32223 4 200 2250 2250 2247 52246 92227 2250 2248 52247 52246 82230 6 0 1 −0 02 0 04 1 Note. N2: N = 50 50; R = 25 20 35; p = 0 4 0 3 0 1; b = 0 07 −0 05 0 03; = −0 02 0 08 0 01 . 5 0 04 0 01 0 06 observe that the rollout procedure produces a signifi- When the number of trials r is large enough (so we cant improvement in the quality of the value function have a reasonably good MC-demand estimate),this approximation,especially for the BPC policy (10%). policy provides a visible improvement (16%) over This can be observed in Table 7 where we performed the original version,as can be observed from the fol- calculations for various two-leg networks. lowing computational examples. However,when the In practice,however,it is too expensive to com- number of trials is not large enough (30 or less), pute and store all the H-values. In order to implement we have observed that this policy delivers a poor this idea effectively,we suggest storing H-values from performance. The tables below provide estimates for several insightful a priori simulations,and interpo- the value of different models (LP,PI LP,MC,CEC, lating these in an online fashion (as needed),for the BPC),as well as standard deviations,lower and upper “second time around” policy. It is an interesting idea to investigate what types of preprocessing simulations bounds obtained through simulation. We also calcu- would provide an insightful information database. late the estimated average ratio P between different models,as well as the standard deviation,minimum, Simulations Using Monte Carlo Demand Estima- and maximum ratio observed during simulation. One Downloaded from informs.org by [18.172.5.91] on 05 August 2016, at 07:10 . For personal use only, all rights reserved. tion. We run a simplified version of the Monte Carlo can observe for instance that the ratio CEC/BPC is policy described in §3.3,that assigns the same weights always ≥1,whereas the ratio MC/PI LP is surprisingly to all the trials and defines: high,ranging between 96%–99%. The computational 1 r results presented in Table 8 show an improvement of OCMCnt= · OC n D t−1 j r j i i=1 up to 20% of MC over CEC. Transportation Science/Vol. 37,No. 3,August 2003 273 BERTSIMAS AND POPESCU Revenue Management in a Dynamic Network Environment Table 6 Increasing Bias in Arrival Rates T DP LP CEC BPC LPb CECb BPCb LP2b CEC2b BPC2b 10 190 190 190 190 196 227 190 190 202 454 190 190 20 380 380 380 380 387 9062 380 380 395 8123 380 380 30 568 0307 570 567 9671 568 0053 578 9176 567 958 568 0149 587 8351 567 9583 568 0271 40 713 9429 740 712 8882 711 4275 751 7866 712 6471 711 3734 763 5732 712 654 710 6125 50 784 8309 815 782 7964 745 0979 818 9707 782 1006 743 3085 822 9414 781 6062 737 3545 60 819 8745 845 817 9585 749 6329 849 1515 817 5189 748 0799 853 303 816 9229 742 664 70 841 6218 855 839 8851 749 8441 855 839 6463 748 1471 855 839 3091 742 664 80 851 0686 855 850 2119 749 8441 855 850 1653 748 1471 855 850 1115 742 664 90 854 0811 855 853 8794 749 8441 855 853 8777 748 1471 855 853 8765 742 664 99 854 7912 855 854 7523 749 8441 855 854 7523 748 1471 855 854 7523 742 664 T DPLP4b CEC4b BPC4b LP CEC BPC 10 190 214 9079 190 190 191 8341 190 190 20 380 411 6247 380 380 361 348 380 380 30 568 0307 605 6702 567 9585 568 0273 671 4161 567 9394 567 8188 40 713 9429 787 1465 712 5613 706 1965 707 4845 712 1564 708 9488 50 784 8309 830 8828 780 0541 723 385 801 1911 779 7808 752 618 60 819 8745 855 815 0074 728 399 831 9429 815 4683 757 3472 70 841 6218 855 838 5854 728 399 855 838 5076 757 5589 80 851 0686 855 850 0349 728 399 855 849 9272 757 804 90 854 0811 855 853 8752 728 399 855 853 8605 761 142 99 854 7912 855 854 7523 728 399 855 854 7513 763 8937 Note. N2: Data: N = 19 19; R = 25 20 35; p = 0 3 0 4 0 1, b = 0 07 −0 05 0 03. 6.2. Computational Performance Under The implementation was done in C and the algo- Cancellations and Overbooking rithms were run on a Dell Pentium III 600MHz oper- The objective in this section is to understand the rela- ating under Linux. tive performance of BPC and CEC in an environment In Table 9,we report the behavior of CEC and with cancellations and overbooking. We considered a BPC as a function of the overbooking penalty. We booking horizon of 15 periods for a hub and spoke observe that CEC leads to consistently higher revenue network with five cities and two classes,of the type by approximately 1%. N42. The arrival process for the highest fare class In Table 10,we report the behavior of CEC and BPC is nonhomogeneous Poisson with rate 0.5 for Peri- as a function of the cancellation probability. With the exception of very high cancellation rate (0.30),CEC ods 1–13 and 5 for Periods 14,15. The arrival pro- outperforms BPC. cess for the lowest fare class is homogeneous Poisson In Table 11,we report the behavior of CEC and BPC with Rate 3. For simplicity,we kept the fare of the as a function of the ratio of the fare of a two-leg higher class in a single-leg itinerary equal to $100 and itinerary versus the fare of a single-leg itinerary. CEC of the lower class equal to $80. We varied the fare of outperforms BPC,but the level of overperformance two-leg itineraries. After experimentation we identi- Downloaded from informs.org by [18.172.5.91] on 05 August 2016, at 07:10 . For personal use only, all rights reserved. decreases as ranges from 1 to 2. fied the following parameters that affected the relative performance of CEC and BPC: We varied the follow- ing parameters: (a) The overbooking penalty (Cj ),(b) 7.Conclusions the probability of cancellation (pc),and (c) the fare of In this paper,we have presented several models and a two-leg itinerary compared to a single-leg itinerary. algorithms for solving the stochastic and dynamic 274 Transportation Science/Vol. 37,No. 3,August 2003 BERTSIMAS AND POPESCU Revenue Management in a Dynamic Network Environment Table 7 Rollout Heuristics T LP DP CEC BPC R(CEC) R(BPC) N2 1 119 519 519 519 519 519 5 10 195 195 195 195 195 195 20 390 390 390 390 390 390 30 585 585 585 585 585 585 40 780 780 780 780 780 780 50 975 975 975 975 975 975 60 1170 1170 1170 1170 1170 1170 70 1365 1365 1365 1365 1365 1365 80 1560 1559 51559 51559 51559 51559 5 90 1755 1745 71745 41745 71745 61745 7 100 1950 1897 51896 41897 11897 21897 4 110 2020 1998 91996 61993 41998 1995 5 120 2090 2064 62061 62031 62063 22042 2 130 2140 2110 62107 32021 92108 92052 8 140 2170 2145 52140 52018 32143 2104 2 150 2200 2175 72167 62018 92172 92163 9 160 2230 2202 42192 92019 32199 62197 2 170 2250 2222 82216 92019 42220 62217 4 180 2250 2236 22233 2019 42234 82230 2 190 2250 2243 82242 32019 42243 12238 1 200 2250 2247 52246 92019 42247 22241 9 N2 2 1191919191919 10 190 190 190 190 190 190 20 380 380 380 380 380 380 30 570 568 0307 567 9671 568 0053 568 0285 568 0258 40 740 713 9429 712 8882 711 4275 713 8018 713 2517 50 815 784 8309 782 7964 745 0979 784 3939 774 4301 60 845 819 8745 817 9585 749 6329 819 2368 815 8581 70 855 841 6218 839 8851 749 8441 840 6865 837 072 80 855 851 0686 850 2119 749 8441 850 6009 846 4442 90 855 854 0811 853 8794 749 8441 853 9772 848 7245 100 855 854 8245 854 7925 749 8441 854 8099 849 8084 Note. N2 1: N = 50 50, R = 25 20 35, p = 0 4 0 3 0 1; N2 2: N = 19 19, R = 25 20 35, p = 0 3 0 4 0 1. NRM problem. We proposed a new efficient algo- • The CEC opportunity cost estimates are uniquely rithm,based on a certainty equivalent approximation defined at each state,whereas for additive bid prices, and compared it with the widely used bid-price con- there may be several dual optimal solutions. trol policy. This policy conceptually improves the cur- • The CEC policy is optimal in the deterministic Downloaded from informs.org by [18.172.5.91] on 05 August 2016, at 07:10 . For personal use only, all rights reserved. regime,whereas BPC is not. rent NRM-approach based on additive bid pricing, • Computationally we observe that the CEC pol- by using more insightful,piecewise linear approxima- icy outperforms the BPC policy when the load factors tions of opportunity cost. It is just as easy to compute, (the ratio of expected demand to available capacity) but as opposed to BPC,it is more “robust” in the fol- tend to be large. When the load factor is small,both lowing sense: policies perform optimally. There is a critical range Transportation Science/Vol. 37,No. 3,August 2003 275 BERTSIMAS AND POPESCU Revenue Management in a Dynamic Network Environment Table 8 N2 1 LP PILP MC CEC BPC P AvgP StdP MinP MaxP EXP 2200 2191 22170 2162 82161 6 CEC/BPC 1 0006 0 0035 0 9931 1 0074 Std 0 44 48 50 45 52 41 54 19 CEC/PILP 0 987 0 0093 0 9569 1 LB 2200 2070 2035 2000 2000 MC/PILP 0 9903 0 0084 0 9645 1 UB 2200 2250 2250 2250 2240 MC/CEC 1 003 0 0067 0 9909 1 0225 N2 2 LP PILP MC CEC BPC P AvgP StdP MinP MaxP EXP 225 215 208 205 202 5 CEC/BPC 1 01 0 028 1 1 075 Std 0 10 55 16 53 18 719 76 CEC/PILP 0 9521 0 0498 0 8537 1 LB 225 195 185 175 175 MC/PILP 0 9664 0 038 0 9024 1 UB 225 225 225 225 225 MC/CEC 1 016 0 034 1 1 0857 N2 3 LP PILP MC CEC BPC P AvgP StdP MinP MaxP EXP 8000 7922 7785 7771 7766 CEC/BPC 1 0007 0 011 0 975 1 02 Std 0 224 3 238 7 285 2 283 8 CEC/PILP 0 9808 0 0166 0 94 1 LB 8000 7500 7200 7050 7050 MC/PILP 0 9828 0 017 0 939 1 UB 8000 8600 8400 8500 8400 MC/CEC 1 002 0 0184 0 9615 1 0425 N4 LP PILP MC CEC BPC P AvgP StdP MinP MaxP EXP 6800 n.a. 6694 46588 6568 8 CEC/BPC 1 0029 0 0051 0 9935 1 0117 Std 0 n.a. 214 44 213 73 215 MC/CEC 1 01624 0 0127 0 9908 1 0421 LB 6800 n.a. 6260 6140 6160 UB 6800 n.a. 7065 6950 6930 Note. N2: N = 50 50, R = 25 20 35, p = 0 3 0 4 0 1, T = 150, NT = 100, r = 50; N2 1: N = 5 5, R = 25 20 35, p = 0 3 0 4 0 1, T = 20, NT = 10, r = 50; N2 3: N = 20 20, R = 25 20 35, p = 0 2 0 3 0 5, T = 50, NT = 50, r = 50; N4: N = 50 50 100 100, R = 20 20 50 30 60 40 45 55, p = 0 15 0 15 0 15 0 15 0 1 0 1 0 1 0 1, T = 200, NT = 50, r = 60. when the load factor is close to one,where it is not • The CEC policy is significantly more robust than clear which policy provides a better performance,but BPC to bias and (correlated) noise in the demand the difference between the two is very small. More- forecast. over,computational exercises show that the value of The following extensions provide insights towards the CEC policy is very close to the value function DP, further developments: and to the perfect information upper bound PILP. —The rollout procedure produces an important • The CEC outperforms the BPC policy in an improvement in the value of both policies,virtually closing the optimality gap. environment where cancellations and overbooking —The Monte Carlo simulation procedure incor- are present under various scenarios of cancella- porates demand variability,and produces a signific tion probabilities,disparity in fares and overbooking ant improvement in the value of the CEC policy penalties. when the number of trials is sufficiently large. Table 9 The Expected Revenue in 200 Simulation Runs as a Table 10 The Expected Revenue in 200 Simulation Runs as a Function of the Overbooking Penalty Function of the Overbooking Penalty Downloaded from informs.org by [18.172.5.91] on 05 August 2016, at 07:10 . For personal use only, all rights reserved. Cj 100 110 120 130 pc 0.05 0.10 0.20 0.30 CEC 24,480 24,450 23,975 23,890 CEC 22,382 21,095 18,760 11,280 BPC 22,355 22,270 21,920 21,567 BPC 19,680 20,160 17,262 12,960 Note. The cancellation probability was 0.01 and the fare of a two-leg Note. The overbooking penalty was $130 and the fare of a two-leg itinerary was equal to the fare of a single-leg itinerary. itinerary was equal to the fare of a single-leg itinerary. 276 Transportation Science/Vol. 37,No. 3,August 2003 BERTSIMAS AND POPESCU Revenue Management in a Dynamic Network Environment Table 11 The Expected Revenue in 200 Simulation Curry,R. 1989. Optimal airline seat allocation with fare classes Runs as a Function of the Ratio of the nested by origins and destinations. Transportation Sci. 24(3) Fare of a Two-Leg Itinerary Versus the 193–204. Fare of a Single-Leg Itinerary . 1992. Real-time revenue management: Bid price strategies for origins/destinations and legs. Scorecard 2(Q). 1 1.5 2 Diamond,M.,R. Stone. 1991. Dynamic yield management on a sin- CEC 22,522 23,400 24,457 gle leg. Unpublished manuscript,Northwest Airlines. BPC 20,587 21,437 24,187 Feng,Y.,P. Lin. 2000. Optimal booking control on a two-leg flight with multiple fares. Working paper,http://www.math.nus. Note. The cancellation probability was 0.1 and the over- edu.sg/∼ matlinp/www/fenglinor.pdf. booking penalty $130. Fuchs,S. 1987. Managing the seat auction. Airline Bus. 7 40–44. Glover,F.,R. Glover,J. Lorenzo,C. McMillan. 1982. The passenger- mix problem in scheduled airlines. Interfaces 12 73–79. Acknowledgments Gunther,D.,V. Chen,E. Johnson. 2000. Airline yield man- The authors thank Eduardo Messmacher for implementing the agement: Optimal bid prices for single-hub problems algorithms with cancellation and overbooking. They are also grate- without cancellations. Working paper,http://gunther.smeal. ful to two anonymous referees for many useful suggestions. psu.edu/13888.html. Karaesman,I.,G. van Ryzin. 1998. Overbooking with substi- tutable inventory classes. Working paper,http://www- References 1.gsb.columbia.edu/divisions/dro/working%20papers/DRO- Belobaba,P. 1987. Air travel demand and airline seat inventory 2002-11.pdf. management. Ph.D. thesis,MIT,Cambridge,MA. Lee,T.,M. Hersh. 1993. A model for airline seat inventory control Bertsekas,D. 1995. Dynamic Programming and Optimal Control,Vol. I. with multiple seat bookings. Transportation Sci. 27 1252–1265. Athena Scientific,Belmont,MA. Littlewood,K. 1972. Forecasting and control of passengers. Proc. ,D. Castañon. 1999. Rollout algorithms for stochastic schedul- AGIFORS 12th Annual Sympos.,95–128. ing problems. J. Heuristics 5 89–108. McGill,J.,G. van Ryzin. 1999. Revenue management: Research ,J. Tsitsiklis. 1998. Neuro-Dynamic Programming. Athena Scien- overview and prospects. Transportation Sci. 33 233–256. tific,Belmont,MA. Popescu,I. 1999. Applications of optimization in probability, , ,C. Wu. 1997. Rollout algorithms for combinatorial opti- finance and revenue management. Ph.D. thesis,Applied Math- mization. J. Heuristics 3 245–262. ematics Department and Operations Research Center,MIT, Bertsimas,D.,J. Tsitsiklis. 1997. Introduction to Linear Optimization. Cambridge,MA. Athena Scientific,Belmont,MA. Robinson,L. 1995. Optimal and approximate control policies for air- ,C. Teo,R. Vohra. 1999. Greedy,randomized and approxi- line booking with sequential nonmonotonic fare classes. Oper. mate dynamic programming algorithms for facility location Res. 43(2) 252–263. problems. Working paper,http://www.fba.nus.edu.sg/qm/ Rothstein,M. 1971. An airline overbooking model. Transportation wkpaper/Index98.html. Sci. 5 180–192. Bitran,G.,S. Mondschein. 1995. An application of yield manage- . 1974. Hotel overbooking as a Markovian sequential decision ment to the hotel industry considering multiple day stays. process. Decision Sci. 5 389–404. Oper. Res. 43(3) 427–443. Simpson,R. 1989. Using network flow techniques to find shadow Brumelle,S.,J. McGill. 1993. Airline seats allocation with multiple prices for market and seat inventory control. Memorandum nested fare classes. Oper. Res. 41 127–137. M89-1,MIT Flight Transportation Laboratory,Cambridge, Chen,D. 1998. Network flows in hotel yield management. Tech- MA. nical report TR1225,Cornell University,School of Operations Subramanian,S. S. J.,C. Lautenbacher. 1999. Airline yield Research and Industrial Engineering,http://citeseer.nj.nec.com management with overbooking,cancellations and no-shows. /363544.html. Transportation Sci. 33 147–167. Chen,V.,D. Gunther,E. Johnson. 2000. An approach to airline yield Talluri,K.,G. van Ryzin. 1998. An analysis of bid-price con- management based on a Markov decision problem. Working trols for network revenue management. Management Sci. 44 paper,http://citeseer.nj.nec.com/chen98markov.html. 1577–1593. Downloaded from informs.org by [18.172.5.91] on 05 August 2016, at 07:10 . For personal use only, all rights reserved. Chi,Z. 1995. Airline yield management in a dynamic network Williamson,E. 1992. Airline network seat control. Ph.D. thesis,MIT, environment. Ph.D. thesis,MIT,Operations Research Center, Cambridge,MA. Cambridge,MA. Wollmer,R. 1992. An airline seat management model for a sin- Cook,T. 1998. Sabre soars. ORMS Today 6 27–31. gle leg route when lower fare classes book first. Oper. Res. 40 26–37. Received: October 2000; revision received: September 2001; accepted: January 2002. Transportation Science/Vol. 37,No. 3,August 2003 277