Airline Yield Management with Overbooking, Cancellations, and No-Shows

JANAKIRAM SUBRAMANIAN

Integral Development Corporation, 301 University Avenue, Suite 200, Palo Alto, California 94301

SHALER STIDHAM JR.

Department of Operations Research, CB 3180, Smith Building, University of North Carolina, Chapel Hill, North Carolina 27599-3180

CONRAD J. LAUTENBACHER

NationsBank, 100 N. Tryon St., NC1-007-12-3, Charlotte, North Carolina 28255-0001

We formulate and analyze a Markov decision process (dynamic programming) model for airline seat allocation (yield management) on a single-leg flight with multiple fare classes. Unlike previous models, we allow cancellation, no-shows, and overbooking. Additionally, we make no assumptions on the arrival patterns for the various fare classes. Our model is also applicable to other problems of revenue management with perishable commodities, such as arise in the hotel and cruise industries. We show how to solve the problem exactly using dynamic programming. Under realistic conditions, we demonstrate that an optimal booking policy is characterized by state- and time-dependent booking limits for each fare class. Our approach exploits the equivalence to a problem in the optimal control of admission to a queueing system, which has been well studied in the queueing-control literature. Techniques for efficient implementation of the optimal policy and numerical examples are also given. In contrast to previous models, we show that 1) the booking limits need not be monotonic in the time remaining until departure; 2) it may be optimal to accept a lower-fare class and simultaneously reject a higher-fare class because of differing cancellation refunds, so that the optimal booking limits may not always be nested according to fare class; and 3) with the possibility of cancellations, an optimal policy depends on both the total capacity and the capacity remaining. Our numerical examples show that revenue gains of up to 9% are possible with our model, compared with an equivalent model omitting the effects of cancellations and no-shows. We also demonstrate the computational feasibility of our approach using data from a real-life airline application.

In the airline industry, the practice of selling iden- cancellation (e.g., nonrefundability or partial re- tical seats for different prices to maximize revenues fundability). The common thread in all these exam- is commonly referred to as yield management (YM) ples is the perishability of the commodity: a seat on (or seat inventory control). Yield management is an a particular flight is worthless after the flight de- example of a more general practice known as reve- parts, just as an unoccupied hotel room generates no nue management (RM) or perishable inventory con- revenue. trol, in which a commodity or service (such as the The YM problem studied in this paper can be use of a hotel room on a particular date) is priced described in its most general form as follows. Con- differently depending on various restrictions on sider a single-leg flight on an airplane of known booking (e.g., advance purchase requirements) or capacity C. Passengers can belong to one of m fare 147 Transportation Science 0041-1655 / 99 / 3302-0147 $01.25 Vol. 33, No. 2, May 1999 © 1999 Institute for Operations Research and the Management Sciences

Copyright © 1999. All rights reserved. 148 / J. SUBRAMANIAN, S. STIDHAM, JR., AND LAUTENBACHER classes, class 1 corresponding to the highest fare and lated papers are ALSTRUP et al (1986), CHATWIN class m to the lowest. Booking requests in each fare (1992), GALLEGO and VAN RYZIN (1994, 1997), and class arrive according to a time-dependent process. YOUNG and VAN SLYKE (1994). Based on the number of seats already booked, we A common assumption in many of the earlier pa- must decide whether to accept or reject each re- pers (e.g., Belobaba (1987), Belobaba (1989), quest. Passengers who have already booked may Wollmer (1992), and Brumelle and McGill (1993)) cancel (with known time- and class-dependent prob- has been that all requests for a fare class arrive ability) at any time up to the departure of the flight. earlier than the requests for higher fare classes. At the time of cancellation, the passenger is re- This assumption is known to be unrealistic. Robin- funded an amount that may be time and class de- son (1995) relaxes this assumption, but still as- pendent. Passengers can also be no-shows at the sumes that different fare classes book during non- time of departure. No-shows are refunded an overlapping time intervals. Lee and Hersh (1993) amount that may be different from the amount re- consider the seat inventory control problem without funded for cancellation. Overbooking is allowed, making any assumption on the arrival pattern. They with corresponding penalties determined by an use dynamic programming methodology to solve for overbooking penalty-cost function. the optimal policy. They do not, however, permit Although, for concreteness, we use the terminol- cancellations, no-shows, or overbooking. ogy of airline yield management in this paper, many The rest of this paper is organized as follows. We of our results are also applicable to other problems of begin in Section 1 with a simple extension of the RM. For example, in the hotel industry, rooms are model of Lee and Hersh (1993) that incorporates sold at different rates, depending on the time until cancellations, no-shows, and overbooking (Model 1). arrival and/or cancellation restrictions. Refunds for The cancellation and no-show probabilities, al- cancellation may also depend on the rate. though allowed to be time dependent, are assumed Our basic approach is to exploit the equivalence of to be the same for all classes. There are no refunds this YM problem to a problem in the optimal control for cancellations or no-shows, but the fare paid in of arrivals to a queueing system, which has been the each class may be time dependent. (Later, in Section subject of an extensive literature over the past twen- 2, we show how to accommodate refunds by an ty-five years (see JOHANSEN and STIDHAM (1980), equivalent-charging scheme borrowed from queue- STIDHAM (1984, 1985, 1988), and STIDHAM and WE- ing-control theory, which results in a net fare that is BER (1993) for surveys). In this way, we are able to time dependent, even if the gross fare is not.) As in solve a substantially more versatile and realistic Lee and Hersh (1993), the planning horizon is di- model than those previously studied in the litera- vided into discrete time periods, in each of which at ture on airline seat allocation. most one event (reservation request or cancellation) Among the references most relevant to this paper can occur. We formulate the problem as a finite- are (in chronological order) LITTLEWOOD (1972), BE- horizon, discrete-time Markov decision process LOBABA (1987, 1989), CURRY (1990), WOLLMER (MDP) in which the state variable is the total num- (1992), BRUMELLE and MCGILL (1993), ROBINSON ber of seats already booked. With booking requests (1995), and LEE and HERSH (1993), all of whom viewed as customer arrivals and cancellations as consider the single-leg problem. Belobaba (1987) service completions, the problem becomes equiva- provides a comprehensive overview of the seat in- lent to a problem in the optimal control of arrivals to ventory control problem and the issues involved. See a queueing system with an infinite number of serv- Belobaba (1989), Lee and Hersh (1993), and LAUT- ers. Exploiting this equivalence, we show that the ENBACHER and STIDHAM (1996) for surveys and ad- YM problem satisfies well-known sufficient condi- ditional references. SMITH,LEIMKUHLER, and DAR- tions for an optimal policy to be monotonic, which, in ROW (1992) review yield management practices as this context, translates to the existence of time-de- they are implemented at American Airlines. pendent booking limits for each class. Previously, WEATHERFORD and BODILY (1992) (see also the optimality of such a policy has been established WEATHERFORD,BODILY, and PFEIFER (1993)) pro- in the YM literature only for the problem without vide a useful taxonomy of perishable asset manage- cancellations or no-shows, and then by complicated ment (PARM) problems, which include YM and re- arguments apparently dependent on the particular lated problems. (In their terminology, our problem is structure of the YM problem. (A1-B1-C1-D1-EI-F3-G3-H3-I1-J1-K1-L5-M2-N3).) In Section 2, we extend the basic model to allow WILLIAMSON (1992) and TALLURI and VAN RYZIN class-dependent cancellation and no-show probabil- (1996) study the multi-leg problem (network RM), ities and refund amounts, with refunds at the time which we do not consider in this paper. Other re- of cancellation or no-show (Model 2). In this case, the

Copyright © 1999. All rights reserved. OVERBOOKING, CANCELLATIONS, AND NO-SHOWS / 149 formulation as an MDP requires a multidimensional putational feasibility of our model, using data from a state variable, because now one must know the real-life airline application with six fare classes and number of seats reserved in each fare class to predict a 331-day booking horizon for a plane with 100 future cancellations and no-shows (because the seats. Next, we consider Model 2, with class-depen- rates at which these occur are now allowed to be dent cancellation and no-show probabilities. We class dependent). The curse of dimensionality pre- solve a small but realistic model, using the multidi- cludes solving problems of more than modest size, mensional MDP formulation discussed in Section 2. especially considering that an airline is faced with Then, we compare this (exact) solution to one-di- solving such problems for hundreds of legs on a daily mensional approximations. basis. In the special case, in which cancellation and In Appendix A, we show how to apply our results no-show probabilities are not class dependent, how- to solve the continuous-time seat-allocation prob- ever, we show how to transform the problem into an lem, in which requests for seats in each fare class equivalent MDP in which the expected lost revenue arrive according to (time-dependent) Poisson pro- from cancellation or no-show refunds is subtracted cesses (cf. Lee and Hersh, 1993). We give two (equiv- from the fare paid at the time of booking. (This is the alent) approaches, one based on uniformization (LIPP- equivalent charging scheme referred to above.) This MAN, 1975; SERFOZO, 1979) and the other on a time equivalent problem satisfies the conditions of Model discretization (Lee and Hersh, 1993). 1, for which the total number of seats currently Appendix B presents our heuristic approximation booked is a sufficient state description, thus reduc- for the case of class-dependent cancellation and no- ing the problem to a one-dimensional MDP and show probabilities. avoiding the curse of dimensionality. The assumption that cancellation and no-show probabilities are class independent is not realistic, but still represents an improvement on previous models, which 1. BASIC DISCRETE-TIME MODEL (MODEL 1) ignore the effects of cancellations and no-shows al- IN THIS SECTION, we introduce and analyze our first together. (In Appendix B, we propose a heuristic discrete-time model for airline seat allocation (Mod- approximation for the case of class-dependent can- el 1), which generalizes the single-seat model of Lee cellation and no-show probabilities which retains and Hersh (1993). There are m fare classes and N the one-dimensional state variable and, thus, decision periods or stages, numbered in reverse greatly enlarges the space of solvable problems.) chronological order, n ϭ N, N Ϫ 1, . . . , 1, 0, with Section 3 contains our numerical results. First, we stage N corresponding to the opening of the flight for apply Model 1 to several single-leg YM problems reservations and stage 0 corresponding to its depar- with cancellations, no-shows, and overbooking, in ture. At each stage, we assume that one (and only which the (time-dependent) cancellation and no- one) of the following events occurs: (1) an arrival of show probabilities are the same for all classes. Our a customer (i.e., a request for a seat) in fare class i, numerical results demonstrate that the optimal i ϭ 1, ..., m; (2) a cancellation by a customer booking limits may not be nested by fare class, in currently holding a reservation; or (3) a null event contrast to the situation without cancellation or (represented as the arrival of a customer of class 0). overbooking. More accurately, the order in which (In Appendix A, we show how this assumption is the classes are nested may be time dependent, based on the net fare: the gross fare minus the expected satisfied naturally when a continuous-time problem cancellation and/or no-show refund. We also com- is approximated either by uniformization or time- pare our results to those for dynamic-programming discretization.) models that do not allow for the possibility of can- In this model, cancellations and no-shows occur at cellation and/or no-shows, such as the model of Lee class-independent rates, which allows us to use a and Hersh (1993). Even if one refines such models by one-dimensional state variable. Later (in Section 2) subtracting the expected lost revenue caused by can- we shall relax this assumption. cellations and no-shows from the fare, these models Let pin denote the probability of a request for a do not include the cancellation and no-show proba- seat in fare class i in period n. Similarly, let qn(x) bilities in their calculations of future state-transi- and p0n(x) denote the probability of a cancellation tion probabilities. We show, by means of numerical and a null event, respectively, in period n, given that examples, that our model can yield substantial in- the current number of reserved seats equals x.We creases in revenues—up to 9% when compared to an assume that qn(x) is a nondecreasing and concave equivalent model omitting the effects of cancella- function of x, for each n. By our assumption that, at tions and no-shows. We also demonstrate the com- most, one request or cancellation can occur at each

stage, we have be piecewise linear,

m k ␲͑ ͒ ϭ ͸ ͑ Ϫ ͒ϩ ͸ ϩ ͑ ͒ ϩ ͑ ͒ ϭ y aj y bj , pin qn x p0n x 1, (1) ϭ iϭ1 j 1

where C ϭ b Ͻ b Ͻ ... Ͻ b and a ୑ 0, j ϭ for all x and n ୑ 1. 1 2 k j 1,..., k. In this case, the airline must pay a to If the event occurring at stage n is the arrival of a 1 each of the first b Ϫ b passengers who volunteer to seat request, the system controller (the booking 2 1 take a later flight, a ϩ a to each of the next b Ϫ agent) decides whether or not to accept the request, 1 2 3 b , and so forth. based on the fare class i of the customer and the 2 Our objective is to maximize the expected total net current state x. If the event is a cancellation or null benefit1 of operating the system over the horizon event, then no decision is made. If the booking agent from period N to period 0, starting from state x ϭ 0, accepts a request for a seat in fare class i at stage n, that is, with no seats booked, at the beginning of ୑ the airline earns revenue rin 0. (By allowing the period N. The problem can be formulated as a dis- revenue to depend on n, we shall be able, in subse- crete-time MDP with stages n ϭ N, N Ϫ 1,...,0, quent analysis, to incorporate the effects of cancel- in which the state x is the current number of booked lation and no-shows by the equivalent charging seats. Because our model allows for the possibility of scheme referred to in the Introduction.) overbooking, cancellations, and no-shows, the state There are also no-shows. We assume that, at the variable need not satisfy the constraint x ୏ C,aswe time of departure, each customer holding a seat have noted above. However, because we start with ␤ reservation is a no-show with probability . Let Y(x) no seats booked at stage N and, at most, one seat denote the number of people who show up for the request can be accepted at each stage (because, at flight, given that the number of reserved seats is x most, one arrives), it follows that at each stage n, just before departure, so that x Ϫ Y(x) is the number x ୏ N Ϫ n. (To reduce the computational burden in of no-shows. Because each customer has a probabil- applications, one may wish to introduce an addi- ity 1 Ϫ ␤ of showing up for the flight, it is clear that tional constraint, x ୏ C ϩ v, where v is the over- Y(x) has a binomial-(x,1Ϫ ␤) distribution. Let C booking pad: the maximal amount by which the denote the capacity of the airplane. If, at the time of airline is willing to overbook. See Remark 4 below.) ϭ the departure, Y(x) y, then we incur an overbook- As a function of the state x in period n, let Un(x) ing penalty ␲(y). We assume that ␲(y) is non-neg- denote the maximal expected net benefit of operat- ative, convex, and nondecreasing in y ୑ 0, with ing the system over periods n to 0. The optimal value ␲ ϭ ୏ (y) 0 for y C. functions, Un, are determined recursively by

REMARK 1. Note that we are assuming that requests m ͑ ͒ ϭ ͸ ͕ ϩ ͑ ϩ ͒ ͑ ͖͒ for seats are independent of the number of seats Un x pinmax rin UnϪ1 x 1 , UnϪ1 x already booked (a realistic assumption), whereas iϭ1 cancellation and no-show probabilities depend on ϩ q ͑ x͒U Ϫ ͑ x Ϫ 1͒ ϩ p ͑ x͒U Ϫ ͑ x͒, the total number of booked seats, but not on the n n 1 0n n 1 number booked in each class. The assumption that 0 р x р N Ϫ n, n у 1, (2) qn(x) is nondecreasing in x expresses the plausible property that the higher the number of seats already U ͑ x͒ ϭ E͓Ϫ␲͑Y͑ x͔͒͒,0р x р N, (3) booked, the higher the probability of a cancellation. 0 The concavity assumption is needed for technical where Y(x) ϳ Bin(x,1Ϫ ␤). reasons (in the proof that a booking-limit policy is Now we show how to write this optimality equa- optimal). It is satisfied in all the applications dis- tion in an equivalent form that is characteristic of a cussed in this paper. It holds, for example, when problem in the optimal control of admission to a ϭ ⅐ qn(x) qn x, as is the case when the discrete-time queueing system. model is derived from a continuous-time model with ϭ͚m Let pn : iϭ1 pin, and let Rn be a random vari- ϭ ϭ a nonhomogeneous Poisson arrival process (see Ap- able with probability mass function P{Rn rin} pendix A).

1 ␲ An alternative objective function is the expected total dis- REMARK 2. The assumption that (y) is convex and counted net benefit. The analysis of this problem is substantially nondecreasing is realistic. For example, ␲(y) could the same. See JANAKIRAM,STIDHAM, and SHAYKEVICH (1994).

Copyright © 1999. All rights reserved. OVERBOOKING, CANCELLATIONS, AND NO-SHOWS / 151 ϭ ୑ pin/pn, i 1,..., m, n 1. Let proof is not affected by permitting dependence on n, as we do in our seat-allocation model. See Johansen ͑ ͒ ϭ ͕ ϩ ͑ ϩ ͒ ͑ ͖͒ Vn x, r : max r UnϪ1 x 1 , UnϪ1 x , and Stidham (1980) and Helm and Waldmann for all x ϭ 0, 1, . . . , N Ϫ n and real numbers r. (1984), for example, for related models allowing time (Here, r is a realization of the random variable R . dependence.) To apply their results to our model, we n ϭ In the present problem, the only values of r that first need to verify that the function, U0(x) ϭ E[Ϫ␲(Y(x))], is concave and nonincreasing in x to have positive probability are of the form r rin for some i ϭ 1, ..., m. In more general queueing- start the induction. We use the following result from the theory of stochastic ordering. (See Example control models, the distribution of Rn may be arbi- trary, with support [0, ϱ).) Then, substituting in Eq. 6.A.2 in SHAKED and SHANTHIKUMAR, 1994.) 2 and using Eq. 1, we obtain the equivalent optimal- LEMMA 1. Let f(y), y ୑ 0, be a nondecreasing convex ity equations function. For each non-negative integer x, let Y(x) be a binomial-(x, ␥) random variable (0 Ͻ ␥ Ͻ 1) and Un͑ x͒ ϭ pnE͓Vn͑ x, Rn͔͒ ϩ qn͑ x͒UnϪ1͑ x Ϫ 1͒ let h(x):ϭ E[f(Y(x))]. Then h(x) is nondecreasing

ϩ͑1 Ϫ pn Ϫ qn͑ x͒͒UnϪ1͑ x͒, convex in x ʦ {0, 1, . . .}.

-р x р N Ϫ n, n у 1 (4) Because ␲٪ is convex and nondecreasing by as 0 sumption, it follows from Lemma 1 and Eq. 6 that ͑ ͒ ϭ ͕ Ϫ ͑ ͑ ͒ Ϫ ͑ ϩ ͒͒ ͖ ٪ Vn x, r max r UnϪ1 x UnϪ1 x 1 ,0 U0 is concave and nonincreasing. Then, by induction on n (cf. Theorem 1 in Lippman and Stidham, ϩU Ϫ ͑ x͒, ٪ n 1 1977), Un is concave and nonincreasing, so that Ϫ ϩ ϭ 0 р x р N Ϫ n, n у 1 (5) Un(x) Un(x 1) is nondecreasing in x 0, Ϫ Ϫ Ϫ ϩ 1,...,N n 1. The difference, Un(x) Un(x ͑ ͒ ϭ ͓Ϫ␲͑ ͑ ͔͒͒ р р U0 x E Y x ,0 x N. (6) 1), is the opportunity cost of accepting a booking at stage n ϩ 1, that is, the expected loss in future In this form, the optimality equations can be seen to revenue that would result from accepting the book- be formally equivalent to those for optimal control of ing. In our model, this opportunity cost plays the admission to a queueing system over a finite horizon same role as the expected marginal seat revenue as studied, for example, in LIPPMAN and STIDHAM (EMSR) in the pioneering work of Belobaba (1989) (1977) (see also STIDHAM, 1978; Johansen and (see also Brumelle and McGill, 1993; Wollmer, Stidham, 1980; HELM and WALDMANN, 1984; 1992). It is also the optimal bid price for our single- Stidham, 1984, 1985, 1988). Lippman and Stidham leg problem, in the sense of Williamson (1992) (see (1977) study a queue with a Poisson arrival process also Talluri and van Ryzin, 1996). For each stage n and a state-dependent exponential service mecha- and each fare class i, define the booking limit b as nism, which, after uniformization, reduces to a dis- in crete-parameter MDP whose finite-horizon optimal- bin :ϭ min͕ x: UnϪ1͑ x͒ Ϫ UnϪ1͑ x ϩ 1͒ Ͼ rin͖. (7) ity equations (see Eqs. 2 and 3 in Lippman and Ϫ ϩ ୏ ϭ Stidham, 1977) have exactly the same structure as (If UnϪ1(x) UnϪ1(x 1) rin for all x, set bin ϱ Ϫ ϩ our optimality Eqs. 4 and 5. (In Appendix A, we shall .) Because UnϪ1(x) UnϪ1(x 1) is nondecreas- discuss how to use uniformization to apply our ing in x, bin is well defined, and an optimal policy model to a continuous-time seat-allocation problem will have the form in which seat requests in each fare class arrive according to a Poisson process, and cancellations are accept a fare-class-i request in state x at stage n governed by exponential distributions.) The model of if and only if 0 р x Ͻ b . Lippman and Stidham (1977) accommodates a hold- in ing cost, which is a convex, nondecreasing function ϭ In the notation of Lee and Hersh (1993), bin of the state. In our model, the holding cost is iden- Ϫ ϩ C sˆ i(n) 1, where sˆ i(n) is the critical booking tically zero. capacity for class i. An optimal policy will therefore Lippman and Stidham (1977) use induction on the accept a fare-class i request at stage n if and only if remaining number of stages, n, to prove that the s ୑ sˆ (n), where s ϭ C Ϫ x is the number of seats i ٪ optimal value function, Un , is concave and nonin- still available (the booking capacity). creasing, from which it follows that an optimal admission policy is monotonic in the state. (Although REMARK 3. In contrast to Lee and Hersh (1993), we their model assumes that rewards, arrival rates, allow the fares rin to depend on n as well as i, and and service rates do not depend on n, the inductive we do not assume that the ordering of fares by class

Copyright © 1999. All rights reserved. 152 / J. SUBRAMANIAN, S. STIDHAM, JR., AND LAUTENBACHER is the same for all stages n. So, there may be two U0͑ x͒ ϭ E͓Ϫ␲͑Y͑ x͔͒͒,0р x р C ϩ v. (10) classes i and j and stages n and k such that We shall use the optimality equations in this form to Ͻ ͑ ͒ Ϫ ͑ ϩ ͒ р rin UnϪ1 x UnϪ1 x 1 rjn , solve for the optimal policy in the numerical examples in Section 3. whereas у ͑ ͒ Ϫ ͑ ϩ ͒ Ͼ rik UkϪ1 x UkϪ1 x 1 rjk . 2. GENERAL MODEL WITH CLASS-DEPENDENT CANCELLATION AND NO-SHOW RATES (MODEL 2) In this case, it is optimal to reject a class i customer and accept a class j customer in state x at stage n, IN THIS SECTION, we introduce our most general whereas the reverse is true at stage k. It follows that model (Model 2), which extends Model 1 by allowing class-dependent cancellation and no-show probabil- the ordering of the booking limits bin in i may not be the same for all n. In this case, the nesting of the ities, as well as refunds at the time of cancellation or classes may be time dependent. As we shall see in no-show. the next section, this phenomenon can occur when As with Model 1, there are m fare classes and N ϭ Ϫ the fare in each class is adjusted by subtracting the stages, numbered in reverse order, n N, N expected cost of cancellation and no-show refunds. 1,...,1,0.Letxi denote the number of seats currently reserved by class i customers, i ϭ 1,..., EMARK ϭ R 4. As noted above, to reduce the computa- m. Our state variable is now x (x1,...,xm), the tional burden, it may be advantageous to introduce a reservation vector. Let pin(x), qin(x), and p0n(x), maximum overbooking level v (called the overbook- respectively, denote the probabilities of a request for ing pad), resulting in an additional state constraint, a seat in fare class i, a cancellation by a customer of 0 Ͻ x Ͻ C ϩ v, at each stage n. The qualitative class i, and a null event in period n, given the properties of optimal policies, such as monotonicity reservation vector x. Once again, we assume that of an optimal policy and the existence of booking one and only one of these events occurs during each limits, continue to hold in this setting. To see that stage. Note that we are now allowing the arrival this is the case, note that we can incorporate this probability to depend on the current system state, as constraint into the recursions as follows. well as distinguishing between cancellations in dif- Ϫ At stage n 1, given the value functions UnϪ1(x), ferent fare classes and allowing the probability of a ୏ ୏ ϩ ϭ ϩ Ϫ for 0 x C v, define UnϪ1(x): UnϪ1(C v) cancellation in class i to depend upon the detailed Ϫ Ϫ ୑ ϩ ϭ M(x C v), for x C v, where M is a positive state description, x (x1,...,xm), rather than just ୑ ϭ͚ number such that M maxi,n{rin}. Note that this the total number of seats booked, x i xi. (In definition ensures that booking requests will always applications, as we shall see, the class i cancellation ϩ be rejected in state C v at stage n, while preserv- probability will typically depend on x through xi, the ing the properties of monotonicity and concavity of number of seats currently reserved in class i.) By ٪ UnϪ1 and hence the monotonicity properties of an our assumption that, at most, one request or cancel- optimal policy at stage n. It then follows by the lation can occur at each stage, we have inductive argument given previously that the func- m m tion Un(x) will also be nonincreasing and concave for ୏ ୏ ϩ ͸ ͑ ͒ ϩ ͸ ͑ ͒ ϩ ͑ ͒ ϭ 0 x C v. pin x qin x p0n x 1, Now, define the opportunity cost functions iϭ1 iϭ1 u (x):ϭ U (x) Ϫ U (x ϩ 1), 0 ୏ x ୏ C ϩ nϪ1 nϪ1 nϪ1 ୑ v. Because an optimal policy now rejects all booking for all x and n 1. requests in state C ϩ v, the optimality Eqs. 2 and 3 If the event occurring at stage n is the arrival of a reduce to seat request, the system controller (the booking agent) decides whether or not to accept the request, m based on the fare-class i of the customer and the ͑ ͒ ϭ ͸ ͕ Ϫ ͑ ͒ ͖ current state x. If the event is a cancellation or null Un x pinmax rin unϪ1 x ,0 iϭ1 event, then no decision is made. A customer whose request for a seat in fare class i is accepted at stage ϩ xqnUnϪ1͑ x Ϫ 1͒ ϩ ͑1 Ϫ xqn͒UnϪ1͑ x͒, n pays a fare rˆ in. A customer in class i who cancels in decision period n receives a refund c . 0 р x р C ϩ v Ϫ 1, n у 1, (8) in With regard to no-shows, we now assume that, at

U ͑C ϩ v͒ ϭ ͑C ϩ v͒q U Ϫ ͑C ϩ v Ϫ 1͒ the time of departure, each customer of class i has a n n n 1 ␤ probability i of being a no-show, dependent on the ϩ ͑1 Ϫ ͑C ϩ v͒qn͒UnϪ1͑C ϩ v͒, n у 1, (9) class i but independent of everything else. A class i

customer who is a no-show is refunded an amount where ei is the ith unit m-vector. di. Let Yi(xi) denote the number of people of class i The structure of this MDP reveals that the seat- who show up for the flight, given that the number of allocation problem is again essentially equivalent to reserved class i seats is xi just before departure, so a problem of admission control to a queueing sys- Ϫ that xi Yi(xi) is the number of no-shows in class i. tem, in which the customers are the requests for Because each class i customer has a probability 1 Ϫ seats and the cancellation process plays the role of ␤ i of showing up for the flight, it is clear that Yi(xi) service mechanism. Now, however, because the dif- Ϫ ␤ ϭ has a binomial-(xi,1 i) distribution. Let Y(x): ferent classes of customers have different cancella- ͚m iϭ1 Yi(xi) denote the total number of customers tion (service) rates and refunds, it is necessary to who show up for the flight. If, at the time of depar- keep track of the number of customers in each class, ture, Y(x) ϭ y, we incur an overbooking penalty rather than just the total number, as a cancellation ␲ ␲ (y). As before, we assume that (y) is non-nega- from a reservation vector heavily laden with passen- ୑ tive, convex, and nondecreasing in y 0, with gers prone to cancel has a higher probability of oc- ␲ ϭ ୏ (y) 0 for y C. currence than one from a plane filled with those Our objective is to maximize the expected total net unlikely to do so. benefit of operating the system over the horizon We now show how to transform the optimality ϭ from period N to period 0, starting from state x (0, Eqs. 11 into an equivalent set of equations in which ...,0),that is, with no seats booked, at the begin- all expected costs (caused by cancellations and no- ning of period N. The problem can be formulated as shows) are assessed at the instant of admission an MDP with stages n ϭ N, N Ϫ 1, . . . , 0, in which (booking of a seat) along with the reward (payment the state, x ϭ (x ,...,x ), is the current reserva- 1 m of the fare). To do this, we borrow a technique (the tion vector. Once again, because our model allows equivalent charging scheme) from the queueing-con- for the possibility of overbooking, cancellations, and trol literature, first proposed in Lippman and no-shows, the state variable need not satisfy the Stidham (1977). This transformation will facilitate constraint x ϭ͚ x ୏ C. However, because we start i i the reduction of the problem to a one-dimensional with no seats booked at stage N and, at most, one seat request can be accepted at each stage (because, MDP in certain cases. ϭ͚ ୏ Ϫ Let Hn(x) denote the total expected loss of reve- at most, one arrives), it follows that x i xi N ϭ ϭ ୑ nue over periods n to 0 caused by cancellations and n at stage n. Let Xn : {x (x1,...,xm): xi 0, ϭ ͚ ୏ Ϫ no-shows. (Another interpretation for Hn(x) is that i 1,...,m; i xi N n}. Thus, at each stage ʦ it is the negative of the value function for the policy n, there is an implicit constraint, x Xn. (As with ͚ that rejects all arrivals, starting from state x at Model 1, one can also introduce the constraint, i ୏ ϩ stage n.) Then, the functions H are given by the xi C v, where v is an overbooking pad.) n ʦ recursive equations As a function of the state, x Xn, in period n, let ˆ Un(x) denote the maximal expected net benefit of operating the system over periods n to 0. The opti- m ˆ ͑ ͒ ϭ ͸ ͑ ͒ ͑ ͒ mal value functions, Un, are determined recursively Hn x pin x HnϪ1 x by iϭ0

m m ϩ ͸ ͑ ͒͑ ϩ ͑ Ϫ ͒͒ ˆ ͑ ͒ ϭ ͸ ͑ ͒ ͕ ϩ ˆ ͑ ϩ ͒ ˆ ͑ ͖͒ qin x cin HnϪ1 x ei , Un x pin x max rˆ in UnϪ1 x ei , UnϪ1 x ϭ iϭ1 i 1

m x ʦ X , n у 1 (13) ϩ ͸ ͑ ͒͑Ϫ ϩ ˆ ͑ Ϫ ͒͒ n qin x cin UnϪ1 x ei iϭ1 m m ͑ ͒ ϭ ͸͑ Ϫ ͑ ͒͒ ϭ ͸ ␤ ʦ ϩ ͑ ͒ ˆ ͑ ͒ H0 x Eͫ xi Yi xi diͬ i xidi , x X0 . p0n x UnϪ1 x , iϭ1 iϭ1 (14) x ʦ Xn , n у 1; (11) ϭ ˆ ϩ ୑ m Now, define Un(x): Un(x) Hn(x), n 0. Then, ˆ ͑ ͒ ϭ Ϫ␲͑ ͑ ͒͒ Ϫ ͸͑ Ϫ ͑ ͒͒ Un represents the maximal expected controllable U0 x Eͫ Y x xi Yi xi diͬ , iϭ1 net benefit of operating the system over periods n to 0, because Hn(x) is an unavoidable loss of revenue. x ʦ X0 , (12) Using Eqs. 11, 12, 13, and 14, we obtain the follow-

Copyright © 1999. All rights reserved. 154 / J. SUBRAMANIAN, S. STIDHAM, JR., AND LAUTENBACHER ϭ ϭ ing recursive optimality equations satisfied by the ASSUMPTION 1. qin(x) qin(xi), for all x (x1,..., ϭ functions Un. xm), where qin(0) 0 and qin(xi) is nondecreasing ୑ ϭ ϭ Ϫ in xi 1, i 1,..., m, n N, N 1,..., 1. m ͑ ͒ ϭ ͸ ͑ ͒ ͕ Ϫ ͓ ͑ ϩ ͒ Ϫ ͑ ͔͒ Assumption 1 says that the probability of a class i U x p x max rˆ H Ϫ x e H Ϫ x n in in n 1 i n 1 cancellation in a particular period depends only on iϭ1 the number of class i seats currently booked, and not

ϩ UnϪ1͑x ϩ ei͒, UnϪ1͑x͖͒ on the reservations in other classes. This is a reasonable assumption in practice. The following ϭ m lemma can be proved easily by induction on n ϩ ͸ ͑ ͒ ͑ Ϫ ͒ ϩ ͑ ͒ ͑ ͒ 1,..., N. qin x UnϪ1 x ei p0n x UnϪ1 x , iϭ1 LEMMA 2. Under Assumption 1,

x ʦ Xn , n у 1; (15) m ͑ ͒ ϭ ͸ ͑ ͒ у Hn x Hin xi , n 0, (17) ϭ U0͑x͒ ϭ E͓Ϫ␲͑Y͑x͔͒͒, x ʦ X0 . (16) i 1

These equations are equivalent to the original opti- where the functions Hin satisfy the recursive equa- ϭ mality Eqs. 11 and 12 in the sense that they gener- tions (i 1,..., m) ate the same optimal booking policy. It follows from Hin͑ xi͒ ϭ ͑1 Ϫ qin͑ xi͒͒Hi,nϪ1͑ xi͒ Eq. 15 that this policy takes the form ϩ q ͑ x ͒͑c ϩ H Ϫ ͑ x Ϫ 1͒͒, accept a class i request in stage n with in i in i,n 1 i

reservation vector x xi у 0, n у 1 (18) H ͑ x ͒ ϭ ␤ x d , x у 0. (19) N rˆ in Ϫ ͓HnϪ1͑x ϩ ei͒ Ϫ HnϪ1͑x͔͒ i0 i i i i i ϭ ϩ Ϫ Ͼ U Ϫ ͑x͒ϪU Ϫ ͑x ϩ e ͒. Now, let Gin(xi): Hi,nϪ1(xi 1) Hi,nϪ1(xi). It n 1 n 1 i ϭ ϩ Ϫ follows from Eq. 17 that Gin(xi) HnϪ1(x ei) That is, in determining whether or not to accept a HnϪ1(x), the marginal expected cancellation cost seat request in fare-class i, we should first calculate associated with a fare-class i booking in state x at the expected net revenue from the booking by sub- stage n, which appears in Eq. 15 and was discussed tracting the marginal expected cancellation cost, above. Moreover, it follows from Eqs. 18 and 19 that ϩ Ϫ the functions G satisfy the recursive equations HnϪ1(x ei) HnϪ1(x), from the gross fare re- in (i ϭ 1,..., m) ceived, rˆ in. This quantity should then be compared Ϫ to the opportunity cost of the booking, UnϪ1(x) ͑ ͒ ϭ ͑ ͑ ϩ ͒ Ϫ ͑ ͒͒ ϩ Gin xi qi,nϪ1 xi 1 qi,nϪ1 xi ci,nϪ1 UnϪ1(x ei), and the request should be accepted if and only if the former exceeds the latter. ϩ ͑1 Ϫ qi,nϪ1͑ xi ϩ 1͒͒Gi,nϪ1͑ xi͒ In principle, the recursive optimality Eqs. 15 and ϩ ͑ ͒ ͑ Ϫ ͒ 16 can be used to calculate the optimal value func- qi,nϪ1 xi Gi,nϪ1 xi 1 , tions, Un, and the associated optimal booking policy, у у ʦ xi 0, n 2 (20) for each fare class i, reservation vector x Xn, and stage n ϭ 0, 1, . . . , N. (This is just the standard Gi1͑ xi͒ ϭ ␤idi , xi у 0. (21) backwards-recursive algorithm of dynamic programming.) However, for our model, in its present, Further simplifications occur if we strengthen As- very general form, this algorithm will often not be sumption 1, by requiring that the cancellation rate computationally feasible because of the curse of di- in each fare class in each period be proportional to mensionality—specifically, the combinatorial explo- the number of seats currently booked in that class: sion in the number of possible states, x ϭ (x ,..., 1 ASSUMPTION 1Ј. q (x) ϭ x q , for all x, where q Ͼ 0, x ), which grows exponentially in m and N. (How- in i in in m i ϭ 1,...,m, n ϭ N, N Ϫ 1,...,1. ever, it is feasible to solve small problems exactly: see Example 5 in Section 3.) For this reason, we look As we shall see (Appendix A), this assumption is for simplifying but realistic assumptions that will reasonable in applications of our discrete-time reduce the size of the state space. model to a system operating in continuous time, in To this end, we begin by making the following which booking requests arrive according to time- assumption. dependent Poisson processes (cf. Lee and Hersh,

1993). In that context, the assumption is equivalent internal effect of the acceptance of the customer to assuming that each customer cancels indepen- (booking request). It is a cost associated directly dently of all other customers, with a cancellation with the accepted customer, namely, the expected rate solely dependent on the customer’s class. amount that will be refunded to that customer re- Using induction on n in Eqs. 20 and 21, we imme- sulting from cancellation or no-show. By contrast, Ϫ ϩ diately obtain the following lemma. the opportunity cost, UnϪ1(x) UnϪ1(x ei), is an external effect, in that it represents the expected LEMMA 3. Under Assumption 1Ј,G (x ) ϭ G (i), in i n loss of revenue from future customers, caused by the independent of x , where the functions G (i) satisfy i n admission of the customer in question. So, we see the recursive equations (i ϭ 1,..., m) that an optimal policy admits the customer (accepts

Gn͑i͒ ϭ qi,nϪ1ci,nϪ1 ϩ ͑1 Ϫ qi,nϪ1͒GnϪ1͑i͒, n у 2; the booking request) if and only if the gross fare (22) exceeds the sum of the internal and external effects.

G1͑i͒ ϭ ␤idi . (23) We are still left with the necessity of evaluating the optimality Eqs. 24 and 25 for each state x ϭ This result makes sense intuitively. It states that ʦ (x1,...,xm) Xn, for each stage n. In other words, the marginal expected cancellation cost associated the curse of dimensionality is still with us. Our next with accepting a request for a seat in fare-class i in result shows that we can reduce the problem to an stage n is simply the expected cancellation cost at- equivalent problem with a one-dimensional state tributable to that particular customer over the re- variable (the total number of booked seats), provided maining horizon, which is independent of the num- that arrivals of booking requests are independent of ber of seats already booked. (It is not difficult to the number of seats already booked, and the indi- solve Eqs. 22 and 23 for an explicit expression for Gi, vidual cancellation rates and no-show probabilities but the expression is complicated and we shall not are independent of the fare class. need it. The recursive Eqs. 22 and 23 are more Љ ϭ ʦ ϭ suitable for numerical calculations.) ASSUMPTION 1 . qin(x) xiqn, for all x Xn, i Ͼ ϭ Ϫ To recapitulate, we have shown how the calcula- 1,..., m, where qn 0, n N, N 1,..., 1. tion of the marginal expected cancellation cost, H n ϭ ʦ ϭ (x ϩ e ) Ϫ H (x), simplifies under Assumption 1Ј,a ASSUMPTION 2. pin(x) pin, for all x Xn, i i n ϭ Ϫ realistic assumption about cancellation rates. In 1,..., m, n N, N 1,..., 1. this case, an optimal seat-allocation policy can be ASSUMPTION 3. ␤ ϭ ␤, i ϭ 1,..., m. found by choosing, in state x at stage n, the maxi- i ϭ Ϫ mizing action in the recursive optimality equations, Let rin : rˆ in Gn(i). which now take the form THEOREM 1. Under Assumptions 1Љ, 2, and 3, the ͑ ͒ Un x optimal value functions, Un(x), depend on x only ϭ͚ through x i xi, and are determined by the recur- m sive optimality equations ϭ ͸ ͑ ͒ ͕ Ϫ ͑ ͒ ϩ ͑ ϩ ͒ ͑ ͖͒ pin x max rˆin Gn i UnϪ1 x ei , UnϪ1 x iϭ1 m ͑ ͒ ϭ ͸ ͕ ϩ ͑ ϩ ͒ ͑ ͖͒ Un x pinmax rin UnϪ1 x 1 , UnϪ1 x m ϭ ϩ ͸ ͑ Ϫ ͒ ϩ ͑ ͒ ͑ ͒ i 1 xiqinUnϪ1 x ei p0n x UnϪ1 x , ϭ i 1 ϩ xqnUnϪ1͑ x Ϫ 1͒

x ʦ Xn , n у 1 (24) m ϩ ͩ 1 Ϫ ͸ p Ϫ xq ͪ U Ϫ ͑ x͒, U ͑x͒ ϭ E͓Ϫ␲͑Y͑x͔͒͒, x ʦ X . (25) in n n 1 0 0 iϭ1 An optimal policy now takes the form 0 р x р N Ϫ n, n у 1 (26) accept a class i request in stage n with reservation vector x U0͑ x͒ ϭ E͓Ϫ␲͑Y͑ x͔͒͒,0р x р N, (27) ϳ Ϫ ␤ N rˆ in Ͼ Gn͑i͒ ϩ UnϪ1͑x͒ Ϫ UnϪ1͑x ϩ ei͒. where Y(x) Bin(x,1 ).

REMARK 5. Using terminology from queueing con- Proof. The proof is by induction on n. By assump- ϭ ϭ Ϫ␲ trol (borrowed in turn from welfare economics), the tion, U0(x) U0(x) E[ (Y(x))] and so depends ϭ͚ ୑ marginal expected cancellation cost, Gn(i), is an on x only through x i xi. Let n 1 and suppose

Copyright © 1999. All rights reserved. 156 / J. SUBRAMANIAN, S. STIDHAM, JR., AND LAUTENBACHER ϭ ϩ Ͻ Ϫ Ϫ UnϪ1(x) UnϪ1(x) for all x. Then, it follows from Un(x 1) rj Gn(j) (which can occur if Gn(i) Ͼ Ϫ Eq. 26 and Assumptions 1, 2, and 3 that Gn(j) ri rj), then it will be optimal to reject fare class i and simultaneously accept fare class j in m period n, even though class i has the higher (gross) ͑ ͒ ϭ ͸ ͕ ϩ ͑ ϩ ͒ ͑ ͖͒ Un x pinmax rin UnϪ1 x 1 , UnϪ1 x Ͼ Ͼ ୑ fare. This could happen if cik cjk, for all n k iϭ1 1, (e.g., if fare class i is fully refundable and fare

m class j is nonrefundable). ϩ ͸ ͑ Ϫ ͒ xiqnUnϪ1 x 1 REMARK 8. We recognize that it is not realistic to iϭ1 assume that cancellation and no-show rates do not m depend on class. We believe, however, that our ϩ ͩ 1 Ϫ ͸ p Ϫ xq ͪ U Ϫ ͑ x͒, n у 1, model is a significant improvement over previous in n n 1 iϭ1 models, in that it explicitly models the effects of cancellations on future seat availability, as well as (28) class-dependent refunds. The class-dependence of ͚ ϩ͚ ϩ where we have used the fact i pin i xiqn refunds permits accurate modeling of the internal ϭ ϭ p0n(x) 1. It follows from Eq. 28 that Un(x) effect, Gn(i), of accepting a request (cf. Remark 5 Un(x) and Eq. 26 holds. This completes the induc- above), whereas the dependence of cancellation and tion and the proof of the theorem. e no-show rates on class influences the solution only through the calculation of the external effect, Un(x) The problem represented by the optimality Eqs. Ϫ ϩ Un(x 1). Typically, the internal effect is a 26 and 27 now has exactly the form of the one- first-order effect, whereas the external effect is a dimensional problem (Model 1) discussed in Section second-order effect. Therefore, we have reason to ϭ Ϫ ϭ 1, with rin rˆ in Gn(i), and qn(x) xqn (a hope that assuming class-independent cancellation concave function of x). So, all the monotonicity re- and no-show rates (as we do, for example, with our sults for Model 1 apply, in particular, the optimality proposed heuristic approximation in Appendix B) of a booking-limit policy. will still give us a good approximation to the optimal REMARK 6. Note that we were able to reformulate solution, while retaining the computational advan- the problem with the one-dimensional state variable tage of a one-dimensional state variable. Our nu- x even though the cancellation and no-show costs merical results (see Example 6 in the next section) are still allowed to be class dependent. This would tend to support this hope, while emphasizing the not have been possible had we continued to charge need for care in choosing the parameters in the cancellation and no-show costs in the periods in heuristic approximation. which they occur, rather than charging the expected cancellation and no-show cost during the period in 3. NUMERICAL EXAMPLES which the seat is booked. IN THIS SECTION, we report the results of numerical ϭ REMARK 7. In most YM applications, rˆ in ri. That solution of several examples. We divide the results is, the gross fare for class i is independent of n. into four subsections, each highlighting a different Assume (without loss of generality) that the classes aspect of the work in this paper. Section 3.1 provides are ordered so that counterexamples to commonly held notions in the YM literature, such as the monotonicity of the book- r1 у r2 у ···у rm . ing limits and bid prices in the number of booking Without cancellations, it is well known that the op- periods remaining. In Section 3.2, we provide an timal booking limits are nested in the same order as example of our algorithm performed on actual air- the classes, line data. We demonstrate that the MDP method b у b у ···у b , can handle real-world-size problems. Section 3.3 in- 1n 2n mn cludes a model of modest size in which the cancella- and this nesting is the same for all stages n.Aswe tion rates are class-dependent. This demonstrates saw in Section 1, however, when the fares rin are the computational feasibility of the multidimen- time dependent, the ordering of the booking limits sional MDP model of Section 2 for small problems. may be different for different stages. In particular, Finally, in Section 3.4, we demonstrate the power of ϭ this may happen in the present case, in which rin modeling cancellations through an example that Ϫ ri Gn(i). Consider two fare classes, i and j, with compares various approaches to solving YM prob- Ͼ Ͻ Ϫ Ͻ Ϫ ri rj (so that i j). If ri Gn(i) Un(x) lems in the presence of class-dependent cancellation

Copyright © 1999. All rights reserved. OVERBOOKING, CANCELLATIONS, AND NO-SHOWS / 157 and no-show rates. We show that significant in- TABLE I creases in revenue can result from fully taking into Parameters for Example 1 account the effects of cancellations and no-shows. Period n For all the examples, we work with the special Parameters 10–54–1 ϭ Ϫ case in which rin ri Gn(i), where ri is the fare p1n 0.0714 0.0 in class i (invariant with n) and Gn(i) is the ex- p2n 0.0 1.0 pected loss of revenue resulting from cancellation or qn 0.0714 0.0 no-show (defined recursively by Eqs. 22 and 23)—in other words, the version that results from applying the equivalent-charging transformation to Model 2 Ϫ2) for the plots corresponding to x ϭ 7 and x ϭ 12, under Assumption 1Ј. For computational purposes, and that the former plot is always above the latter. we introduce an overbooking pad v. When cancella- The capacity remaining is also identical (s ϭ 2) for tion and no-show rates are independent of the class the plots corresponding to x ϭ 3 and x ϭ 8, and the (Assumption 1Љ), the result is a version of Model 1 former plot is always above the latter. This indicates (as we saw in Section 2). The recursive optimality two things: (1) when we have the possibility of can- equations in this case take the form cellations, the functions un and the optimal policy m depend on the total capacity, C, and the capacity ͑ ͒ ϭ ͸ ͕ Ϫ ͑ ͒ Ϫ ͑ ͒ ͖ remaining, s ϭ C Ϫ x; and (2) for a given s, the u Un x pinmax ri Gn i unϪ1 x ,0 n iϭ1 functions are monotone (nonincreasing) in C.Itis clear from Figure 2 that the booking limits for class ϩ ͑ Ϫ ͒ ϩ ͑ Ϫ ͒ ͑ ͒ xqnUnϪ1 x 1 1 xqn UnϪ1 x , 2 are not monotonic in n. р р ϩ Ϫ у 0 x C v 1, n 1, EXAMPLE 2. This example is taken from Lee and ͑ ϩ ͒ ϭ ͑ ϩ ͒ ͑ ϩ Ϫ ͒ Hersh (1993). We consider four booking classes with Un C v C v qnUnϪ1 C v 1 ϭ ϭ ϭ ϭ fares r1 200, r2 150, r3 120, and r4 80. ϭ ϩ ͑1 Ϫ ͑C ϩ v͒qn͒UnϪ1͑C ϩ v͒, The capacity of the airplane is C 10. There are no cancellations, no-shows, and no overbooking. The у ϭ n 1, values of the parameters pin, i 1, 2, are shown in Table II. (Because Lee and Hersh do not allow can- ͑ ͒ ϭ ͓Ϫ␲͑ ͑ ͔͒͒ р р ϩ U0 x E Y x ,0 x C v. ϭ cellations, qn 0, for all n.) Figure 3 plots un(x) (cf. Remark 4). These are the equations that we use versus n for different x values. In this example, the for the numerical solution of the one-dimensional un functions are monotone in n because of the ab- examples (Sections 3.1 and 3.2). For the multidi- sence of cancellations. mensional examples (Sections 3.3 and 3.4) we use the corresponding special case of Eqs. 24 and 25, with the addition of an overbooking pad v.

3.1 Counterintuitive Results

EXAMPLE 1. Example 1 shows that the functions un(x) behave counterintuitively in that they are not always monotone in n. We consider two booking ϭ ϭ classes with fares r1 100 and r2 10. The refund ϭ ϭ amounts are cin ri, i 1, 2. The maximum overbooking level is v ϭ 3. The overbooking cost is given as 55 per seat overbooked. There are no no- ϭ shows. The values of the parameters pin, i 1, 2, and qn are shown in Table I. Figure 1 plots the bid prices, un(x) with respect to n for two different capacities, C ϭ 5 and C ϭ 10, and various fixed values of x. Note first that the functions un(x) are not monotonically increasing in n, in contrast to the examples in Lee and Hersh (1993). Second, if we observe carefully, we note that ϭ Ϫ ϭ the capacity remaining, s C x, is identical (s Fig. 1. un(x) versus n for different x values for Example 1.

TABLE III Parameters for Example 3

Period n Parameters 30–26 25–19 18–12 11–54–1

p1n 0.182 0.086 0.143 0.241 0.25 p2n 0.182 0.086 0.143 0.241 0.25 p3n 0.318 0.2 0.143 0.0 0.0 p4n 0.318 0.2 0.143 0.0 0.0 qn 0.0 0.014 0.014 0.017 0.0167

tion of the overbooking level. The maximum overbooking level is v ϭ 10. Cancellations take place at

rate qn. On cancellation, each fare class is refunded ϭ ϭ ϭ a different amount cin, c1n 200, c2n 120, c3n ϭ 60, and c4n 0. Class 4 is non-refundable. There are no no-shows. The values of the parameters pin, ϭ i 1, 2, and qn are shown in Table III. Fig. 2. Partition of the state space by critical values for Exam- Figure 4 shows a plot of un(x) versus n for differ- ple 1. ent x values, whereas Figure 5 plots un(x) versus x for different values of n. Note that un(x) is nondecreasing in x. Figure 6 partitions the state space into EXAMPLE 3. We consider four booking classes with ϭ ϭ ϭ ϭ the various acceptance regions. fares r1 200, r2 150, r3 120 and r4 80. ϭ The capacity of the airplane is C 20. The over- 3.2 Real Airline Data booking cost is a piecewise linear and concave func- EXAMPLE 4. (This example was previously reported in SUBRAMANIAN,CAMPBELL, and PHILLIPS (1996).) TABLE II We obtained airline demand and capacity data from Parameters for Example 2 a major U.S. airline. To protect the interests of this Period n airline and to guard against release of sensitive Parameters 30–26 25–19 18–12 11–54–1 data, we have held back (or modified) certain details.

p1n 0.08 0.06 0.10 0.14 0.15 The following example uses this data to construct a p2n 0.08 0.06 0.10 0.14 0.15 realistic airline YM problem for a single leg with an p 0.14 0.14 0.10 0.16 0.00 3n airplane of capacity C ϭ 100, six fare classes, and a p 0.14 0.14 0.10 0.16 0.00 4n 331-day booking horizon.

Fig. 3. un(x) versus n for different x values for Example 2. Fig. 4. un(x) versus n for different x values for Example 3.

Fig. 7. Booking limits versus time for Example 4: real airline data.

point of sale, and passenger type (i.e., individual or group). For a given date, we selected only the local traffic for individual bookings, and restricted point of sale to the United States. We calculated the ar-

Fig. 5. un(x) versus x for different n values for Example 3. rival and cancellation parameters as follows.

1. Each snapshot was divided into subperiods. 3.2.1 Calculation of Parameters 2. The booking probability for fare class i in subpe- We received historical airline demand data for all riod n (in snapshot k) was calculated as traffic on one flight leg for several departure dates (including both local and multileg traffic). The de- number of bkgs in fare class i mand data are given by booking-class, snapshot, and kth snapshot pin ϭ . Nk

3. The cancellation rate in subperiod n was calculated as

total cancellations in kth snapshot qn ϭ . Nk ͑load booked at the beginning of kth snapshot͒

The number of subperiods Nk was calculated so that

m ͸ ϩ ͑ ϩ ͒ Ͻ pin C v qn 1. iϭ1

3.2.2 Results Figure 7 shows the plot of the booking limits versus the time for each of the booking classes. To understand clearly the dynamics of the bid price with respect to time, it will be helpful to refer to Table IV, which shows a mapping of the booking periods to weeks-before-departure for this example. Note that the booking periods grow shorter as departure approaches. This is natural because the event density (expected number of events per unit time) typically increases as takeoff nears. In Fig. 7, the six booking classes are lettered in decreasing Fig. 6. Partition of the state space by critical values for Exam- order of fare, with class A representing the highest ple 3. fare class and class F the lowest.

TABLE IV Mapping of Periods to Weeks before Departure

Snapshot 1 2 34567891011121314151617181920 wbd1234567891113151719212325333947 period 35 54 94 125 173 243 288 306 339 370 462 465 467 470 489 490 491 494 495 496

wbd, weeks before departure.

3.2.3 Computation-Time Analysis and class 3 a discount fare with both purchase and Solving a single-leg problem with 100 seats and cancellation restrictions. 500 periods on a PC with a 100-MHz Intel 486 pro- Finding the optimal booking policy for this exam- cessor takes approximately ten seconds. This time ple took approximately 30 seconds of CPU time on a increases roughly as the product of the number of machine with a 200-MHz Pentium processor run- seats and number of periods in the model. We are ning the Linux operating system. The details of the currently investigating performance on a high-end optimal value functions, bid prices, and optimal UNIX server and are pursuing other strategies to booking limits are available from the authors upon improve run time. The desired target is to solve a request. We have omitted them here because of the single-leg problem of this magnitude in one second. difficulty in displaying the output from a multidi- This would translate to recalculation of 500 legs for mensional model in a compact yet meaningful way. 100 departure dates on a 6-processor UNIX server in (In Example 6 below, we will examine the optimal roughly one hour. value functions and optimal booking limits for a smaller multidimensional example.) Here we do 3.3 Class-Dependent Cancellations show, however, that class-dependent cancellation rates can cause the nesting of the net fare to be time EXAMPLE 5. We now solve an example in which the dependent (cf. Remarks 3 and 7 above). Table VI cancellation rates are class dependent. Although the shows the net fare of the three fare classes for the size of the model is modest, this does not entirely final periods in the booking horizon. Note that the diminish the usefulness of the results. In larger nesting order of classes 1 and 2 reverses toward the problems, it is typically the end effects—when both end of the horizon. the time remaining until departure and the remaining capacity are small—that are most crucial to the 3.4 Power of Cancellations performance of a booking policy. ϭ EXAMPLE 6. The cornerstone of the approach dem- We consider three booking classes, with fares r1 ϭ ϭ onstrated in this paper is the inclusion of customer 5, r2 3, and r3 1. The airplane capacity is 30, with an overbooking pad of 2. The booking horizon is cancellations. To highlight the effects of allowing ϭ ϭ cancellations, we consider a small example with 40 periods. Class 1 is fully refundable, c1n d1 5, whereas class 2 receives only a partial refund, c ϭ class-dependent cancellation and no-show rates. 2n ϭ d ϭ 1, and class 3 is nonrefundable, c ϭ d ϭ 0. Available capacity is C 4 with an overbooking pad 2 3 3 ϭ ϭ ϭ The penalties for overbooking by one and two seats, of v 2. There are 2 classes, with r1 3 and r2 ϭ ϭ respectively, are 4 and 10. The values of the param- 1. Class 1 is fully refundable, c1n d1 3, whereas ϭ ϭ ϭ class 2 is nonrefundable, c2n d2 0. Penalties of eters pin and qin, i 1, 2, 3 are given in Table V. In this example, class 1 is intended to be a fully- 2 and 6 correspond to overbooking levels of 1 and 2, refundable full-fare ticket, class2amildly dis- respectively. The remaining parameters are sum- counted ticket with some cancellation restrictions, marized in Table VII. We compare four methods for solving this example.

TABLE V 1. Completely ignore cancellations. Parameters for Example 5

Period n TABLE VI Parameters 40–21 20–11 10–65–10 Net Fare versus n for Example 5

p1n 0.05 0.1 0.2 0.3 0.0 Period n p 0.2 0.2 0.2 0.2 0.0 2n Class 36 37 38 39 40 p3n 0.2 0.3 0.0 0.0 0.0 q1n 0.013 0.013 0.015 0.015 0.03 1 2.967000 2.929000 2.891000 2.853000 2.816000 q2n 0.05 0.05 0.0 0.0 0.0 2 2.878000 2.873000 2.869000 2.865000 2.860000 q3n 0.0 0.0 0.0 0.0 0.0 3 1.0 1.0 1.0 1.0 1.0

TABLE VII TABLE VIII Parameters for Example 6 Comparison of Class-Independent Cancellation Rates for Example 6 Period n Parameters 16–13 12–98–54–10 Period n Method 16–98–54–10 p1n 0.0 0.1 0.3 0.4 0.0 p2n 0.3 0.5 0.0 0.0 0.0 3a 0.0 0.025 0.05 0.1 q1n 0.0 0.0 0.05 0.1 0.2 3b 0.0 0.05 0.1 0.2 q2n 0.0 0.0 0.0 0.0 0.0 3c 0.0 0.02 0.04 0.08

2. Adjust fares by expected lost revenue due to can- We compare the four methods by the following cellations, but ignore effects of cancellations on procedure. For Methods 1–3, we solve the simplified, future seat availability. one-dimensional problem optimally. The optimal 3. Approximate the class-dependent cancellation booking limits from each of these methods are then and no-show rates with a single class-indepen- evaluated in the context of the actual (multidimen- dent rate. sional) problem, and the results compared to those of 4. Solve the problem as stated. the optimal solution as obtained through Method 4. The expected revenues obtained by each of the Method 1 is the easiest method and reflects the methods are summarized in Table IX, where % Sac- general approach in the YM literature. The effects of rificed is the additional revenue that could be gained cancellations and overbooking on both the net fare by solving the problem optimally (Method 4), ex- received and on the number of passengers booked pressed as a percentage of the revenue from the are ignored. [Following the previous literature, how- given method. Note that Method 2 actually exhibits ever, we do add a pad to the physical capacity and worse performance than does Method 1, even though then allocate this limit—without cancellations or it appears to be a more accurate model in that can- overbooking—as if it were the true capacity. In ef- cellation and no-show refunds are subtracted from fect, this approach decomposes the problem, first the gross fare. Evidently, adjusting the fare in this setting an overbooking limit (the pad), perhaps way without also taking into account the effects of based on a simplified cancellation and overbooking cancellations on future seat availabilities sends the model, then applying a seat allocation model that wrong signal to the algorithm for this example. To assumes no cancellations or overbooking.] Method 2 understand why this might be the case, consider a is a first step toward incorporating cancellations class i Ͼ 1 with a large cancellation/no-show refund explicitly. Fares are adjusted by subtracting the and a high cancellation rate. After subtracting the (readily calculated) expected lost revenue. The ef- expected cancellation and no-show refunds from the fects of cancellations on the state variable are still fare, the net fare for class i may be so low that it omitted from the MDP optimality equations. Method ends up being rejected to save seats for later arriv- 3 incorporates both the reduction in net fare and the ing higher-fare requests. But, because Model 2 does fluctuations of total passengers booked produced by not take into account the fact that a class i customer, cancellations, but does so by introducing a single if accepted, has a high probability of canceling later cancellation and no-show rate that is assumed to (thus making the seat available anyway), it tends to hold across all classes. Note that Methods 1–3 use penalize this class more than it should. one-dimensional MDP models (Model 1). Method 4 This result indicates the importance of accurately solves the problem optimally as stated, using the modeling both the internal and the external effects multidimensional MDP model (Model 2). of cancellations, as we do in Models 3c and 4, for When using Method 3, for the purposes of comparison, we solved the problem with three different rates, summarized in Table VIII. For Method 3a, we TABLE IX averaged the cancellation and no-show rates of the Summary of Methods 1–4 two classes. In Method 3b, we simply used the rates Method U (0, 0) % Sacrificed corresponding to class 1. We chose the class-inde- 16 1 5.86 9.39 pendent rates in Method 3c in such a way as to 2 5.74 11.67 approximate as closely as possible the solution ob- 3a 6.22 3.05 tained in Method 4. In this particular example, that 3b 5.05 26.93 consisted of using 40% of the class-1 cancellation 3c 6.38 0.47 4 6.41 and no-show rates.

Fig. 9. Class-2 booking limits for the different methods for Example 6.

It is possible that we might also accept requests when there are more than two customers, but only if they are distributed in a certain way. Table X dis- Fig. 8. Un(0) versus n for the different methods for Example 6. plays the optimal booking limits for class 2 customers. Note the slight deviations from Figure 9. example. In particular, the YM practitioner should be wary of simplistic approaches, such as simply 4. CONCLUSIONS AND SUGGESTIONS FOR FUTURE WORK calculating the net fare by subtracting the expected cancellation and no-show refund from the gross fare WE HAVE CONSIDERED a single-leg airline YM prob- and then using one of the standard YM approaches, lem with multiple fare classes and time-dependent such as EMSR or Lee and Hersh’s model. Such an arrival probabilities. Currently booked customers approach, as we have seen in this example, could may cancel at any time or become no-shows at the actually lead to lower revenue. time of departure, with probabilities and refunds

Figure 8 compares the values of Un(0) for each of that, in general, may be both class and time depen- the four methods. Note how important it is to choose dent. Overbooking is permitted. the single-leg cancellation rate properly. (Periods 0–5 We have formulated the problem as a discrete- are omitted from the graph because all 4 methods time MDP and used dynamic programming (back- produce identical values of Un(0, 0) during those peri- ward induction) to analyze it. The analysis has two ods.) components: (1) characterization of the structure of Figure 9 plots the optimal booking limits for class an optimal policy; and (2) numerical calculation of 2 for Methods 1–4. In this figure, we use booking the parameters of an optimal policy. Our analysis limit to mean the maximum number of reservations exploits the equivalence of the YM problem to the to accept in a given period. Thus, a booking limit of well-studied problem of optimal control of admission 4 implies that a passenger would be accepted pro- to a queueing system. In its most general form (Mod- vided 3 or fewer passengers had been accepted pre- el 2), the MDP has a multidimensional state space, viously. Because Method 4 uses class-dependent because it is necessary to keep track of the number cancellation and no-show rates, the optimal booking of seats booked in each fare class. When cancellation limits depend on the number of passengers booked and no-show probabilities are independent of the fare in each class. In Figure 9, we have rounded down the class, we have used a transformation (the equivalent- policy for Method 4. For example, if we put down the charging scheme) to convert the problem to an equiv- booking limit as three, that means that we always alent problem (Model 1) with a one-dimensional state accept whenever there are two or fewer customers in variable: the total number of seats booked. In this case, the system, regardless of how they are distributed. we have used well-known results from queueing-con-

TABLE X Booking Limits for Class 2 from Method 4

Period n

(x1, x2) 12345678910111213141516 (0,0)111111111 1111111 (1,0)111111111 1111111 (0,1)111111111 1110000 (2,0)111111111 1110000 (1,1)111111111 0000000 (0,2)111111000 0000000 (3,0)111111111 0000000 (2,1)111110000 0000000 (1,2)111000000 0000000 (0,3)110000000 0000000 (4,0)100000000 0000000 (3,1)000000000 0000000 (2,2)000000000 0000000 (1,3)000000000 0000000 (0,4)000000000 0000000 (5,0)000000000 0000000 (4,1)000000000 0000000 (3,2)000000000 0000000 (2,3)000000000 0000000 (1,4)000000000 0000000 (0,5)000000000 0000000 (6,0)000000000 0000000 (5,1)000000000 0000000 (4,2)000000000 0000000 (3,3)000000000 0000000 (2,4)000000000 0000000 (1,5)000000000 0000000 (0,6)000000000 0000000

1, accept; 0, reject. trol theory to show that an optimal policy is monotonic maining horizon length and the remaining capacity in the state variable and thus is characterized by book- become small. Our experience and that of other re- ing levels for each fare class. These booking levels are searchers suggest that this is where the action is, in determined by the optimal bid prices, which are easily the sense that revenues are most sensitive to model characterized in terms of the dynamic-programming accuracy in this region. optimal value functions. Finally, we have considered a small problem with In our numerical examples, we have demon- class-dependent cancellation and no-show rates and strated that an optimal booking policy can have compared the performance of four methods. In order counterintuitive properties when cancellations and of increasing accuracy (and complexity), these are: no-shows are included. For example, booking limits (1) Model 1, ignoring cancellations and no-shows need not be monotonic in the number of booking (essentially the model of Lee and Hersh, 1993); (2) periods remaining. Also, the opportunity cost of a Model 1, using equivalent charging—subtracting booking now may depend on both the remaining the expected cancellation and no-show refunds from capacity and the number of seats already booked, the gross fare—but ignoring the (probabilistic) ef- rather than just the remaining capacity. Using data fects of cancellations on future seat availabilities; (3) from a real airline application, we have shown that Model 1, incorporating both cancellation and no- our one-dimensional model is computationally feasi- show refunds and probabilities, but using approxi- ble for realistic-size problems, in terms of both the mate probabilities independent of class; (4) Model 2, availability of data and the running times of the with both refunds and probabilities dependent on algorithm. We have also demonstrated computa- class (as given). For this example, revenue actually tional feasibility for the multidimensional model on decreases when going from Method 1 to Method 2. modest-size problems, which suggests that it might An increase of nearly 9% can be achieved by incor- be used to explore the end effects as both the re- porating both refunds and cancellation and no-show

probabilities, with careful use of the one-dimen- varying lengths, each small enough so that the prob- sional heuristic approximation (Method 3). The rev- ability of more than one event (arrival or cancella- enue increment in going from Method 3 to Method 4 tion) in a subinterval is negligible. is less than 1%, which suggests that the full multidimensional model may not always be needed. Ad- Approximation using Uniformization ditional numerical work is needed to see if these In the uniformization approach the continuous- qualitative conclusions are valid over a wider range time system is observed at discrete, randomly of parameter values. spaced points in time, each point corresponding to In principle, the methodology of this paper can be the arrival of a seat request, a cancellation, or a null applied to the multi-leg YM problem. The hurdle is event. Null events, which do not change the state of the combinatorial explosion of the state space in the the system, are introduced from a Poisson process number of legs. We are currently working on solving with a state- and time-dependent rate in such a way the two-leg problem exactly and efficiently. For the that the total rate at which the observation points general multi-leg problem, we hope to develop effi- occur is independent of the state. In addition, this cient heuristics to approximate the optimal policy. total event rate and the total number of periods N are chosen large enough so that the expected horizon length is close to the actual horizon length T (by the APPENDIX A: APPLICATION TO SYSTEM WITH law of large numbers). TIME-DEPENDENT POISSON ARRIVAL PROCESS To construct the time-dependent uniformization IN THIS APPENDIX, we show how to apply the MDP and the equivalent discrete-time MDP, we must ϭ model of Section 1 to a continuous-time seat-alloca- specify the probabilities pin, i 1,...,m, and qn, tion problem. In the continuous-time problem, the for each decision period n and state x. Beginning ␭ ϭ ␭ ϭ ␭ decision horizon is the time interval [0, T]. Seat with period N, let iN : i(0) (i 1,...,m), N ϭ ϭ͚m ␭ ␮ ϭ ␮ ⌳ ୑ ␭ requests for fare class i (i 1, ..., m) arrive : iϭ1 iN, and N : (0). Choose N N, and according to a time-dependent Poisson process with set rate ␭ (t), 0 ୏ t ୏ T. So, the probability that a class i ␭ ␮ ϩ iN N i request occurs in the interval (t, t dt)is p ϭ , i ϭ 1,...,m, q ϭ . ␭ ϩ iN ⌳ N ⌳ i(t)dt o(dt), independent of the previous arrivals N N and everything else that occurred in [0, t). Cancel- For each period n, N Ͼ n ୑ 1, assume ⌳ , ⌳ Ϫ , lations occur in a memoryless, possibly time-depen- N N 1 ...,⌳ ϩ have been chosen and let dent, manner. Specifically, each customer in the sys- n 1 tem at time t has a probability ␮(t)dt ϩ o(dt)of 1 1 1 ϭ ϩ ϩ ϩ canceling in (t, t ϩ dt), independent of everything t ··· . ⌳N ⌳NϪ1 ⌳nϩ1 that occurred in [0, t). A customer whose request for ϭ ␭ ϭ ␭ ϭ ␭ ϭ a seat in fare class i is accepted pays the fare r Let cin : ci(t), in : i(t)(i 1,...,m), n : i ͚m ␭ ␮ ϭ ␮ ⌳ ୑ ␭ ϩ Ϫ (assumed independent of t, to keep the exposition iϭ1 in, and n : (t). Choose n n (N ␮ simple). A customer in class i who cancels at time t n) n, and set ୏ ୏ receives a refund ci(t), 0 t T. Each customer ⌳ who holds a seat reservation in class i just before ␥ ϭ n n ⌳ , ␤ n departure has a probability i of being a no-show, in which case the customer receives a refund di,asin ␭ ϭ in ϭ the discrete-time model. pin ⌳ , i 1,...,m, To approximate this model by a discrete-time n model, we divide the time interval [0, T] into N ␮ ϭ n periods, where N is a sufficiently large integer. qn . ⌳n There are two ways of doing this, both of which lead to an MDP model of the form presented in Section 2. Upon substitution of these expressions in Eqs. 4 and The first method is an extension to time-dependent 5, the optimality equations for this problem take the processes of the approach introduced into queueing form control by Lippman (1975) and commonly called uni- 1 formization. This approach results in periods of ran- ͑ ͒ ϭ ͓␭ ͓ ͑ ͔͒ ϩ ␮ ͑ Ϫ ͒ Un x ⌳ nE Vn x, Rn x nUnϪ1 x 1 dom length, with an exponential distribution that is n independent of the state but may be time dependent. ϩ ͑⌳n Ϫ ␭n Ϫ x␮n͒UnϪ1͑ x͒], The second approach is simply to divide the interval [0, T] into N deterministic subintervals, possibly of 0 р x р N Ϫ n, n у 1, (A1)

Vn͑ x, r͒ ϭ max͕r Ϫ ͑UnϪ1͑ x͒ Ϫ UnϪ1͑ x ϩ 1͒͒,0͖ optimal policy to follow at time t may be found by calculating the optimal policy for the discrete-

ϩU Ϫ ͑ x͒, parameter MDP with n periods remaining, where n 1 ⌳Ϫ1 ϩ⌳Ϫ1 ϩ ... ϩ⌳Ϫ1 ϭ N NϪ1 NϪnϩ1 t, when N and ⌳ ⌳ 0 р x р N Ϫ n, n у 1, (A2) N,..., 1, are sufficiently large. Although a theoretical proof that this approach provides approximately optimal policies for the con- U ͑ x͒ ϭ E͓Ϫ␲͑Y͑ x͔͒͒,0р x р N. (A3) 0 tinuous-time problem with time-dependent parameters does not seem to be available, we have proposed Comparing Eqs. A1 and A2 to the optimality Eqs. 2 this approach as an intuitively plausible and com- and 3 in Lippman and Stidham (1977), we see once putationally feasible heuristic. In any case, in the again that our model takes the same form as the scenario proposed by Lee and Hersh (1993), in which finite-horizon arrival-control model in Lippman and it is assumed that the time-dependent Poisson ar- Stidham (1977), with the arrival and service rates rival processes for each fare class have constant allowed to depend on the number of periods remain- arrival rates over fixed time intervals (called booking. Specifically, with n periods remaining and x ing periods), the validity of the approximation fol- customers in the system, the time Zn until the next lows by a straightforward extension of the argu- event occurs is exponentially distributed with mean ments in Lippman (1976). ⌳ rate n. The next event is an arrival (booking re- ␭ ⌳ Approximation using Decision Periods of quest) with probability n/ n, a departure (cancel- ␮ ⌳ Deterministic Length lation) with probability x n/ n, and a null event ⌳ Ϫ ␭ Ϫ ␮ ⌳ with probability ( n n x n)/ n. (The require- Here, we divide the horizon into N deterministic ⌳ ୑ ␭ ϩ Ϫ ␮ ୏ ment that n n (N n) n ensures that 0 intervals, which can, however, be of unequal length. ⌳ Ϫ ␭ Ϫ ␮ ⌳ ୏ ( n n x n)/ n 1. The length of these intervals is chosen to be small For the case of time-homogeneous parameters, enough so that the probability of more than one LIPPMAN (1976) has observed that the MDP that event (arrival/cancellation) occurring in an interval results from uniformization can be used to solve is small, and the probability of each type of event approximately a continuous-time Markovian deci- can be approximated by the mean rate for that event sion process with a finite horizon of fixed, determin- times the length of the interval. Lee and Hersh istic length T. More precisely, by choosing the uni- (1993) follow essentially this approach, although formization parameter ⌳ and the number of periods, they do not allow cancellations and no-shows. ⌳ϭ ⌳ 3 ϱ 3 ϱ ⌬ ϭ N, so that N/ T and then letting , N , Suppose the length of the nth period is n, n one can show that optimal policies for the discrete- N, . . . , 1, where, as usual, we number the periods parameter MDP converge to optimal policies for the in reverse chronological order. Beginning with pe- original continuous-time MDP. The condition that ␭ ϭ ␭ ϭ ␭ ϭ riod N, let iN : i(0) (i 1,..., m), n : N/⌳ϭT ensures that the discrete-parameter MDP ͚m ␭ ␮ ϭ ␮ iϭ1 iN, N : (0), and set has an expected horizon length T; moreover, as N and ⌳ approach ϱ, the actual horizon length con- piN ϭ ␭iN⌬N , i ϭ 1,...,m, qN ϭ ␮N⌬N . verges to T, by the law of large numbers. In addi- Ͼ ୑ ϭ⌬ ϩ⌬ tion, for any t,0୏ t ୏ T, an approximately optimal For each period n, N n 1, let t N NϪ1 ϩ ... ϩ⌬ ␭ ϭ ␭ ϭ policy to follow at time t may be found by calculating nϩ1. Let in : i(t)(i 1,..., m), ␭ ϭ͚m ␭ ␮ ϭ ␮ the optimal policy for the discrete-parameter MDP n : iϭ1 in, n : (t), and set with n periods remaining, where (N Ϫ n)/⌳ϭt, ϭ ␭ ⌬ ϭ ϭ ␮ ⌬ when both N and ⌳ are sufficiently large. pin in n , i 1,...,m, qn n n . Our continuous-time seat-allocation problem dif- ⌬ For sufficiently small n, it follows from the proper- fers from that considered by Lippman (1976) in that ties of the exponential distribution that these ex- it has time-dependent parameters in addition to a pressions for the discount factor ␥ and the proba- finite horizon. As a result, the discrete-parameter n bilities, p and q , are accurate within an error that MDP that we have constructed involves an addi- in n is o(⌬ ). Upon substitution of these expressions in tional level of approximation. First, we approximate n ␭ ␮ Eqs. 4 and 5, the optimality equations for this prob- the continuously varying parameters, i(t) and (t), ␭ ␮ lem take the form by parameters in and n, that are constant over random (exponentially distributed) intervals. Then U ͑ x͒ ϭ ␭ ⌬ E͓V ͑ x, R ͔͒ ϩ x␮ ⌬ U Ϫ ͑ x Ϫ 1͒ we choose the number of periods, N, and the total- n n n n n n n n 1 ⌳ ϭ event rate, n, for each period n N,...,1so ϩ ͑1 Ϫ ͑␭n ϩ x␮n͒⌬n͒UnϪ1͑ x͒, large that the length of each such random interval approaches zero. We expect that an approximately 0 р x р N Ϫ n, n у 1, (A4)

Vn͑ x, r͒ ϭ max͕r Ϫ ͑UnϪ1͑ x͒ Ϫ UnϪ1͑ x ϩ 1͒͒,0͖ preted as the expected average cancellation rate per customer in period n. ϩ UnϪ1͑ x͒, METHOD 2. Define 0 р x р N Ϫ n, n у 1, (A5) m ͑ ͒ ϭ ͓Ϫ␲͑ ͑ ͔͒͒ р р U0 x E Y x ,0 x N. (A6) ͑ ͒ ϭ ͸ ͑ ͒ qn x : fin x qin , (A8) ϭ Note that these equations are in the same form as i 1 the optimality Eqs. A1, A2, and A3 for the uni- where fin(x) is the expected number of class-i reser- formization approximation, as can be seen by replac- vations given that the total number of reservations ⌬ ⌳ Ϫ1 ing n by ( n) in Eq. A4. is x. With the above setup, the optimality Eqs. 4, 5, and 6 APPENDIX B: HEURISTIC MODEL FOR CLASS- hold, and the problem is seen to be equivalent to DEPENDENT CANCELLATIONS AND NO-SHOWS control of admission to a queueing system. Note

RECALL THAT WE WERE able to transform our MDP that, in Method 2, the service rate qn(x) is a possibly model to an equivalent model with a one-dimen- nonlinear function of x, the number of customers sional state variable, provided that Assumption 1Љ present. As long as qn(x) is concave and nondecreas- holds. For convenience, we reproduce Assumption 1Љ ing in the state x, however, the inductive argument below. (Theorem 1 of Lippman and Stidham, 1977) contin- ues to be applicable and implies that the optimal Љ ϭ ϭ ,ASSUMPTION 1 . qin(x) xiqn, for all x, i 1,..., value function U ٪ is concave and nonincreasing Ͼ ϭ Ϫ n m, where qn 0, n N, N 1,..., 1. so that the optimal admission policy is monotonic in ϭ͚ ϭ This assumption may not be realistic in applica- the state. When qn(x) i finxqin qˆ nx (Method tions. As an extreme example, in the case of nonre- 1), then qn(x) is a linear function of x and, hence, fundable fares, the cancellation or no-show probabil- trivially concave. But when we use Method 2 to ϭ͚ ity is close to zero, whereas cancellation and no- estimate qn(x) i fin(x)qin, then we have to show rates tend to be positive and are often non- choose the functions fin(x) carefully so that qn(x)is negligible among passengers who pay full fare. concave. We plan to explore this approach in more Assumption 1Ј (reproduced below) is more realistic, depth in a future paper. inasmuch as it allows cancellation rates to be class dependent. ACKNOWLEDGMENTS Ј ϭ ASSUMPTION 1 . qin(x) xiqin, for all x, where THE AUTHORS WOULD LIKE to acknowledge the con- Ͼ ϭ ϭ Ϫ qin 0, i 1,..., m, n N, N 1,..., 1. tribution of Anatoly Shaykevich to an earlier version of this paper (Janakiram, Stidham, and Shaykevich, Assumption 1Ј may be more realistic, but then, we 1994). Anatoly’s economic insights inspired us to are faced with the problem of keeping track of an develop the equivalent charging scheme described in enlarged state x. To get around this problem, we Section 2. We also thank R.L. Phillips (President propose a heuristic approximation scheme, in which and CEO, DFI.Aeronomics) and G.C. Campbell (Vice we estimate the number of seats, x , booked in each i President, DFI.Aeronomics) for valuable assistance fare class i, based on the observed total number of in developing the ideas in Section 3.2. We are grate- seats x. From these estimates, we can compute ful to G. Colville (currently at Delta Airlines) for q (x), the estimated total cancellation rate, given n giving us permission to use real airline demand and that x seats are currently booked in period n.We capacity data in our analysis. Finally, we thank Jeff suggest below two intuitive ways to estimate q (x), n Rutter (DFI.Aeronomics) for help with transferring given x and q , i ϭ 1,..., m. in and manipulating large volumes of airline demand METHOD 1. Define data.

m ͑ ͒ ϭ ͸ REFERENCES qn x : finxqin , (A7) iϭ1 J. ALSTRUP,S.BOAS,O.MADSEN, AND R. VIDAL, “Booking Policy for Flights with Two Types of Passengers,” Eur. where fin is the expected fraction of class i custom- J. Oper. Res. 27, 274–288 (1986). ers with current reservations in period n.Ifwe P. P. BELOBABA, “Airline Yield Management: An Overview ϭ simplify the above equation, we obtain qn(x) x of Seat Inventory Control,” Transp. Sci. 21, 63–73 ͚ ϭ ϭ͚ i finqin xqˆ n, where qˆ n : i finqin can be inter- (1987).

P. P. BELOBABA, “Application of a Probabilistic Decision and Their Applications. Academic Press, San Diego, Model to Airline Seat Inventory Control,” Opns. Res. California, 1994. 37, 183–197 (1989). B. SMITH,J.LEIMKUHLER, AND R. DARROW, “Yield Manage- S. BRUMELLE AND J. MCGILL, “Airline Seat Allocation with ment at American Airlines,” Interfaces 22, 8–31 (1992). Multiple Nested Fare Classes,” Opns. Res. 41, 127–137 S. STIDHAM, “Socially and Individually Optimal Control of (1993). Arrivals to a GI/M/1 Queue,” Management Sci. 24, R. CHATWIN, Optimal Airline Overbooking. Ph.D. Disser- 1598–1610 (1978). tation, Department of Operations Research, Stanford S. STIDHAM, “Optimal Control of Admission, Routing, and University, Stanford, CA, 1992. Service in Queues and Networks of Queues: A Tutorial R. CURRY, “Optimal Seat Allocation with Fare Classes Review,” in Proceedings ARO Workshop: Analytic and Nested by Origins and Destinations,” Transp. Sci. 24, Computational Issues in Logistics R and D, George 193–204 (1990). Washington University, Washington, DC, 330–377, G. GALLEGO AND G. VAN RYZIN, “Optimal Dynamic Pricing 1984. of Inventories with Stochastic Demand over Finite Ho- S. STIDHAM, “Optimal Control of Admission to a Queuing rizons,” Management Sci. 40, 999–1020 (1994). System,” IEEE Trans. Automat. Control, AC-30, 705– G. GALLEGO AND G. VAN RYZIN, “A Multiproduct Dynamic 713 (1985). Pricing Problem and its Applications to Network Yield S. STIDHAM, “Scheduling, Routing, and Flow Control in Management,” Operations Research 45, 24–41 (1997). Stochastic Networks,” in Stochastic Differential Sys- W. E. HELM AND K.-H. WALDMANN, “Optimal Control of tems, Stochastic Control Theory and Applications, vol- Multiserver Queues in a Random Environment,” ume IMA-10, W. Fleming and P. L. Lions (eds) IMA J. Appl. Prob. 21, 602–615 (1984). Volumes in Mathematics and its Applications, Spring- S. JANAKIRAM,S.STIDHAM, AND A. SHAYKEVICH, Airline er-Verlag, New York, 529–561, 1988. Yield Management via Arrival Control to a Stochastic S. STIDHAM AND R. WEBER, “A Survey of Markov Decision Input-Output System. Paper presented at ORSA/TIMS Models for Control of Network of Queues,” Queueing National Meeting, Detroit, October, 1994. Syst. Theory Appl. 13, 291–314 (1993). S. G. JOHANSEN AND S. STIDHAM, “Control of Arrivals to a Stochastic Input-Output System,” Adv. Appl. Probab. J. SUBRAMANIAN,G.CAMPBELL, AND R. PHILLIPS, Research 12, 972–999 (1980). on Bid-Price Dynamics. Paper presented at INFORMS National Meeting, Atlanta, Georgia, 1996. C. LAUTENBACHER AND S. STIDHAM, “The Underlying Markov Decision Process in the Single-Leg Airline K. TALLURI AND G. VAN RYZIN, An Analysis of Bid-Price Yield-Management Problem,” Transp. Sci. 33, 136– Controls for Network Revenue Management, Technical 146 (1999). Report, Graduate School of Business, Columbia Uni- T. LEE AND M. HERSH, “A Model for Dynamic Airline Seat versity, 1996. Inventory Control with Multiple Seat Bookings,” L. WEATHERFORD AND S. BODILY, “A Taxonomy and Re- Transp. Sci. 27, 252–265 (1993). search Overview of Perishable-Asset Revenue Manage- S. A. LIPPMAN, “Applying a New Device in the Optimiza- ment,” Opns. Res. 40, 831–844 (1992). tion of Exponential Queuing Systems,” Opns. Res. 23, L. WEATHERFORD,S.BODILY, AND P. PFEIFER, “Modeling 687–710 (1975). the customer arrival process and comparing decision S. A. LIPPMAN, “Countable-State, Continuous-Time Dy- rules in perishable asset revenue management situa- namic Programming with Structure,” Opns. Res. 24, tions,” Transp. Sci. 27, 239–251 (1993). 477–489 (1976). E. WILLIAMSON, Airline Network Seat Control, Ph.D. Dis- S. A. LIPPMAN AND S. STIDHAM, “Individual versus Social sertation, Flight Transportation Laboratory, Massa- Optimization in Exponential Congestion Systems,” chusetts Institute of Technology, Cambridge, MA, Opns. Res. 25, 233–247 (1977). 1992. K. LITTLEWOOD, “Forecasting and Control of Passenger R. WOLLMER, “An Airline Seat Management Model for a Bookings,” AGIFORS Symp. Proc. 12, 95–117 (1972). Single-Leg Route when Lower Fare Classes Book L. ROBINSON, “Optimal and Approximate Control Policies First,” Opns. Res. 40, 26–37 (1992). for Airline Booking with Sequential Nonmonotonic Y. YOUNG AND R. VAN SLYKE, Stochastic Knapsack Models Fare Classes,” Opns. Res. 43, 252–263 (1995). of Yield Management, Technical Report 94-76, Poly- R. SERFOZO, “An Equivalence between Continuous and technic University, Brooklyn, NY, 1994. Discrete-Time Markov Decision Processes,” Opns. Res. 27, 616–620 (1979). (Received: August 1997; revisions received: April 1998, August M. SHAKED AND J. G. SHANTHIKUMAR, Stochastic Orders 1998; accepted: December 1998)