THE STABILITYOF A CAPACITATED,MULTI-ECHELON PRODUCTION-INVENTORYSYSTEM UNDER A BASE-STOCK POLICY
PAUL GLASSERMAN Columbia University, New York, New York SRIDHARTAYUR Camegie Mellon University, Pittsburgh, Pennsylvania (Received August 1992; revisions received February, May 1993; accepted June 1993)
Most models of multilevel production and distribution systems assume unlimited production capacity at each site. When capacity limits are introduced, an ineffective policy may lead to increasingly large order backlogs: The stability of the system becomes an issue. In this paper, we examine the stability of a multi-echelon system in which each node has limited production capacity and operates under a base-stock policy. We show that if the mean demand per period is smaller than the capacity at every node, then inventories and backlogs are stable, having a unique stationary distribution to which they converge from all initial states. Under i.i.d. demands we show that the system is a Harris ergodic Markov chain and is thus wide-sense regenerative. Under a slightly stronger condition, inventories return to their target levels infinitely often, with probability one. We discuss cost implications of these results, and give extensions to systems with random lead times and periodic demands.
M ost models of multi-echelon production and We show that our system is indeed stable under the distributionsystems assume unlimited produc- natural capacity condition, namely, that the mean tion capacity and unlimited order size at each site. demand per period be smaller than the per-period Under this assumption, various conditions on costs production capacity at every node. This condition is and model structure have been shown to imply the not itself surprising; the interest lies in determining optimality of certain policies. Rather less is known just what it implies. We show that for general station- about what happens when capacity limits are taken ary demands, this condition suffices to ensure that the into account. For capacitated systems, a more fun- system has a unique stationary distribution to which damental question than optimality of a policy is sta- it converges from all initial states. Under independent bility: Does a given policy allow the system to meet and identically distributeddemands, we show that the demands, or does the system become increasingly state of the system constitutes a Harims ergodic backlogged? Markov chain, and thus inherits the wide-sense re- In this paper, we analyze the stability of a multi- generative structure of that class of processes. Under echelon system in which each node follows a base- a slightly stronger condition, the system is regenera- stock policy, modified because of capacity con- tive in the classical sense and we identify explicit straints. Under a standard base-stock policy, the regenerationtimes. These properties have useful con- operation of each node is determinedby a target level sequences for simulation, and it was the simulation- of safety stock. As demands deplete inventories, each based optimization method of Glasserman and Tayur node produces goods to restore inventories to their (1992a) that motivated this investigation. We also ex- target levels. When production capacity is limited, it amine stability in the presence of lead times and de- may take several periods of production to offset de- mands influenced by a randomly fluctuating mand in a single period. Speaking loosely, the system environment, including the case of periodic demands. is stable if, on average, it can produce finished goods Our model is similar to those of Clark and Scarf at a greater rate than they are demanded. (1960), Federgruen and Zipkin (1984), and Rosling
Subject classifications: Inventory/production, multistage: Stability under base-stock policy. Probability, regenerative processes: Harris recurrent inventory processes. Area of review: STOCHASTIC PROCESSES AND THEIR APPLICATIONS.
Operations Research 0030-364X/94/4204-0913 $01.25 Vol. 42, No. 5, September-October 1994 913 ? 1994 Operations Research Society of America 914 / GLASSERMAN AND TAYUR (1989) in most respects, except for the capacity limits. regeneration times. Section 4 discusses cost implica- A related continuous-time model is that of Svoronos tions of our stability results. Section 5 covers systems and Zipkin (1991); other variants can be found in with fixed lead times and two models of variable lead Graves, Rinnooy Kan and Zipkin (1992). The litera- times, giving stability conditions in each case. In ture on capacitated systems is much more limited. Section 6 we generalize the demand process, allowing For a single-stage capacitated system, Federgruen demands to be influenced by a (possibly periodic) and Zipkin (1986a, b) show that a base-stock policy is random environment. optimal under rathergeneral cost assumptions. Tayur (1992) provides a method for computing the optimal 1. THE MODEL base-stock level. Our basic model is a serial system in which each Given the difficulty of finding optimal policies for stage has limited capacity and follows a base-stock general capacitated systems, it makes sense to restrict policy for echelon inventory, i.e., for cumulative inventory attention to a specific class of operating rules. Base- downstream from that stage. Where applicable, we stock policies are attractive because they are simple note extensions to an assembly system. In all cases, and are known to be optimal in certain settings. The inventories are reviewed periodically (i.e., the system stability results given here are part of the justification evolves in discrete time) and unfilled orders are back- for the gradient estimation method of Glassermanand logged. Demands are nonnegative but otherwise ini- Tayur (1992a), which can be used to find optimal tially arbitrary;we introduce restrictions as they are basestock levels. needed. A discussion of lead times is postponed to We know of no previous work on the stability of Section 5. multi-echelon systems; but the stability of single- stage systems has been studied extensively, often in 1.1. The Base-Stock Policy the setting of storage processes and dams. Prabhu There are m stages, indexed by i = 1, ... , m. Stage (1965) includes some results of this type. General 1 supplies external demands, i + 1 single-stage models are studied in Brockwell, Resnick stage supplies stage i for i = 1, . . m - and m draws raw and Tweedie (1982) in continuous time, and in Glynn ., 1, stage material from an unlimited source-an outside (1989) in discrete time. Many additional references sup- Within each events occur in follow- can be found in those papers. plier. period, the order: at i + 1 from Once our model is appropriately set up, existing ing First, production stage the previous period advances to stage i, i = 1, . . m - general tools can be used to prove stability results. In ., 1. Second, demands arrive at 1 are filled this regard, a key step in our analysis is representing stage and or backlogged according to the available inventory. the state of the system through echelon shortfalls. These are differences between (cumulative) base- Lastly, the production level for the current period is set. This is the sequence of events in Clark and Scarf. stock levels and (cumulative) inventoriqs. The short- Much of the falls satisfy a recursive equation that facilitates their subsequent literature assumes produc- tion levels are set before analysis. In particular, through this recursion we are demands are revealed. The Clark-Scarf able to apply the method of Loynes (1962) to find a sequence simplifies our analysis. To describe the of stationary distribution, as Baccelli, Massey and Tow- operation the system we use the notation: sley (1989) and Baccelli and Liu (1992) do for certain following queueing systems. The recursion is also useful in Dn = the demand in period n; establishing Harris ergodicity through a coupling ar- Si = the base-stock level for echelon i; gument, as in Thorisson (1983), Asmussen (1987), and = Sigman (1988). While these techniques are reasonably c= the production capacity at stage i. familiarin queueing theory, they seem to be less well At stage 1, established in the stochastic production-inventorylit- = the inventory-backlog in period n, erature. One purpose of this paper is to show how nP they can be used in this setting as well. and for i = 2, ..., The details of our model are presented in Section 1. = the installationinventory at i in Section 2 shows the existence of a stationary regime n stage period n. for the echelon shortfalls and convergence to station- Thus, In ? 0, i = 2, ... , m is the inventory available arity from all initial conditions. In Section 3, after a for production at stage i - 1, and In is stock for brief review of Harris chains, we show that our sys- external demands when it is positive and the size of tem is Harris ergodic, then give conditions for explicit the backlog when it is negative. GLASSERMAN AND TAYUR / 915 Under a (modified)basestock policy, stage i sets its production level in each period to try to restore the echelon inventory position So, we may analyze the stability of Yn - (Yn, Ynn)and then interpret the results for inventories. In general, the shortfall at stage i satisfies (A /)-Dn Yn'l = Yn + Dn - Rn- to level s'. Without capacity constraints, this would A base-stock policy attempts to reduce the shortfallto be achieved by setting production equal to the smaller zero, while never driving it below zero. However, as of Dn and the available inventory. Since, however, in (1), production at stage i is constrained by the production cannot exceed c', it may take multi- capacity c' and, for i < m, by the available inventory. ple periods of production to offset demand in a single Since stage m draws raw material from an infinite period, even if ample inventory is available for source, we have production. To make this more explicit, we let Yn+= Yn' + Dn - min{Yn + Dn, cm} Rn the production at stage i in period n. -max{O, Yn + Dn - cm}. (6) Then the base-stock policy sets For i = 1, ... , m - 1, the available inventory is limited to Ii % so we have Rn min{s' + D, -n(I + n+Ii) In+1 c'} Yi +l = Ye + Dn - min{Y, + Dn, cI }. i=1,...,m-1 (1) Using (5) and simplifying, we get and yi {0 i i yi+1D Y'1= max{,fo Yn + Dn C , Y+ln + Dn RI = min{sm + Dn - + + i1m), cm}. (2) (In+ -(S-+1 )}. (7) The first term inside the minimum in (1) is the differ- ence between the target cumulative inventory s' for Equations 6 and 7 are the key to our analysis. The first of these is a Lindley equation, and this will be impor- stages 1 throughi and the actual inventoryI, + ** + tant in what follows. with Ii and the In - D,n; stage i attempts to drive this difference to Compared Rn, zero. The next two terms inside the minimum reflect shortfalls lend themselves more easily to an analysis the supply and capacity constraints, respectively. of stability. Summarizingdevelopments thus far, we Since stage m draws raw material from an infinite have: source, the supply constraint is absent in (2). The Lemma 1. The echelon shortfalls satisfy Yn+j = evolution of the system is completely specified by (1), 4(Yn, Dn) where 0: R? x R +-> R+ is defined by (2) and the following rules for the inventories: (6)-(7). In particular, 4 is increasing and continuous. = I' + Rn _ In+1 Dn;. Similar recursions hold in an assembly system, as I'+, = I' + RIn- Rnl i = 2, . M..m we now explain. In an assembly system, each node i has a set r(i) of predecessor nodes with indices These reflect the downstream flow of material. greater than i. If i is a root, then 7r(i) is empty and 1.2. Echelon Shortfalls node i draws raw material from an infinite source. Otherwise, node i combines materialfrom all nodes in Physical inventory levels are arguably the most nat- V-(i)in equal quantities. Thus, period-n production at ural descriptors of the state of the system. But, as is node i is limited by min{IfF j E v-(i)}. Proceeding as often the case in these types of systems, it turns out before, we obtain to be mathematically more convenient to work with echelon quantities. For i = 1, ... , m define the Y, +1 = max{O, Yn + Dn - i, period-n shortfall for echelon i by max{Yn + D, - (s' - si)}} (8) jEiw(i) Y'n = S'_2 In - (3) where the maximum over an empty set is taken to be j=1 zero. The shortfalls determine the inventories, because Remark. There is some similarity between the evo- Il= - (4y; lution of our serial system and that of queues in 916 / GLASSERMAN AND TAYUR tandem. In both cases, material passes through a Massey and Towsley of acyclic fork-joinqueues. For sequence of stages in series. However, the connec- the converse, use the fact that Y+1 > Y' + D, - cl tion does not go beyond that. Notice, in particular, to conclude that if E[DO] > ci, then YO must be that in (ordinary) tandem queues the service mecha- infinite. nism at each stage does not depend on the status of Part iii follows from part ii through a coupling ar- other stages, whereas in our system the target pro- gument. The process {Yn, n > O} is said to admit duction at each stage depends on the inventory at all coupling if for all pairs of initial states it is possible to downstream stages. Hence, there is no direct connec- construct two copies of the process started in those tion between (6)-(7) and recursions for quantities as- states in such a way that the two copies coincide after sociated with tandem queues. At the same time, a finite (random)time. A process that admits coupling techniques used to analyze queueing systems serve, can have, at most, one stationary distribution. To with modification, as the basis of our analysis. show that Y admits coupling, note that Ym does (again from Loynes) and argue from (7) that Yi couples a 2. THE STATIONARYREGIME finite, random time after (Yi+',... Ym) has coupled. Suppose, now, that the demands form a stationary process. In this setting, through the method of Remarks Loynes, the conclusion of Lemma 1 is sufficiently strong to imply the existence of a stationaryversion of i. The same argument works for the assembly sys- the echelon shortfalls. (In fact, it would suffice for 4 tem, proceeding by induction down the branches of to be increasing and continuous in its first argument the precedence tree. for all values of its second argument.) Moreover, the ii. Theorem 1 could alternatively be proved by ap- natural stability condition pealing to general results for (max, +) recursions in Baccelli and Liu (1992) or Glasserman and Yao E[Do] < min{c1 : i = 1, ... , m}, (9) (1992). The method of Baccelli and Liu associates ensures that there is just one finite stationary distri- a randomly weighted graph with recursions (like bution. Here and throughout, > denotes convergence (6) and (7)) involving only max and +. Glasserman in distribution. and Yao use a matrix formulation encompassing (max, +) and other types of recursions. In both > Theorem 1. Suppose that the demands {Dn, n O} approaches, the stability condition takes the form are stationary. y < 0, where the constant y depends on the par- i. There exists a (possibly infinite) stationary pro- ticular recursion and cannot be computed easily in general. It can be shown that for our model y = cess {Ynvn ? O}satisfying Yn,1 = k(Y, DJ) - for all n, such that if YO= 0 almost surely (a.s.) E[DO] minic', so Theorem 1 is consistent with then Yn => Y'P the general results. ii. Suppose the demands are ergodic as well as sta- tionary. If (9) holds, then YOis almost surely 3. REGENERATION finite; if, for some i, E[DO] > c', then YO = oo a.s., for allj = 1, ..., i. The previous section established conditions for the iii. For stationary, ergodic demands satisfying (9), stability of the echelon shortfall process when de- Y => Yofor all YO. mands are stationary and ergodic. We now examine the regenerative structure of {Yn, n > 0} when Outline of Proof. A detailed proof of each of the {Dn, n > O}is an i.i.d. sequence. (In Section 6 we assertions in the theorem is given in Glasserman and relax the i.i.d. assumption.) Regenerative properties Tayur (1992b). Here, we outline how a proof may be are valuable in establishing convergence of costs constructed by appealing to related results in the and also simulation estimators. Indeed, it was the literature. simulation-based application in Glasserman and Part i follows from Loynes via Lemma 1. In part ii, Tayur (1992a) that initially motivated this the assertion for ym follows from Loynes' analysis investigation. of the single-server queue. For Yi, i < m, proceed by We show that the stability condition of Section 2 induction on i from m down to 1 to show, using (7), suffices to ensure that {Yn, n > O} possesses the that (9) implies that the stationary distribution is fi- regenerative structure of a Harris ergodic Markov nite; this step is similar to the analysis in Baccelli, chain. Under a stronger condition, we show that the GLASSERMANAND TAYUR / 917 vector of shortfalls returns to the origin infinitely remains valid, as does the associated central limit often, with probability one. theorem (under second-moment assumptions). More- over, if X is Harris ergodic, then for all initial 3.1. Harris Recurrence conditions the distribution of X,, converges to 7- in Many of the attractive properties of classical regen- total vatiation; that is, erative processes have been shown to hold for the sug lPx (Xn E:A) - (A)) -0 somewhat weaker regenerative structure of Harris ACE recurrent Markov chains. We briefly review key def- as n -> oo for all x E S. Indeed, this total variation initions and results of this frameworkto apply them to convergence to a probability measure completely our model. More extensive coverage can be found in characterizes Harris ergodicity. Nummelin (1984) and Asmussen (1987); the treatment A powerful tool in the analysis of Harris ergodic in Sigman is particularlyrelevant to our application. Markov chains is a connection with coupling; see, for The general setting for Harris recurrence is a example, Thorisson (1983), Asmussen and Thorisson Markov chain X = {X, n > O} on a state-space S (1987), and Sigman (1988) for background. The main with Borel sets 9a. Let denote the law of X when P, result is this: A Markov chain with an invariantprob- XO = x. Then X is Harris recurrent if there exists a ability measure admits coupling if and only if it is o-finite measure (S, I), not identically zero, such qfron Harris ergodic. Since we already used a coupling that, for all A E , argument for Y in Section 2, it is now easy to prove this: +i(A) > O Px( I 1 {Xn CzA}= X n=O Theorem 2. Let demands {Dn, n > O} be i. i. d. with =1 forallxES. (10) E[Do] < minic'. Then {Yn, n > O}is a Harris ergodic Thus, every set of positive q+-measureis visited infi- Markov chain. nitely often from all initial states. Every Harris recur- Proof. Since Yn1 = O(Yn, D), n > 0, Y is a rent Markov chain has an invariantmeasure 7- that is Markov chain when D is i.i.d. We know from unique up to multiplicationby a constant. The sets of Theorem 1 that Y has an invariant (i.e., stationary) positive n-measure are precisely those that are visited distribution and that Y admits coupling. Thus, Y is infinitely often from all initial states. If ir is finite Harris ergodic. (hence a probability, without loss of generality), then X is calledpositive Harris recurrent. If, in addition,X As a result of Theorem 2, Y inherits the regenera- is aperiodic, then it is Harris ergodic. tive structure of Harris ergodic Markov chains and The connection with regeneration enters as fol- the attendant ratio formula and convergence results. lows. If X is Harris recurrent, then there exists a The same holds for the inventory levels: (discrete-time) renewal process {Tk, k > 1} and an integer r > 1 such that Corollary 1. Under the conditions of Theorem 2, the inventoryprocess {(I,.'.., Im), n > 0} is a Harris {(XTk +n, ion 0), (Tn+k+l - Tn+k, nfl O)} ergodic Markov chain. has the same distribution for all k ? 1 and is inde- = pendent of Proof. Equations (3)-(5) put Y, and I,, (In, ... Inm)in one-to-one correspondence for all n. Conse- < I = n ? 0} is {Tl, ... lo TRk (Xn, 0 : n k -r)}- quently, {I, Markov if Y is, and I is Harris ergodic if Y is. When r > 1, there may be dependence between consecutive cycles {X, Tk- 1 < n < Tk}, in contrast 3.2. Explicit Regeneration Times to the classical case of independent cycles (and this is While Harris recurrence ensures the existence of indeed the case in our model). However, if X is pos- (wide-sense) regeneration times {rk, k ? 1}, it does itive Harris recurrent and S -* R is iff: 7r-integrable, not provide a means of identifying these times. Ex- then the regenerative ratio formula plicit regeneration times are not needed for conver-
Tk-1 gence results, but they are useful in, for example, E [ f(Xn) computing confidence intervals for simulationestima-
= n= - I tors. We now a sufficient ? E,[f(Xo)] ETk (11) give condition for {Y,, n 0} to have readily identifiable regeneration times. 918 / GLASSERMAN AND TAYUR Theorem 3. Let demands be i.i.d. with E[Do] < 4. COST IMPLICATIONS minic . Define so 0 and suppose that The condition in Theorem 3 motivates an investiga- tion into what ranges of parameters can be optimal P(Do < s'- s'-1) > O i = 1, . . ., m. (12) when we impose costs. The stability results of the Then Y returns to the origin infinitely often, with previous two sections also make it possible to give a probability one. partial characterizationof infinite-horizoncosts, and this may be useful in optimization. i an inventory holding Proof. If E[DO] < ci, then P(DO < c1) > 0. Conse- To each echelon we assign quently, under the conditions of the theorem there cost h', i = 1, ... , m. Backorders at stage 1 are penalized at ratep. There is no fixed cost for produc- exists an E with E < minic1 and E < mini(s' - s such that 8 _ P(Do < e) > 0. Since Y has a finite tion in a period; if there were, a base-stock policy stationary distribution, there exists a constant b > 0 would be unattractive. Costs are incurred at the end such that the set Bb C Rmdefined by of each period, so the cost in period n is m _ .. = - Bb f(Yl ,YM) 0 1
Proof. In light of Corollary3, we may assume that the -Yn+l; shortfall process starts at the origin. We claim that if the first equality uses (7) with c'+1 replaced by c' - s St+ s Ci, i 7...Sm 1, then for all n we and the second equality substitutes (17) evaluated at have n into the first equality. It follows that the claim holds at n + 1 for Y1, ... Y'. A similar shows Yn ? Yn ? * Ynm (15) , argument that (17) is preserved at each transition. and It follows from the claim just proved that reducing ci+1 to c' does not decrease any shortfalls; hence, it Y'n+1 =max{ 0, Yn + Dn -ci} i = 1, . ,m. (1 6) does not increase any echelon inventory levels. More- By hypothesis, (15) holds at n = 0. From (7) we see over, since Y1 is unchanged, backorder penalties are > - that (15) implies (16) whenever c' s-+1 s', and, not increased; so, total costs are not increased. in turn, (16) implies that (15) holds at n + 1 if the ce's are increasing. 5. LEAD TIMES Under the assumption that Y0 = 0, (16) shows that the evolution of Y is independent of the base-stock We now examine variants of our basic model in which ' levels, so long as they satisfy s'+1 - si ci. How- it may take several periods for production at stage i to - ever, the echelon inventories s - Y", i = 1, ...,m become available inventory at stage i 1. We show increase with the base-stock levels. Thus, if we re- that for fixed lead times, our results continue to hold duce Si+ 1 to sl1 + Cl1 + . .. + ci, i = 1, . 1, essentially without modification. When each order we lower holding costs without increasing back- draws a random lead time and moves in parallel with orders, since Y' is unchanged. other orders, it suffices to add that the mean lead time be finite. When shipments between stages are FIFO Suppose now that the capacity levels are subject to (in a sense to be made precise), a stronger condition control, possibly within a range of values. For is needed for stability. 920 / GLASSERMAN AND TAYUR 5.1. Fixed Lead Times production advances by one lead time position each Suppose that production at stage i in period n be- period. The same holds for arbitrarye1. comes available at stage i - 1 in period n + f' + 1, We reduce the operation of this system to one with i = 2, . . . , m. At stage 1, El is the lead time from final additional stages but no lead times. This reduction production to external availability. Our earlier model rests on the following lemma, for a system without lead times. used E' = 0, i = 1, . . ., m. We now let the lead times be any fixed, nonnegative integers. Lemma2. Suppose that for some i > 1, we have ci = Once we introduce lead times, installation invento- ct1 ands' = s- 1. If IP 6 c', then R'- = Itfor all ries no longer give a complete description of the phys- n ? 1 and, consequently, R7-' = R1 for all n > 2. ical state of the system: We must record, as well, In other words, in each period stage i - 1 produces inventories in transit. As in Section 1, let R' denote exactly as much as stage i produced in the previous production at stage i in period n. The physical state is period. now (I, R-1 S R-e,)z=l. (18) Proof. Since stages i and i - 1 have the same base- stock level, the shortfall for echelon i - 1 equals the The variables R1 -i +j, j = 1, .. ., (', indicate how shortfall for echelon i plus the inventory between the much new inventory becomes available at stage i - 1 two stages; i.e., Y'-' = Y' + In for all n. In partic- in the next e1periods. ular, in period 1 the echelon-(i - 1) shortfallis at least Consider the system illustrated on the left side in I'; so, R 1 = I1, under our hypothesis that IP S Figure 1. There is a lead time of f3 = 2 between stages ci= c-1. Now suppose that R-1 = Ik for all k = i and i - 1, illustrated by the line segment between 1, ..., n - 1. Then the only inventory between the production facility (the circle) and the storage stages i and i - 1 at the start of period n is the facility (the square) for stage i. Think of this line previous period's production at stage i; that is, It = divided into = 2 A segment as being e1 positions. Rn1., which is no greater than c11 = ci. As noted, 1 quantity R -2 is placed in the first position dur- Yi It, so in period n, stage i - 1 produces as ing period n - 2; this is the stage-i production in that much as the supply of inventory allows; i.e., R'-7 = period. In period n - 1, the quantity R'n-2 advances It. The first assertion is thus proved by induction. one position and a quantityR 1 is placed in the first The second assertion follows: If stage i - 1 depletes position. In period n, the quantityRn-2 arrives at the its inventory in each period, then R7-'=1 - _R for all storage facility and increments It; the quantity R'1 n. advances to the second lead time position; and R' is With this result, we can mimic the operation of a placed in the first lead time position. Thus, stage-i system with fixed lead times using additional stages and no lead times. Introduce f' dummy stages be- tween (genuine) stages i and i - 1, each having capacity ci and base-stock level si. This augmented system operates as an ordinary serial system without lead times. (The right side of Figure 1 illustrates the insertion of dummy nodes.) From Lemma 2 we see that the effect of these dummy nodes is to advance production at stage i by one node each period. So, period-n production at stage i becomes available at stage i - 1 in period n - e1+ 1. This reproduces the effect of the lead time W. The assumption in Lemma 2 that the initial inventory IP does not exceed ci is not a restriction: In the lead time model, the quantity one position downstream from stage i is just the previous period's production at stage i and so cannot exceed c1. Figure 1. Replacing a lead time with dummy nodes. As before, our analysis simplifies if we work with Stock advances by one dummy node each shortfalls rather than inventories. For i = 1, ... , m period, thus mimicking the effect of a lead and j = 1, ... , f, denote by yi; the echelon shortfall time. corresponding to thejth dummy node upstream from GLASSERMAN AND TAYUR / 921 stage i - 1 and let yi,i +1 = Yi denote the echelon frameworkof subsection 5.1. Without this restriction, shortfall corresponding to stage i. Paralleling (3) we we need to introduce infinitely many dummy nodes have between each pair of stages. The dummy capacities i-l ek-1 and base-stock levels are as in subsection 5.1. Also, i ik as before, Y'J denotes the shortfall for Yz _ + E R k-fk-r echelon thejth dummy node upstream from stage i - 1, but now j has no upper bound. The stage-i shortfall is still Yi. The state of the shortfalls in period n is (Yb; Yn, j- 1, 2, ... ). Mimic the operation of the original . . . .~. . . system as follows. If, in the original system, a quan- and replacing (5) we have tity of production R1 at stage i draws lead time Li, then this quantity moves (at the end of the period) R__+ = Yn~J- YnJ+1 j = 1, ..., t directly to the finished inventory at the (Li + 1)-st In= Y 1 _ YR + (si-r dummy node upstream from node i - 1. The short- falls Y1J, = L1 + 1, L + 2, .. , all drop byR', and with the obvious modifications for i = 1. Since j = the system with dummy nodes has no lead times, the the shortfalls Y, j 1,..., L' remain unchanged. augmented set of shortfalls {(Ye,=1,..., i + 1, Subsequently, the quantity R1 advances by one dummy stage in each period, and so becomes avail- i= 1, ... ., in), n ? O}satisfies a recursion of exactly - the type studied in previous sections. The next result able for use by genuine stage i 1 exactly L' periods thus follows. after it is completed at stage i. Let us call an array {yi; yi', j = 1, 2,...; i = Theorem 4. If the lead times fo,i = 1, ., m, are 1, . .. , m} of shortfalls finite if all entries are finite fixed, nonnegative integers, then Theorems 1 and 2 and if, in addition, all but finitely many increments hold for the augmented shortfalls. yif _ yivj+1 are zero. The second condition means that there are only finitely many dummy stages with in- As an immediate consequence, we have: ventory. We now have: Corollar 4. If the demands {De,i n O} are i.i.d. Theorem 5. Suppose n > O}is stationary with E[Do] < minic, then the Markov chain {(Dn, Ln), and ergodic. Suppose that E[Do] < and {(In, Rn-1, ***, Rn fi)i1,n? O}is Harris ergodic. minic' E[L' ] < oo, i = 1, ..., m. Then the augmented If any of the lead times is strictly positive, then shorfallprocess {(Yn; Yn>,j = 1, 2, ..., i = 1, there are at least two stages in the augmented system m),n > O}has a uniquefinite stationary distribution with the same base-stock level: If, say, ei > 1, then to which it converges from all finite initial states. the f' dummy nodes between (genuine) stages i and i - 1 all use level si. Consequently, condition (12) in Proof. If Yn denotes the array of period-n shortfalls, Theorem 3 cannot be satisfied unless the demand then Yn+ 1 is completely determined by Yn, Dn, and distribution has mass at zero. In this setting, (12) is Ln. Moreover, the mapping from (Yn, Dn, Ln) to actually necessary for installation inventories to re- Yn+1is component-wise increasing and continuous in turn infinitely often to their full-inventory level Yn for all values of Dn and Ln It follows as in the 52 - (S', - .l, sm Sm-1). For if P(DO = 0) = system without lead times that there is a stationary 0 and e1 1, then for all n, either Yin > 0 or else there process {Y,n n B O}, satisfying the same recursion, is inventory in transit through stage i. such that if YO= 0, then Yn > Yo. We now argue that YOis finite, almost surely. 5.2. Parallel Lead Times The evolution of Ym is still governed by the Lindley We now consider a system with random lead times. equation, so, under our stability condition, YO is Production in period n at stage i becomes available finite, almost surely. We claim that for all j = 1, stock at stage i - 1 after Ln periods; the sequence of 2, .. ., Y'J is finite, almost surely. To prove this, we vectors {(L,1 ..., Lm),n > 0} is stationary. We refer construct an auxilliary G/G/oo queue, modeling the to this mechanism as parallel lead times because movement of inventory from stage m to stage m - 1. different shipments do not interfere with each other. The queue evolves in discrete time. There is an arrival Overtaking is possible. at time n precisely if there is production at stage m in If the lead times were bounded, then with minor period n; i.e., if Y' + Dn > 0. The service time of modification this model could be fit into the the customer arriving at time n is L'. The number 922 / GLAS$ERMAN AND TAYUR of customers in this auxilliary queue is the number of induction, that all components of Y0 are finite almost shipments in progress from stage m to stage m - 1, surely, and (by coupling) that this is the only finite and multiplying the number in system by cm gives an stationary distribution and that Y, converges to it upper bound on the total inventory in transit between from all finite YQ. these stages. We know that ym couples with its sta- > tionary version in finite time. Hence, the arrivalpro- When {(Dn, Ln), n 0} are i.i.d., {Yn, fln 0} is cess to our infinite-server queue couples with a a Markov chain. Coupling, together with the exis- tence of a stationary distribution, proves Harris er- stationary version in finite time. For a G/G/?osystem for n > with stationary arrivals and service times and integra- godicity. The state space {Yn, O}, R"xo is more complicated than those we considered earlier, ble service times, we know from Theorem 2.3.1 of but with the topology of component-wise conver- Franken et al. (1982) that the queue has a unique finite gence Rn" is metrizable as a complete, separable stationary distribution(and couples with its stationary metric space (Billingsley 1968, p. 218) and this suffices version in finite time). for general results on Harris chains. As in our previ- Returningto the shortfalls, the event {YOJ = oo}has ous models, regeneration of the shortfalls implies re- probabilityzero or one if demands are ergodic. Since generation of the inventory levels. Y: is finite, the only way to have Yo' infinite is to have infinite inventory in dummy stages j + 1, j + 5.3. FIFO Lead Times 2, .... However, this inventory is bounded by the auxilliary G/G/oo queue and cannot be infinite; so, In our final model of lead times, each shipment from stage i must wait until all previous shipments from i Yo' is finite almost surely, j = 1, 2, have been transportedbefore initiating its transition. We Inow argue that only finitely many YO - This models a system in which a single vehicle moves 0 ii, j 1, 2, ..., are nonzero. Each such in- crement is at least as great as production in stage m in stock between each pair of stages; the vehicle com- some previous period. When the stage-m shortfall is pletes a roundtrip for each period in which the up- stationary, so is the sequence of production levels at stream stage has production. For period-n production stage m. But then for the total dummy-nodeinventory at stage i, the roundtrip travel time is L'. For this system, much stronger and less easily verified condi- co tions are needed for stability. We use the notation of i E (POM' yRm'i+') subsection 5.2. j=1 to be finite, only finitely many terms can be nonzero. Theorem6. Under the conditions of Theorem5, there > This further implies that for any finite YO,(YWi, j - exists a stationary version {Yn, n O}of the short- 1, 2, ... ) couples with its stationary version in finite falls for which Yn 4' YOif YO 0. If, in addition, time: These shortfalls couple with those of an initially P(Yb + Do > 0, Yjo+1,1+ 5i+1 - 5i
6. RANDOM ENVIRONMENTSAND PERIODIC Yn'+ = max{&m- Yn, Yn + Dn - cm} (21) DEMANDS Yn+= max{1 - S, Yn + Dn - C IYi+l +Dn We now return to the basic model of Section 1 to - (&i+1- Si)} (22) consider systems with more general demand patterns and, correspondingly, more general production rules. We now give conditions for stability. Let {0,n, Our new assumption is that demands are influenced - oo < n < oo}be a stationary version of 0 and let E by an environment that is itself subject to random denote expectation with respect to this stationary fluctuations. Base-stock levels may be adjusted to version. changes in the environment. Theorem 7. Suppose that the environment {( n ? We model the environment as a Markov chain with n0, O} is a Haris ergodic Markov chain and that de- a general state space. This is no real restriction; mands and base-stock levels are governed by 0. Sup- rather, it means that the state of the environment is pose the base-stock levels are bounded above. If sufficiently rich to include all relevant information < ooand < then n ? about the past. We first require the environment to be E[So] E[Do] minic', {(Yn, 0,), O} is a Harris ergodic Markov chain. Harris ergodic, then allow it to be periodic, thus capturing, e.g., seasonal demand patterns. Proof. That {(Yn,, On), n ? O} is Markov follows Models of this type are not new to inventory the- from (21)-(22) and the fact that (Dn, Sn) g(0n), just ory. Iglehart and Karlin (1962) find optimal policies as in Lemma 3.1 of Sigman. The result follows once when the demand distributionis governed by a finite- we show that this Markov chain has a stationary state Markov chain. More recently, Song and Zipkin distribution and admits coupling. (1993) consider a countable-state Markov environ- To construct a stationary distribution, drive the ment and show that an environment-dependent system with {fon, n > O}, and thus stationary de- basestock policy is optimal for their cost structure. mands and base-stock levels. Equations 21 and 22 Song and Zipkin also discuss modeling applications show that Yn+ 1 is increasing and continuous in Yn for and review related work. all values of the other arguments in these recursions. This shows that the distributionof Yn converges to a 6.1. Ergodic Environment stationary distribution YOif YO= 0; see Theorem 1. Throughout this section 0 = {0,n, n > 0} is a Harris Moreover, YOcan be constructed on the same prob- ergodic Markov chain representing the state of the ability space as e to make (YO, 60) stationary for world. Demands vary with 0, so we let the base-stock {(Yn, On), n > O}.The proof that Y0 is finite proceeds levels vary too. Denote by Sn = (S1, ..., S') the much as in Theorem 1. vector of base-stock levels in period n. Our key To show coupling, observe that because e is Harris assumption is that (Dn, Sn) = g(0n) for some func- ergodic there exists a finite random time N at which tion g. In the terminology of Sigman, demands and 0 couples with its stationary version. Subsequently, base-stock levels are govermed by the environment. any two copies of Y driven by the same 0 are driven 924 / GLASSERMAN AND TAYUR by the same stationary version. It suffices to show ACKNOWLEDGMENT that any such copy of Y couples with one started at The first author is supported by the National Science zero. At some finite Nm > N, the maximum in (21) is Foundation under grant MSS-9216490. attained by the first term; otherwise Ym would de- crease to -oo. Subsequently, ym agrees with a copy started at zero. Now proceed by induction on i from REFERENCES m down to 1. Some time after (yi+11 ... , Y7) has ASMUSSEN, S. 1987. Applied Probability and Queues. coupled with a copy started at zero, the maximum in John Wiley, New York. (22) must be attained by either the first or the third ASMUSSEN,S., ANDH. THORISSON.1987. A Markov Chain term, and at that time Yi couples. Approach to Periodic Queues. J. Appl. Prob. 24, 215-225. Remark. A referee points out that the extension from BACCELLI,F., AND Z. LIU. 1992. On a Class of Stochastic Theorem 2 to Theorem 7 can be argued based on a Recursive Sequences Arising in Queueing Theory. general coupling result of Borovkov and Foss (1992). Anns. Prob. 20, 350-374. BACCELLI, F., W. A. MASSEY AND D. TOWSLEY. 1989. 6.2. Periodic Demands Acyclic Fork-Join Queueing Networks. J. ACM 36, Perhaps the greatest limitation of the usual assump- 615-642. BOROVKOV,A., AND S. Foss. 1992. Siberian Adv. in tion of demand stationarity is that it rules out seasonal Math. 1, 16-81. or, more generally, periodic effects. We now intro- BROCKWELL,P. J., S. I. RESNICKAND R. L. TWEEDIE. duce periodicity in demands throughperiodicity in the 1982. Storage Processes With General Release Rule environment. and Additive Inputs. Adv. Appl. Prob. 14, 392-433. BILLINGSLEY, P. 1968. Convergence of Probability 7 in Theorem 8. Let the conditions of Theorem be Measures. John Wiley, New York. effect, except that now 0 is positive Haris recurrent CLARK, A. J., AND H. SCARF. 1960. Optimal Policies for > withperiod d > 1. Then {(Yn, 0)n), n O}is positive a Multi-Echelon Inventory Problem. Mgmt. Sci. 6, Haris recurrent. 475-490. FEDERGRUEN,A., AND P. ZIPKIN. 1984. Computational Proof. As in subsection VI.3 of Asmussen or subsec- Issues in an Infinite-Horizon Multiechelon Inven- tion 2.4 of Nummelin, the state space of e can be tory Model. Opns. Res. 32, 818-836. partitioned into d sets E1, ... , Ed such that ' =_ FEDERGRUEN,A., AND P. ZIPKIN. 1986a. An Inventory {0nd+i, n > O}has state-space Ei, i = 1, ..., d. By Model With Limited Production Capacity and Un- Proposition 3.14 of Nummelin, each 0' is Harris re- certain Demands, I: The Average Cost Criterion. current on Ei, i = 1, ... , d, and is, in fact, positive Math. Opns. Res. 11, 193-207. Harris recurrent because 0 is positive Harris recur- FEDERGRUEN,A., AND P. ZIPKIN. 1986b. An Inventory rent. By construction, each (h is aperiodic and thus Model With Limited Production Capacity and Un- certain Demands, II: The Discounted Cost Harris ergodic. Now, just as in Theorem 7, {(Yn, O)n), Criterion. Math. Opns. Res. 11, 208-215. n > O} has a stationary distribution (YO, the 00); FRANKEN, P., D. KONIG, U. ARNDT AND V. SCHMIDT. of this did not use Harris proof step ergodicity, just 1982. Queues and Point Processes. John Wiley, the existence of a stationary version of (0. Let Zi be Chichester, England. > the process {(Ynd+i, Ond+i), n O}on R+ x Ei, i = GLASSERMAN. P., AND S. TAYUR. 1992a. Sensitivity 1, ... , d. Then the distribution 79 defined by Analysis for Base-Stock Levels in Multi-Echelon Production-Inventory Systems. Mgmt. Sci. (to * = E E= 7r'( ) P((fo, 60o) *160 Ei ) appear). is stationary for Zi. Moreover, {(Yn, On), n > O} GLASSERMAN,P., AND S. TAYUR. 1992b. The Stability of admits coupling from any two initial states in R+nx a Capacitated, Multi-Echelon Production-Inventory Ei, because 0 admits coupling from any two states System Under a Base-Stock Policy, Working Paper in Et (by Harris ergodicity of Oi) and Y couples once 92-36, GSIA, Carnegie Mellon University, 0 couples, as in Theorem 7. It follows that each Z1, Pittsburgh, Penn. GLASSERMAN,P., AND D. D. YAO. 1992. The Stochastic i = 1, ..., d admits coupling and is thus Harris Vector Equation Y, + 1 = An 0 Y,, E Bn With = ... ird ergodic. Taking if 7rr + + satisfies condi- Stationary Coefficients. Adv. Appl. Prob. (to tion 10 and shows that {(Yn, O)n),n > O} is Harris appear). recurrent. Since each ir- is finite, so is the stationary GLYNN, J. E. 1989. A Discrete Time Storage Process distribution of (Y, 0), which is therefore positive With a General Release Rule. J. Appl. Prob. 26, Harris recurrent. 566-583. GLASSERMAN AND TAYUR / 925
GRAVES,S., A. RINNOOYKAN AND P. ZIPKIN(EDS.). 1992. SIGMAN, K. 1988. Queues as Harris Recurrent Markov Logistics of Production and Inventory, Vol. 4. Chains. QUESTA 3, 179-198. North-Holland, Amsterdam. SONG, J. S., AND P. ZIPKIN. 1993. Inventory Control in a IGLEHART,D. L., AND S. KARLIN. 1962. Optimal Policy Fluctuating Demand Environment. Opns. Res. 41, for Dynamic Inventory Process With Nonstationary 351-370. Stochastic Demands. In Studies in Applied Proba- SVORONOS, A., AND P. ZIPKIN. 1991. Evaluation of One- bility and Management Science, K. Arrow, S. for-One Replenishment Policies for Multiechelon Karlin and H. Scarf (eds.). Stanford University Inventory Systems. Mgmt. Sci. 37, 68-83. Press, Stanford, Calif. TAYUR, S. 1992. Computing the Optimal Policy for Ca- LOYNES, R. M. 1962. The Stability of a Queue With pacitated Inventory Models. Comm. Statist. Stoch. Non-Independent Interarrival and Service Times. Models 9, 585-598. Proc. Cambridge Philos. Soc. 58, 497-520. THORISSON, H. 1983. The Coupling of Regenerative NUMMELIN,E. 1984. General Irreducible Markov Chains Processes. Adv. Appl. Prob. 15, 531-561. and Non-Negative Operators. Cambridge Univer- VAN HOUTUM, G. J., AND W. H. M. ZIJM. 1990. Compu- sity Press, London. tational Procedures for Stochastic Multi-Echelon PRABHU, U. 1965. Queues and Inventories. John Wiley, Production Systems. Centre for Quantitative New York. Methods, Nederlandse Philips Bedrijven, B.V. ROSLING, K. 1989. Optimal Inventory Policies for ZIPKIN, P. 1986. Stochastic Leadtimes in Continuous Assembly Systems Under Random Demands. Time Inventory Models. Naval Res. Logist. Quart. Opns. Res. 37, 565-579. 33, 763-774.