Lectures on scaling methods in stochastic networks Philippe Robert INRIA, Domaine de Voluceau, B.P. 105, 78153 Le Chesnay Cedex, FRANCE E-mail address: [email protected] CHAPTER 1 Rescaled Markov Processes and Fluid Limits Contents 1. Introduction 3 2. Rescaled Markov Processes 5 2.1. Fluid Limits 5 2.2. Examples 5 2.3. Fluid Limits as Dynamical Systems 10 3. Fluid Limits of a Class of Markov Processes 11 4. Relations with Skorohod Problems 19 4.1. The M/M/1 Queue 20 4.2. Jackson Networks 21 5. Criteria for Ergodicity Properties 27 5.1. A Criterion for Ergodicity 27 5.2. Absorption at 0 of Fluid Limits 30 5.3. An Interesting Example 33 6. Local Equilibrium of a Rescaled Markov Process 38 An M/M/1 Queue in a Markovian Environment 39 7. Bibliographical Notes 43 1. Introduction It is in general quite difficult to have a satisfactory description of an ergodic Markov process describing a stochastic network. When the dimension of the state space d is greater than 1, the geometry complicates a lot any investigation : Ana- lytical tools of Chapter ?? for dimension 1 cannot be easily generalized to higher dimensions. Note that the natural order on the real line plays an important role for Wiener-Hopf methods. The example of queueing networks seen in Chapter ?? for which the stationary distribution has a product form should be seen as an interest- ing exception, but an exception. In the same way as in Chapter ??, it is possible nevertheless to get some insight on the behavior of these processes through some limit theorems. In this chapter, limit results consist in speeding up time and scaling appropriately the process itself with some parameter. The behavior of such rescaled stochastic processes is analyzed when the scaling parameter goes to infinity. In the limit one gets a sort of caricature of the initial stochastic process which is defined as a fluid limit (see the rigorous definition below). As it will be seen, a fluid limit keeps the main characteristics of the initial stochastic process while some stochastic fluctuations of second order vanish with this procedure. In “good cases”, a fluid limit is a deterministic function, solution of some ordinary differential equation. As it can be expected, the general situation is somewhat more complicated. These 3 4 1. RESCALED MARKOV PROCESSES AND FLUID LIMITS ideas of rescaling stochastic processes have emerged recently in the analysis of sto- chastic networks, to study their ergodicity properties in particular. See Rybko and Stolyar[1] for example. In statistical physics, these methods are quite classical, see Comets[2]. The chapter is organized as follows: Section 2 gives the basic definitions con- cerning the scaling of the stochastic process, fluid limits in particular are introduced. Several examples are presented and discussed. Section 3 introduces a large class of Markov processes for which general results on the existence of fluid limits hold. Sec- tion 4 investigates the relation between fluid limits and solutions of some Skorohod problem (see Annex ??). Section 5 establishes a relation between ergodicity of the initial stochastic process and stability of fluid limits at 0. An interesting example of a null-recurrent Markov process having fluid limits converging to 0 concludes the section. Section 6 studies fluid limits of Markov processes having a subset of their coordinates at equilibrium. In the following, (X(x, t)) denotes an irreducible c`adl`ag Markov jump process on a countable state space starting from x , i.e. such that X(x, 0) = x . The notation (X(t)) can beS used when the dependence∈ S on the initial point is∈ not S ambiguous. As before, (ω, dx), ω Ω, denotes a Poisson point process on R Nξ ∈ with parameter ξ R+, all Poisson processes used are assumed to be a priori independent. The∈ topology on the space of probability distributions induced by the Skorohod topology on the space of c`adl`ag functions D([0,T ], Rd) is used. The reading of Annex ?? is recommended, the definition and the main results on this topology are briefly recalled there. Note that the assumption of a countable state space forbids to consider queue- ing networks with general service distributions or general interarrival distributions. This is not a real restriction since, in general, fluid limits do not really depend on distributions but on their averages. Hence the assumption that all distributions considered are exponential is only formally restrictive. As usual, it simplifies much the description of the underlying Markov process: Forward recurrence times vectors for interarrival intervals and services can be safely ignored. Note that among clas- sical models there are some exceptions to this fact: The G/G/1 Processor-Sharing queue has a fluid limit which depends on the distribution of services; see Jean-Marie and Robert[3]. Since this topic is quite recent and still evolving (some important questions are still open, see Section 7 for a quick survey), this chapter is only an introduction to fluid limits. Generally speaking, the main goal of this chapter is to advertise the use of scaling methods in the study of complex Markov processes. For this reason, simple examples are scattered in the text to illustrate some of the problems encountered. [1] A.N. Rybko and A.L. Stolyar, On the ergodicity of random processes that describe the func- tioning of open queueing networks, Problems on Information Transmission 28 (1992), no. 3, 3–26. [2] Francis Comets, Limites hydrodynamiques, Ast´erisque (1991), no. 201-203, Exp. No. 735, 167–192 (1992), S´eminaire Bourbaki, Vol. 1990/91. MR 93e:60194 [3] Alain Jean-Marie and Philippe Robert, On the transient behavior of some single server queues, Queueing Systems, Theory and Applications 17 (1994), 129–136. 2. RESCALED MARKOV PROCESSES 5 A different scaling is presented in Section ?? of Chapter ?? for the simple case of the M/M/ queue, see Hunt and Kurtz[4] for a presentation of the general case. For this scaling,∞ some of the methods of this chapter can be used (martingale parts of the associated Markov processes also vanish asymptotically for example). 2. Rescaled Markov Processes Throughout this chapter, it is assumed that the state space can be embedded in a subset of some normed space (Rd for example), denotesS the associated norm. k·k Definition 1. For x , (X(x, t)) denotes the process (X(x, t)) renormalized so that for t 0, ∈ S ≥ 1 X(x, t)= X x, x t . x k k k k As usual, if there is no ambiguity on x, the notation (X(t)) will be used. The scaling consists in speeding up time by the norm of the initial state and the space variable by its inverse. The rescaled process starts from x/ x , a state of norm 1. k k Only continuous time Markov processes are considered in this chapter. In discrete time, if (Xn) is a Markov chain, the corresponding rescaled process can be also defined by X X(x, t)= ⌊kxkt⌋ , x k k if X0 = x and t R+, where y is the integer part of y R. Most of the results of this∈ S chapter∈ are also valid⌊ in⌋ this case. ∈ 2.1. Fluid Limits. Definition 2. A fluid limit associated with the Markov process (X(t)) is a sto- chastic process which is one of the limits of the process X(x, x t) X(x, t) = k k x k k when x goes to infinity. k k Strictly speaking, if Qx is the distribution of (X(x, t)) on the space of c`adl`ag d functions D(R+, ), a fluid limit is a probability distribution Q on D(R+, R ) such that S Q = lim Qx e n n for some sequence (x ) of whose norm converges to infinity. By choosing an n e appropriate probability space,S it can be represented as a c`adl`ag stochastic process (W (t)) whose distribution is Q. A fluid limit is thus an asymptotic description of sample paths of a Markov process with a large initial state. e 2.2. Examples. [4] P.J. Hunt and T.G Kurtz, Large loss networks, Stochastic Processes and their Applications 53 (1994), 363–378. 6 1. RESCALED MARKOV PROCESSES AND FLUID LIMITS 2.2.1. A Random Walk on Z. For λ > 0, λ is a Poisson process on R with intensity λ (as usual (A) denotes the numberN of points of Poisson process in Nλ Nλ a Borelian subset A of R). Let (Yi) be an i.i.d. sequence of integer valued random variables having a finite second moment. For x Z, the process (X(x, t)) defined by ∈ Nλ(]0,t]) X(x, t)= x + Yi i=1 X is clearly a Markov process. The associated rescaled process (X(x, t)) is given by N (]0,|x|t]) 1 λ X(x, t)= x + Y , x i i=1 | | X if t 0 and x Z. The independence properties of the Poisson process and the ≥ ∈ sequence (Yi) give that x X(x, t) λE(Y )t − x − 0 | | N (]0,|x|t]) 1 λ E(Y ) = (Y E(Y )) + 0 [ (]0, x t]) λ x t] x i − 0 x Nλ | | − | | i=1 | | X | | is a martingale whose increasing process is 1 λt var(Y )+ E(Y )2 . x 0 0 | | Doob’s Inequality (Theorem ?? page ??) shows that for ε> 0 and t 0, ≥ 1 P sup X(x, s) sgn x λE(Y ) s ε λt var(Y )+ E(Y )2 , − − 0 ≥ ≤ x ε2 0 0 0≤s≤t | | where sgn z = 1 if z 0 and 1 otherwise.