Maury Bramson University of Minnesota Stability and Heavy Traffic Limits for Queueing Networks May 15, 2006 Springer Berlin Heidelberg NewYork Hong Kong London Milan Paris Tokyo Contents 1 Introduction . 5 1.1 The M=M=1 Queue . 6 1.2 Basic Concepts of Queueing Networks . 8 1.3 Queueing Network Equations and Fluid Models . 17 1.4 Outline of Lectures . 20 2 The Classical Networks . 23 2.1 Main Results . 24 2.2 Stationarity and Reversibility . 29 2.3 Homogeneous Nodes of Kelly Type . 33 2.4 Symmetric Nodes . 38 2.5 Quasi-Reversibility . 45 3 Instability of Subcritical Queueing Networks . 59 3.1 Basic Examples of Unstable Networks . 60 3.2 Examples of Unstable FIFO Networks . 66 3.3 Other Examples of Unstable Networks . 77 4 Stability of Queueing Networks . 83 4.1 Some Markov Process Background . 86 4.2 Results for Bounded Sets . 97 4.3 Fluid Models and Fluid Limits . 106 4.4 Demonstration of Stability . 121 4.5 Appendix . 132 5 Applications and Some Further Theory . 145 5.1 Single Class Networks . 146 5.2 FBFS and LBFS Reentrant Lines . 150 5.3 FIFO Networks of Kelly Type . 153 5.4 Global Stability . 161 5.5 Relationship Between QN and FM Stability . 169 1 Introduction Queueing networks constitute a large family of models in a variety of settings, involving \jobs" or \customers" that wait in queues until being served. Once its service is completed, a job moves to the next prescribed queue, where it remains until being served. This procedure continues until the job leaves the network; jobs also enter the network according to some assigned rule. In these lectures, we will study the evolution of such networks. Two aspects of their evolution have been the object of considerable interest over the last two decades. The first is the question of when a network is stable. That is, when is the underlying Markov process of the queueing network positive Harris recurrent? When the state space is countable and all states communicate, this is equivalent to the Markov process being positive recurrent. The other topic is the existence of heavy traffic limits for queueing networks. That is, when does a sequence of networks, under diffusive scaling, converge to a reflecting Brownian motion? We will be interested in both questions, while devoting more effort to the former. A unifying theme in our approach to both questions will be the application of fluid models, which may be thought of as being, in a general sense, dynamical systems that are associated with the networks. The goal of this chapter is to provide a quick introduction to queueing networks. We will provide basic vocabulary and attempt to explain some of the concepts that will motivate later chapters. The chapter is organized as follows. In Section 1.1, we discuss the M=M=1 queue, which is the \simplest" queueing network. It consists of a single queue, where jobs enter according to a Poisson process and have exponentially distributed service times. Both the problems of stability and heavy traffic limits are not difficult to resolve in this setting. Using M=M=1 queues as motivation, we proceed to more general queueing networks in Section 1.2. We introduce many of the basic concepts of queueing networks, such as the discipline (or policy) of a network determining which jobs are served first, and the traffic intensity ρ of a network, which provides a natural condition for deciding its stability. In Section 1.3, we provide a pre- liminary description of fluid models, and how they can be applied to provide 6 1 Introduction conditions for the stability of queueing networks and the existence of heavy traffic limits. In Section 1.4, we summarize the topics we will cover in the remaining chapters. These include the product representation of the stationary distri- butions of certain classical queueing networks in Chapter 2, and examples of unstable queueing networks in Chapter 3. Chapters 4 and 5 introduce fluid models and apply them to queueing network stability. Chapter 6 applies fluid models to heavy traffic limits. 1.1 The M=M=1 Queue The M=M=1 queue, or simple queue, is the most basic example of a queue- ing network. It is familiar to most probabilists and is simple to analyze. We therefore begin with a summary of some of its basic properties to motivate more general networks. The setup consists of a server at a workstation, and \jobs" (or \cus- tomers") who line up at the server until they are served, one by one. After service of a job is completed, it leaves the system. The jobs are assumed to arrive at the station according to a Poisson process with intensity 1; equiva- lently, the interarrival times of succeeding jobs are given by independent rate -1 exponentially distributed random variables. The service times of jobs are given by independent rate-µ exponentially distributed random variables, with µ > 0; the mean service time of jobs is therefore m = 1/µ. We are inter- ested here in the behavior of Z(t), the number of jobs in the queue at time t, including the job currently being served (see Figure 1.1). Fig. 1.1. Jobs enter the system at rate 1 and depart at rate µ. There are currently 2 jobs in the queue. The process Z(·) can be interpreted in several ways. Because of the inde- pendent exponentially distributed interarrival and service times, Z(·) defines a Markov process, with states 0; 1; 2; : : :. (M=M=1 stands for Markov input and Markov output, with one server.) It is also a birth and death process on f0; 1; 2; : : :g, with birth rate 1 and death rate µ. Because of the latter, it is easy to compute the stationary (or invariant) probability measure πm of Z(·) when it exists, since the process will be reversible. Such a measure satisfies πm(n + 1) = mπm(n) for n = 0; 1; 2; : : : : 1.1 The M=M=1 Queue 7 since it is constant, over time, on the intervals [0; n] and [n + 1; 1). It follows that when m < 1, πm is geometrically distributed with n πm(n) = (1 − m)m ; n = 0; 1; 2; : : : : (1.1) All states clearly communicate with one another, and the process Z(·) is −1 positive recurrent. The mean of πm is m(1 − m) , which blows up as m " 1. When m ≥ 1, no stationary probability measure exists for Z(·). Using standard reasoning, one can show that Z(·) is null recurrent when m = 1 and is transient when m > 1. One can also interpret Z(·) as a continuous time biased simple random walk, which is reflected at 0. This viewpoint is natural when one sets 1 Z^r(t) = Zr(r2t) (1.2) r for a sequence of such processes Zr(·), with m replaced by mr. When r(mr − 1) ! β as r ! 1; (1.3) for some β, Z^r(·) converges in distribution to a reflecting Brownian motion with drift β and variance 2; this is the analog of the standard invariance principle for random walk in one dimension. In this setting, one can think of a given system Z^r(·) as having a service (or processing) capacity that approximates the rate at which \work" enters the system; this capacity might be chosen for economic reasons. When mr ≈ 1, Z^r(·) will be approximated by such a Brownian motion. The number of jobs typically in the system at a given time will be of order r. The behavior of Z(·) that was observed in the last two paragraphs pro- vides the basic motivation for these lectures, in the context of the more general queueing networks which will be introduced in the next section. We will in- vestigate when the Markov process corresponding to a queueing network is stable, i.e., is positive Harris recurrent, and when a sequence of such queue- ing networks has a heavy traffic limit, i.e., converges to a reflecting Brownian motion. For M=M=1 queues, we explicitly constructed a stationary probability measure to demonstrate positive recurrence of the Markov process. Typically, however, such a measure will not be explicitly computable, since it will not be reversible. This, in particular, necessitates a new, more qualitative, approach for showing positive recurrence. We will present such an approach in Chapter 4. The limit we stated for Z^r(·) is a variant of the invariance principle for random walk, and as such, is not surprising. In the setting of more general queueing networks, the formulation of the analogous limits will no longer be obvious, and requires a new framework. We will present such an approach in Chapter 6. 8 1 Introduction 1.2 Basic Concepts of Queueing Networks The M=M=1 queue admits natural generalizations in a number of directions. It is unnecessary to assume that the interarrival and service distributions are exponential. For general distributions, one employs the notation G=G=1; or M=G=1 or G=M=1, if one of the distributions is exponential. (To emphasize the independence of the corresponding random variables, one sometimes uses the notation GI instead of G.) The single queue can be extended to a finite system of queues, or a queue- ing network, where jobs, upon leaving a queue, line up at another queue, or station, or leave the system. The queueing network in Figure 1.2 is also a reentrant line, since all jobs follow a fixed route. j=1 j=2 j=3 Fig. 1.2. A reentrant line with 3 stations. Depending on a job's previous history, one may wish to prescribe different service distributions at its current station or different routing to the next station.