Product-Form in Queueing Networks

Product-form in queueing networks

VRIJE UNIVERSITEIT

Product-form in queueing networks

ACADEMISCH PROEFSCHRIFT

ter verkrijging van de graad van doctor aan de Vrije Universiteit te Amsterdam, op gezag van de rector magniﬁcus dr. C. Datema, hoogleraar aan de faculteit der letteren, in het openbaar te verdedigen ten overstaan van de promotiecommissie van de faculteit der economische wetenschappen en econometrie op donderdag 21 mei 1992 te 15.30 uur in het hoofdgebouw van de universiteit, De Boelelaan 1105

door Richardus Johannes Boucherie

geboren te Oost- en West-Souburg

Thesis Publishers Amsterdam 1992 Promotoren: prof.dr. N.M. van Dijk prof.dr. H.C. Tijms Referenten: prof.dr. A. Hordijk prof.dr. P. Whittle Preface

This monograph studies product-form distributions for queueing networks. The celebrated product-form distribution is a closed-form expression, that is an analytical formula, for the queue-length distribution at the stations of a queueing network. Based on this product-form distribution various solution techniques for queueing networks can be derived. For example, aggregation and decomposition results for product-form queueing networks yield Norton’s theorem for queueing networks, and the arrival theorem implies the validity of mean value analysis for product-form queueing networks. This monograph aims to characterize the class of queueing networks that possess a product-form queue-length distribution. To this end, the transient behaviour of the queue-length distribution is discussed in Chapters 3 and 4, then in Chapters 5, 6 and 7 the equilibrium behaviour of the queue-length distribution is studied under the assumption that in each transition a single customer is allowed to route among the stations only, and finally, in Chapters 8, 9 and 10 the assumption that a single customer is allowed to route in a transition only is relaxed to allow customers to route in batches. In all cases necessary and sufficient conditions for a product-form queue-length distribution are given and solution techniques for product-form queueing networks are illustrated. For the equilibrium behaviour emphasis will be on the study of the influence of state-dependent routing on the existence of product-form distributions, that is queueing networks in which customers may be blocked while routing or in which routing may be otherwise influenced by the state of the queueing network are studied. I have chosen to analyse product-form queueing networks for a number of reasons. First, as is observed above, product-forms are useful for practical systems since various solution techniques (also for non product-form queueing networks) are based on product-form results for queueing networks. Second, I have been interested in balance equations and product- forms are usually obtained when a process satisfies some notion of local

v vi Preface balance. Third, because product-forms analysis of queueing networks is a method to obtain exact results, and finally because it is more fun. The basic method used to analyse queueing networks in this monograph is a method based on the approach used in physics. In my opinion, a physical interpretation or a physical analogue of a queueing network is very important. This can be seen from various results obtained in this monograph. For example, the dual process discussed in Chapter 5 is a direct consequence of the physical interpretation of a queueing network in terms of potential energy, Norton’s theorem as obtained in Chapter 6 is a consequence of the electrical-circuit analogue of a queueing network, and the theory on queueing networks with negative and positive customers as discussed in Chapter 10 shows similarities with Dirac’s theory of holes as described in quantum mechanics. Also, the interpretation of the balance equations yielding the equilibrium distribution of a queueing network is a physical interpretation of the behaviour of the queueing network. Global balance which forms the basis of the equilibrium analysis states that the flow out of a state equals the flow into a state. Although flow represents probability flow, the intuitive relation with flow of liquids is obvious. A queueing network is shown to possess a product-form equilibrium distribution if the flow into and out of each station of the queueing network is balanced. Therefore, the physical interpretation of a queueing network is very important. In fact, it is shown in this monograph that a queueing network possesses a product-form distribution if and only if a physical interpretation of the rate into and out of a station is possible. Although product-form distributions are the topic of all chapters of this monograph, the various chapters are independent of each other to a large extent. All chapters, except for the introductory chapters, are mainly based on publications and research reports. Therefore, all chapters are self-contained. The following table gives the papers which are the basis of the chapters of this monograph:

[3] Transient product form distributions in queueing networks. (with P.G. Taylor) Research report, The University of Western Australia, Department of Mathematics, August 1990, submitted.

[4] A note on the transient behaviour of the Engset loss model. To appear: Stochastic Models

[5] A dual process associated with the evolution of the state of a queueing network at its jumps. Research Memorandum 1991-39, Vrije Uni- Preface vii

versiteit, Faculteit der Economische Wetenschappen en Econometrie, Amsterdam, submitted.

[6] A generalization of Norton’s theorem. (with N.M. van Dijk) Research Memorandum 1991-61, Vrije Universiteit, Faculteit der Economische Wetenschappen en Econometrie, Amsterdam, submitted.

[7] Aggregation of Markov chains. To appear: Stoch. Proc. Appl.

[8] Spatial birth-death processes with multiple changes and applications to batch service networks and clustering processes. (with N.M. van Dijk) Adv. Appl. Prob. 22, 433-455 (1990).

[9] Product forms for queueing networks with state dependent multiple job transitions. (with N.M. van Dijk) Adv. Appl. Prob. 23, 152-187 (1991).

[10] Local balance in queueing networks with positive and negative customers. (with N.M. van Dijk) Research Memorandum 1992-1, Vrije Universiteit, Faculteit der Economische Wetenschappen en Econome- trie, Amsterdam, submitted.

Sections are numbered within a chapter. A reference to Section 3 is to Section 3 of the current chapter and a reference to Section 2.3 is to Sec- tion 3 of Chapter 2. Formulas are numbered within a section. A reference to formula (5.7) is to formula (5.7) appearing in Section 5 of the current chapter, and a reference to formula (3.5.7) is to formula (5.7) appearing in Section 5 of Chapter 3. A similar convention is used for theorems and ﬁgures.

I should like to take this opportunity of thanking the National Operations Research Network in the Netherlands (LNMB – Landelijk Netwerk Mathe- matische Besliskunde) for providing an excellent training programme for Ph.D.-students. In particular, I wish to thank the LNMB for providing funds allowing me to visit The University of Western Australia, Perth, Australia, from July 1, 1990 until August 12, 1990. With respect to this visit my thanks are also due to dr. P.G. Taylor for the fruitful discussions and joint research during this visit. I should like to thank my colleagues at the Department of Econometrics of the Vrije Universiteit for stimulating discussions about this, that and the other during coﬀee, lunch and tea breaks. viii Contents

I am also very grateful to prof.dr. H.C. Tijms and prof.dr. N.M. van Dijk for their encouragement, comments and advice during the past years in which I was employed at the Vrije Universiteit, and to R.D. Nobel for carefully reading parts of this monograph. Finally, I owe a special debt of gratitude to Carla for her constant support during the past years.

Amsterdam, February 1992

Richard Boucherie Contents

1 Introduction 1 1 Motivation ...... 1 2 Outline ...... 4

2 Preliminaries 11 1 Basic results on Markov chains ...... 11 2 Queueing network model ...... 19 3 Balance equations ...... 25 4 Product-form distributions; literature ...... 29 5 Review of assumptions ...... 38

3 Transient product-form distributions 39 1 Introduction ...... 39 2 Model and canonical representation ...... 40 2.1 Canonical form for the open network ...... 42 2.2 Canonical form for the closed network ...... 44 3 Suﬃcient conditions ...... 46 4 Necessary conditions for the open network ...... 50 5 Necessary conditions for the closed network ...... 56 6 Discussion and general remarks ...... 59

4 Transient behaviour of the Engset loss model 63 1 Introduction ...... 63 2 Engset loss model ...... 64 3 Transient queue-length distribution ...... 66

5 Dual processes 75 1 Introduction ...... 75 2 The primal process ...... 77 3 Deﬁnition of the dual process ...... 83

ix x CONTENTS

4 The dual routing function ...... 88 4.1 Similar transitions ...... 90 4.2 Similar subtransitions ...... 93 4.3 Similar transitions; reversed potential ...... 94 4.4 Similar subtransitions; reversed potential ...... 95 5 Relation between the primal and the dual process . . . . . 96 5.1 Equal probability ﬂow ...... 97 5.2 Palm probabilities ...... 100 6 Examples ...... 105 6.1 Complementary slackness relations ...... 105 6.2 Blocking examples ...... 106 6.3 Alternative transition rates; dual states ...... 109 6.4 Alternative transition rates; traﬃc equations . . . . 110 6.5 Self-dual process ...... 111 6.6 Customer-vacancy duality ...... 112

6 Norton’s theorem 115 1 Introduction ...... 115 2 Electrical circuit theory ...... 117 3 Queueing network model ...... 120 4 Decomposition into clusters ...... 124 4.1 Routing: Conditions ...... 124 4.2 Routing: Examples ...... 129 4.3 Service: Conditions ...... 134 4.4 Service: Examples ...... 135 5 Norton’s theorems ...... 138 6 Practical applications of Norton’s theorems ...... 146 7 Examples ...... 149 7.1 Global throughput determines local behaviour; workload balancing ...... 149 7.2 Internal blocking ...... 151 7.3 Nested aggregation ...... 152 8 Literature ...... 153 8.1 Literature on Norton’s theorem ...... 153 8.2 Literature on factorization into subnetworks . . . . 155 8.3 Weak coupling ...... 155 8.4 Relation to electrical circuit theory ...... 156 CONTENTS xi

7 Amalgamation of Markov chains 159 1 Introduction ...... 159 2 Model and cross-balance ...... 160 3 Examples ...... 167 3.1 Disjoint irreducible sets; reducible process . . . . . 168 3.2 Disjoint irreducible sets; irreducible process . . . . 168 3.3 State space truncation ...... 169 3.4 Time-reversal ...... 170 3.5 Nearly disjoint irreducible sets ...... 170 3.6 Construction method ...... 175 3.7 Product-form queueing networks ...... 180

8 Strong reversibility 187 1 Introduction ...... 187 2 Model ...... 188 3 Applications ...... 199 3.1 Queueing networks with single changes ...... 199 3.2 Queueing networks with multiple changes ...... 204 3.3 Clustering processes ...... 207

9 Group-local-balance 217 1 Introduction ...... 217 2 Model ...... 218 3 Decomposition theorem ...... 227 4 Network applications: service-form examples ...... 231 4.1 Station-dependent service-rates ...... 232 4.2 Cluster-dependent service-rates ...... 234 5 Network applications: routing-form examples ...... 236 5.1 No blocking; batch routing ...... 237 5.2 No blocking; independent routing ...... 237 5.3 Upper limit blocking; reversible routing ...... 243 5.4 Minimal workload blocking ...... 244 6 Dual processes ...... 245 7 Speciﬁc examples ...... 246 7.1 Discrete-time queueing networks ...... 246 7.2 Deterministic anticipative blocking ...... 252 7.3 Random anticipative blocking ...... 254 7.4 Clusters of stations ...... 255 xii CONTENTS

10 Negative customers 259 1 Introduction ...... 259 2 Model and local balance ...... 263 3 State space truncation based on local balance ...... 272 3.1 Lower bounds ...... 272 3.2 Upper bounds ...... 281 4 Traﬃc equations ...... 285 4.1 Uniqueness of the solution ...... 285 4.2 Feedforward networks ...... 288 4.3 Balanced networks ...... 289 4.4 Pathological network ...... 290 4.5 Characteristic equations ...... 291 5 Two station queueing network ...... 293

11 Concluding remarks 297 Chapter 1

Introduction

1 Motivation

Queueing is encountered in many practical systems. For example, in a supermarket or post-oﬃce queueing at the counters is not uncommon. A queueing network is a mathematical model for systems in which queueing or congestion can occur at various stages. The basic building block of a queueing network is a station or queue, the mathematical model for a single service-facility. Customers or jobs arrive at a service-facility to receive a desired service. In the mathematical model both the arrival and the service of customers is described by stochastic processes, that is the arrival process and the required amount of service are subject to random- ness or uncertainty. Examples of stations of a queueing network are a post-oﬃce where customers are served at counters, a telephone-exchange with calls that occupy lines, a machine which produces parts or pallets, and a CPU which processes programmes. From these examples the following queueing networks arise: a postal network; a telephone network; a manufacturing system (job-shop) consisting of several machines; and a computer consisting of a CPU, memory devices and I/O-ports. Typical quantities of interest in practical systems are:

– queue-length (number of customers at the service-facility);

– sojourn-time (total time that a customer spends in the system);

– throughput (number of customers completing service per time-unit).

Queueing theory aims to obtain these quantities through analysis of a

1 2 Introduction queueing network representing the practical system. The following classi- ﬁcation of methods for analysing queueing networks can be given [18]:

1. exact closed-form expressions;

2. application of numerical techniques;

3. approximate analytical techniques;

4. experimental and simulation procedures.

Exact closed-form expressions for queueing networks yield exact analytical results for the quantities of interest. In contrast, some queueing networks can be formulated well-suited for a direct numerical evaluation, for example using recursive algorithms. Often practical phenomena such as congestion or complex service-facilities prohibit exact analytical analysis, while the size of queueing models prohibits an efficient numerical solution. Approximations may then be used to obtain the desired quantities, for example by observing simplified but closely related models, e.g., by replacing parts of the queueing network by a single station. Appropri- ate approximations may not be available or may still be computationally expensive. As a last resort simulation procedures can be used to obtain quantitative information on the queueing network. This monograph will focus on exact analysis of queueing networks yielding closed-form expressions for the distribution of the queue-length at the stations of a queueing network. From this distribution other quantities of interest such as the throughput or sojourn-time can be obtained. The evolution of the queueing network is determined by the transition matrix. From this transition matrix the transient, that is time- dependent, distribution of the queue-length can in principle be characterized, and by using a technique from matrix calculus (the so-called eigenvalue-eigenvector-expansion) an exact expression for the transient queue-length distribution can be obtained from the transition matrix. For practical systems, however, this expansion method is not applicable since the size of the transition matrix grows explosively fast with the number of stations of the queueing network. For example, the state space of a queueing network consisting of 55 stations at which the number of customers is constrained not to exceed 30 consists of 3055 states. As a consequence 3055 eigenvalues and eigenvectors each consisting of 3055 elements must be determined. If the equilibrium distribution of the queue-length at the 1.1 Motivation 3 stations is of interest only, mere calculation of this distribution as eigenvector of the transition matrix still requires the calculation of 3055 elements. The number of protons in the universe equals 1080 ≈ 3055. Techniques different from the eigenvalue-eigenvector-expansion of the transition matrix are thus required to obtain the characteristics of interest for queueing networks. These techniques should explore and make use of the special structure of queueing networks to obtain a closed-form expression (explicit formula in terms of the system-parameters such as the arrival and service-rate) for the queue-length distribution. Due to the complexity of queueing networks a general technique yielding a closed-form expression for the evolution of the queue-length distribution at the stations of a queueing network starting from an arbitrary initial configuration is not available. Only for some very special cases a closed-form expression for the transient queue-length distribution (that is with explicit time-dependence) has been derived. In contrast, the equilibrium distribution, that is the distribution averaged over a long time-period or the distribution when the system no longer changes its characteristics under stochastic influences, can be obtained in a closed form for various classes of queueing networks. Since many practical queueing processes have been observed to approach statistical equilibrium rather rapidly, the equilibrium behaviour often gives sufficiently accurate information on the behaviour of the system. The equilibrium concept for the analysis of queueing networks is not based on an expansion of the transition matrix, but is based on balance of customer-flows. A queueing network is in equilibrium if at each state or customer-configuration the flow out of that configuration due to customers entering and leaving the stations is balanced by the flow into that configuration due to customers entering and leaving the stations. As a queueing network consists of a number of separate stations a natural, but more detailed version, is that of station balance:

“At each state the ﬂow out of a station due to the departure of customers is balanced by the ﬂow into that same station due to the arrival of customers.”

This more detailed version is not generally satisfied and imposes special restrictions or conditions on the behaviour of the queueing network. If a queueing network satisfies station balance, also referred to as local balance or partial balance, then a closed-form expression for the equilibrium distribution can be obtained. This closed-form expression is called 4 Introduction product-form. A product-form is a product of simple terms for each individual station of the queueing network. Due to the simple structure of the product-form distribution, the product-form equilibrium distribution is a powerful analytical tool, not only as an exact mathematical expression for the queue-length distribution, but also as the basis for many approximate analytical techniques. Queueing theory has been surprisingly successful in capturing the most important features of congestion phenomena in practical systems such as communication networks, computer systems, manufacturing systems and traffic networks, and has become a generally accepted tool in the design and performance analysis of such systems. The product-form expression for the equilibrium queue-length distribution is a powerful analytical tool to obtain exact and approximate results for quantities of interest. There- fore, exploring the boundaries of the class of queueing networks satisfying the assumptions resulting in a product-form equilibrium distribution and analysing the structure of the stochastic processes describing these queueing networks is of practical and of theoretical interest.

2 Outline

Chapter 2 introduces the general framework of queueing networks that will be discussed in this monograph and presents the basic assumptions that will be used throughout:

– a single type of customers is considered only;

– service-times are negative-exponentially distributed;

– customers are not ordered at the stations.

These assumptions allow the queueing network to be modelled by a con- N tinuous-time Markov chain at S ⊂ IN0 . A staten ¯ ∈ S is a vector n¯ = (n1, . . . , nN ), where ni denotes the number of customers present at station i, i = 1,...,N. Under suitable assumptions the distribution P (¯n, t) of staten ¯ at time t can be determined from the Kolmogorov forward equations and the equilibrium distribution π(¯n) can be determined by letting t approach ∞. Section 2.1 presents some basic results on continuous-time Markov N chains at S ⊂ IN0 . In particular, stability and regularity of the matrix of transition rates are discussed. For a stable, regular, continuous-time 1.2 Outline 5

Markov chain the time-dependent queue-length distribution, P (¯n, t), can be obtained as solution to the Kolmogorov forward equations. The Kol- mogorov forward equations are the basis of the analysis of queueing networks presented in this monograph. The forward equations are a set of linear differential equations for the probability mass of staten ¯, P (¯n, t), and express the intuitively obvious relation that the change of mass at staten ¯ equals the mass enteringn ¯ minus the mass leavingn ¯. For t → ∞, the equilibrium queue-length distribution, π(¯n), can be obtained from the transient distribution P (¯n, t). Also, for t → ∞, the (time-dependent) Kol- mogorov forward equations reduce to the equilibrium Kolmogorov equations. These equations are generally referred to as global balance equations and will form the basis of the equilibrium analysis of queueing networks presented in this monograph. The global balance equations are extensively discussed in Section 2.3. Here different and more stringent forms of balance sufficient for global balance are discussed (e.g. detailed balance and local balance). Section 2.2 formally introduces the Markov chain representing the queueing network. Finally, Section 2.4 discusses product-form distributions and presents a brief overview of the literature on product- form queueing networks. This monograph can be seen to consist of three parts. The first part, consisting of Chapters 3 and 4, studies the time-dependent queue-length distribution. The second part studies standard queueing networks and is made up of Chapters 5, 6 and 7. The third part, Chapters 8, 9 and 10, studies queueing networks in which customers can leave the stations simultaneously. In all chapters product-form queue-length distributions are studied. As will be shown in the second and third part, a product-form equilibrium distribution can be found only if a measure can be found that satisfies local balance. Chapters 3 and 4 consider the transient queue-length distribution. Un- der the assumption that arriving customers are always accepted at a station, Chapter 3 proves that a necessary and sufficient condition for a queueing network to have a transient product-form is that all stations are infinite-server queues. Sufficiency is rather a trivial result. If all stations are infinite-server queues then a station assigns a separate server to each arriving customer independent of the other customers present at the stations. As a consequence customers route among the stations unaffected by the other customers, i.e., all customer-processes are statistically independent. Since all customers are assumed to be identical, this implies that the transient queue-length distribution is of product-form. Necessity 6 Introduction is a very strong result. Necessity implies that except for the trivial case discussed above a simple closed-form transient queue-length distribution does not exist. It seems highly unlikely that relaxing of the assumption that arriving customers are always accepted at the stations yields more general product-form results. Also, generalizing the routing process to allow congestion dependent routing will not give more general product-form results. A product-form is a simple and elegant distribution and except for pathological cases it is not to be expected that complicating the queueing network will simplify the queue-length distribution. To illustrate this statement, Chapter 4 considers the Engset loss model, a simple model in which customers arriving at a station are accepted up to a specific level only. The Engset loss model represents a telephone-exchange with s lines and N subscribers. At random times, independent of the behaviour of the other subscribers, a subscriber makes a call for a random amount of time. These calls are accepted by the telephone exchange as long as a free line is available. If all lines are busy the call is lost. The Engset loss model can be represented as a closed two-station queueing network. If a customer is not making a call it is at station 1 and if a customer decides to make a call it routes to station 2. Customers arriving at station 2 are accepted only if the queue-length does not exceed s. This relaxes the assumption that all arriving customers are accepted at a station. Chapter 4 proves that the transient distribution is of product-form if and only if N ≤ s. If N ≤ s all customers arriving at station 2 are accepted since the number of lines exceeds the number of subscribers. In fact, if N ≤ s the queueing network is equivalent to a closed queueing network consisting of two infinite-server queues. The main result of Chapter 4 is that the transient queue-length distribution can be expressed as a sum of two product-form distributions if and only if N = 2s+1. In this very special case the equilibrium distribution for the Engset loss model can be written as a sum of two equilibrium distributions for related infinite-server networks. In my opinion, this result is the basis for further research on closed-form transient queue-length distributions. Chapter 5 considers a dual process associated with the evolution of the state of a queueing network at its jumps. The dual process is related to the transition structure of queueing networks. Therefore transitions of the “primal” process are reconsidered and studied in detail. An important observation is that a transitionn ¯ → n¯ −ei +ej in which a customer routes from station i to station j virtually passes through staten ¯ − ei, the state in which the customer has left station i and has not yet arrived at sta- 1.2 Outline 7 tion j. This observation is confirmed by the fact that the transition rates can be interpreted to represent the potential difference between states. In this interpretation first the potential difference between staten ¯ − ei and staten ¯ must be overcome by the service-potential, and second the potential difference betweenn ¯ − ei andn ¯ − ei + ej is overcome by the routing-potential. Based on the potential-interpretation of the transition rates and the behaviour of the primal process at jump-moments the dual process is defined. For the dual process a transitionm ¯ → m¯ + ej − ei virtually passes through statem ¯ + ej, that is first a unit is created at station j and second a unit leaves station i. In this interpretationm ¯ is the state in which one customer is routing among the stations. The dual process can be chosen such that it describes the evolution of the state observed by a customer in transit. This gives a new interpretation and generalization of the arrival theorem and gives insight into the behaviour of queueing networks at jump-moments. Norton’s theorem, a well-known and often used result in practical situations, is discussed in Chapter 6. Norton’s theorem and the related Thevenin’s theorem are results from electrical-circuit-theory. These theorems state that in certain configurations of the electrical circuit a subcir- cuit may be replaced by a single generator and impedance. In queueing theory a similar result called Norton’s theorem for queueing networks is available for queueing networks with a simple transition structure. This result is an analogue of the result from electrical-circuit-theory. Chap- ter 6 presents a general framework for aggregation and decomposition of product-form queueing networks with state-dependent routing and service. By analogy with electrical-circuit-theory, the stations are grouped into clusters or subnetworks such that the process decomposes into a global process and a local process. Moreover, the local process factorizes into the subnetworks. The global process and the local processes can be analysed separately as if they were independent. The global process describes the behaviour of the queueing network in which each cluster is aggregated into a single station, whereas the local processes describe the behaviour of the subnetworks as if they are not part of the queueing network. The decomposition and aggregation method formalized in Chapter 6 allows to first analyse the global behaviour of the queueing network and subsequently analyse the local behaviour of the subnetworks of interest or to aggregate clusters into single stations without affecting the behaviour of the rest of the queueing network. Conditions are provided such that: – the global equilibrium distribution for aggregated clusters is of 8 Introduction

product-form; – the global equilibrium distribution can be obtained by merely monitoring the global behaviour; – the computation of a detailed distribution, including its normalizing constant, can be decomposed into the computation of a global and a local distribution; – the marginal distribution for the number of customers at the stations of a cluster can be obtained by merely solving local behaviour. As a special application, Norton’s theorem for queueing networks is extended to queueing networks with state-dependent routing such as due to capacity constraints at stations or at clusters of stations, and state- dependent service such as due to service-delays at clusters of stations. So far queueing networks with product-form equilibrium distribution are studied. Although for a wide class of queueing networks product-form equilibrium distributions are proven to exist, the class of queueing networks with a product-form equilibrium distribution is a very restricted class. Chapter 7 aims to extend this class to a class with a more general form of equilibrium distribution. To this end, a set of stochastic processes is introduced. For this set of stochastic processes the amalgamated process, a process for which the transition rates are an amalgamation of the transition rates of the processes in the set, is introduced. A sufficient condition is given, called cross-balance, a generalization of global balance to a set of processes, under which the equilibrium distribution of the amalgamated process is shown to be the same amalgamation of the equilibrium distributions of the processes in the set. Cross-balance is a general equilibrium concept and is shown to contain the time-reversal argument for proving a distribution to be the equilibrium distribution. In addition, a construction method for constructing the equilibrium distribution is obtained. Chapters 8 and 9 extend the framework of local balance to queueing networks with simultaneous state-dependent service-completions such as due to concurrent service or discrete-time-slotting, and with state- dependent batch routing. Chapter 8 considers birth-and-death processes. These processes are characterized by balance in probability flow for each type of transition separately. This type of partial balance is called strong reversibility and generalizes reversibility to processes with simultaneous jumps of multiple components. Strong reversibility reduces to “standard” 1.2 Outline 9 reversibility if simultaneous service-completions are excluded. The framework of birth-and-death processes is a very general framework. It models not only queueing networks with batch routing and service, but also chemical processes such as clustering processes from polymerization chemistry, where units may react to form larger units or disintegrate to form a number of smaller units. This behaviour differs from the behaviour of queueing networks, where the integrity of units (customers) is guaranteed. In the framework of birth-and-death processes this complication can be included without additional effort. In Chapter 8 a relationship is established between: 1. a product-form equilibrium distribution; 2. a partial symmetry condition on the transition rates; 3. a solution of a deterministic concentration or traffic equation. Chapter 9 particularizes to queueing networks. Based on a key-notion of group-local-balance, necessary and sufficient conditions are given for the equilibrium distribution to be of product-form. For practical queueing networks a local solution of the group-local-balance equations can usually be obtained. From this local solution a local process is defined. A necessary and sufficient condition for the queueing network to satisfy group-local- balance, i.e., for the global equilibrium distribution to satisfy the group- local-balance equations is that the local process is strongly reversible. The equilibrium distribution is shown to be consistent with the local solution. By using the results of Chapter 8 on strongly reversible processes, this result simplifies the investigation of the more complicated group-local- balance equations. In addition, Chapter 9 presents a decomposition theorem. This theorem states that the equilibrium distribution consists only of a part completely determined by the service-rates and a part completely determined by the routing probabilities. As a consequence, the service and routing-characteristics can be analysed separately. General batch service and batch routing examples yielding a product-form are hereby concluded. Known results on batch discrete-time and continuous-time queueing networks are hereby unified and extended, as will be illustrated by various examples. Chapters 2 - 9 discuss standard product-form distributions in queueing networks: provided that a groupg ¯ that can leave the stations can enter the stations too, the local balance equations can be defined and a product-form distribution is obtained; straightforward summation of the 10 Introduction local balance equations implies that this distribution is the equilibrium distribution. In contrast, recently a product-form equilibrium distribution is obtained for a queueing network wherein two customers may leave distinct stations simultaneously but one customer is allowed to enter at a time only [37]. This queueing network is modelled by a queueing network with positive and negative customers. Positive customers behave as normal customers and enlarge the queue-length by one when they enter a station. Negative customers show a different behaviour. Upon entrance in a station they reduce the queue-length by one. Customers at the stations are of single type (positive). Upon departure from a station a customer either remains a positive customer or becomes a negative customer. In the latter case the customer may route to another station to reduce the queue-length by one. As a consequence a simultaneous “departure” of two customers has occurred. The “standard” notion of local balance does not apply to this queueing network since two customers cannot enter two stations simultaneously. Applications of queueing networks with positive and negative customers are neural networks, where positive customers increase the potential of a neuron by one unit and negative customers decrease the potential of a neuron by one unit; multiple resource systems, where positive customers represent requests for service and negative customers represent decisions to cancel a request. Chapter 10 studies these queueing networks and introduces a new type of local balance to account for the product-form equilibrium distribution. This new type of local balance relates the probability flow out ofn ¯ to the probability flow inton ¯0 6=n ¯. As a consequence, the local balance equations cannot be straightforwardly summed to obtain the global balance equations. However, local balance as introduced in Chapter 10 allows manipulations similar to standard local balance, for example to obtain blocking results. Local balance for queueing networks with positive and negative customers reduces to standard local balance when negative customers are excluded. Finally, Chapter 11 presents some concluding remarks and an overview of the key-results obtained in this monograph. Chapter 2

Preliminaries

This chapter reviews and discusses the basic assumptions and techniques that will be used in this monograph. Proofs of basic results given in this chapter are omitted, but can be found in standard textbooks on Markov chains and queueing theory, e.g. [1], [16], [17], [55], [83], [87], [88]. Results from these references are used in this chapter without reference except for cases where a specific result (e.g. theorem) is inserted into the text. Section 1 introduces the Kolmogorov forward equations and the assumptions sufficient for these equations to be valid. The Kolmogorov forward equations are the basis of the analysis of queueing networks presented in this monograph. A formal queueing network is described in Section 2 and a Markov chain representing this queueing network is derived. Section 3 discusses different types of local balance used to determine the equilibrium distribution of a queueing network. Product-form distributions are introduced in Section 4. In addition, Section 4 briefly reviews the literature on product-form queueing networks. Finally, Section 5 reviews the assumptions used throughout this monograph.

1 Basic results on Markov chains

Consider a stochastic process {N(t), t ∈ T } taking values in a countable N state space S. Applications will usually assume that S ⊂ IN0 and that t represents time. Then a staten ¯ = (n1, . . . , nN ) ∈ S is a vector with components ni ∈ IN0, i = 1,...,N. For a discrete-time stochastic process T is the set of integers: T = IN0, whereas for a continuous-time stochastic process T is the positive real line: T = IR+ ∪ {0} = [0, ∞). A vector N n¯ ∈ IR is called non-negative if ni ≥ 0, i = 1,...,N, and positive if

11 12 Preliminaries it is non-negative and non-null. In this monograph emphasis will be on continuous-time stochastic processes. Therefore, in the sequel all results are given for continuous-time stochastic processes only. A stochastic process is a stationary process if (N(t1),N(t2),...,N(tn)) has the same distribution as (N(t1 + τ),N(t2 + τ),...,N(tn + τ)) for all n ∈ IN, t1, t2, . . . , tn ∈ T , τ ∈ T . The stochastic process {N(t), t ∈ T } is a Markov process if for every k ≥ 1, 0 ≤ t1 < ··· < tk < tk+1, and any n¯1,..., n¯k+1 in S, the joint distribution of (N(t1),...,N(tk+1)) is such that

P {N(tk+1) =n ¯k+1|N(t1) =n ¯1,...,N(tk) =n ¯k}

= P {N(tk+1) =n ¯k+1|N(tk) =n ¯k} , (1.1) whenever the conditioning event (N(t1) =n ¯1,...,N(tk) =n ¯k) has positive probability. For a Markov process the state at a given time contains all information about the past evolution necessary to probabilistically predict the future evolution of the Markov process. A Markov process is time-homogeneous if the conditional probability P {N(t + s) =n ¯0|N(t) =n ¯} is independent of t for all s > 0,n, ¯ n¯0 ∈ S. For a time-homogeneous Markov process the transition probability from staten ¯ to staten ¯0 in time s is deﬁned as

P (¯n, n¯0; s) = P {N(t + s) =n ¯0|N(t) =n ¯} , s > 0.

The transition matrix P (t) = (P (¯n, n¯0; t), n,¯ n¯0 ∈ S) has non-negative entries (1.2a) and row sums equal to one (1.2b). The Markov property (1.1) immediately implies that the transition probabilities satisfy the Chapman- Kolmogorov equations (1.2c). In addition, assume that the transition matrix is standard (1.2d). For alln, ¯ n¯0 ∈ S, s, t ∈ T , a standard transition matrix satisﬁes:

P (¯n, n¯0; t) ≥ 0; (1.2a)

X P (¯n, n¯0; t) = 1; (1.2b) n¯0∈S P (¯n, n¯00; t + s) = X P (¯n, n¯0; t)P (¯n0, n¯00; s); (1.2c) n¯0∈S lim P (¯n, n¯0; t) = δ(¯n, n¯0). (1.2d) t↓0 2.1 Basic results on Markov chains 13

δ(¯n, n¯0) is the Kronecker-delta, δ(¯n, n¯0) = 1 ifn ¯ =n ¯0 and δ(¯n, n¯0) = 0 ifn ¯ 6=n ¯0. For a standard transition matrix it is natural to extend the definition of P (¯n, n¯0; ·) to [0, ∞) by setting P (¯n, n¯0; 0) = δ(¯n, n¯0). Then for alln, ¯ n¯0 the transition probabilities are uniformly continuous on [0, ∞). Furthermore, each P (¯n, n¯0; t) is either identically zero for all t > 0 or never zero for t > 0 (Lévy’sdichotomy [16, Theorem II.5.2]). For a standard transition matrix the transition rate from staten ¯ to staten ¯0 can be defined as P (¯n, n¯0; h) − δ(¯n, n¯0) q(¯n, n¯0) = lim . h↓0 h For alln, ¯ n¯0 ∈ S this limit exists. Forn ¯ 6=n ¯0 this limit is finite, whereas forn ¯ =n ¯0 the limit may be infinite. For practical systems the limit for n¯ =n ¯0 is finite too. A Markov process is called a continuous-time Markov chain if for alln, ¯ n¯0 ∈ S the limit exists and is finite (1.3a). Furthermore, in the sequel it is assumed that the rate matrix Q = (q(¯n, n¯0), n,¯ n¯0 ∈ S) is stable (1.3b), and in addition it is assumed that the rate matrix is conservative (1.3c). Then for alln, ¯ n¯0 the rate matrix satisfies:

0 ≤ q(¯n, n¯0) < ∞, n¯0 6=n ¯; (1.3a)

0 ≤ q(¯n) = −q(¯n, n¯) < ∞; (1.3b)

X q(¯n, n¯0) = 0. (1.3c) n¯0∈S If the rate matrix is stable, the deﬁnition of the transition rates implies that the transition probabilities can be expressed in the transition rates. This gives forn, ¯ n¯0 ∈ S

P (¯n, n¯0; h) = δ(¯n, n¯0) + q(¯n, n¯0)h + o(h) for h ↓ 0, (1.4) where o(h) denotes a function g(h) with the property that g(h)/h → 0 as h → 0. For small positive values of h, forn ¯0 6=n ¯, q(¯n, n¯0)h may be interpreted as the conditional probability given that the process is in state n¯ at time t that it makes a transition to staten ¯0 during (t, t+h). Similarly, if q(¯n) is ﬁnite, q(¯n)h is the conditional probability given that the process is in staten ¯ at time t that it leaves this state during (t, t + h). As a consequence, q(¯n, n¯0) can be interpreted as the rate at which transitions occur, i.e. as transition rates. To elaborate on the transition rates and on the role of stability, consider the conditional probability that the process 14 Preliminaries remains inn ¯ during (s, s + h) if the process is inn ¯ at time s. This conditional probability is given by [16, Theorem II.5.5]

P {N(τ) =n, ¯ s < τ < s + h|N(s) =n ¯} = e−q(¯n)h, h > 0.

The exit-time from staten ¯, ρ(¯n), deﬁned as

ρ(¯n) = inf {t : t > 0,N(t + s) 6=n ¯} given that the process is in staten ¯ at time s, has a negative-exponential distribution with mean q(¯n)−1. It can be shown that for every initial state N(0) =n ¯, {N(t), t ∈ T } is a pure-jump process, which means that the process jumps from state to state and remains in each state a strictly positive sojourn-time with probability 1. For the Markovian case, the process remains at staten ¯ for a negative-exponentially distributed sojourn-time with mean q(¯n)−1. In addition, conditional on the process 0 q(¯n,n¯0) departing from staten ¯ it jumps to staten ¯ with probability q(¯n) . This second interpretation is sometimes used as a definition of a continuous- time Markov chain and is used to construct such processes. From the Chapman-Kolmogorov equations two systems of differential equations for the transition probabilities can be obtained. To this end, observe that for a standard transition matrix every element P (¯n, n¯0; ·) has a continuous derivative in (0, ∞), which is continuous at zero if the rate matrix is stable [16, Theorem II.12.8]. Conditioning on the first jump of the Markov chain in (0, t] yields the so-called Kolmogorov backward equations (1.5a), whereas conditioning on the last jump in (0, t] gives the Kolmogorov forward equations (1.5b). The validity of this method is discussed below. These equations read forn, ¯ n¯0 ∈ S, t ≥ 0,

dP (¯n, n¯00; t) = X q(¯n, n¯0)P (¯n0, n¯00; t), (1.5a) dt n¯0∈S dP (¯n, n¯00; t) = X P (¯n, n¯0; t)q(¯n0, n¯00). (1.5b) dt n¯0∈S

If the rate matrix is stable, then starting from the initial state N(0) =n ¯, a first jump of the Markov chain exists for t > 0. As a consequence conditioning on this first jump is allowed. In contrast, the last jump of the Markov chain in (0, t] is not properly defined. It may be that also for a stable rate matrix jumps will accumulate in such a way that {N(t)} will make infinitely many jumps in finite time. In this case {N(t)} is not 2.1 Basic results on Markov chains 15 properly defined from the rate matrix for all t > 0. For an example of this behaviour see [83, Remark 2.4.5]. An additional assumption on the rate matrix guaranteeing the existence of a last jump in (0, t] is regularity. A pure-jump Markov chain is regular if for every initial state N(0) = n¯ the number of transitions in finite time is finite with probability 1. For a regular Markov chain the last jump before t is well-defined and conditioning on the last jump before t is allowed. Thus if a pure-jump Markov chain is stable and regular, then for all t > 0 the evolution of the process is uniquely determined by the transition rates, that is specification of the transition rates is sufficient to completely characterize the process. Regularity is a property of the rate matrix. It can be shown [67] that the rate matrix is regular if and only for some λ > 0 the system of equations X q(¯n, n¯0)x(¯n0) = λx(¯n), n¯ ∈ S, n¯0∈S has no bounded solution other than {x(¯n) = 0, n¯ ∈ S}. This characterization of regularity may be difficult to apply in practical situations. A simple sufficient condition ensuring regularity of a Markov chain is the existence of a uniform finite upper bound on q(¯n). If such a bound exists, i.e. if a constant C exists such that q(¯n) ≤ C < ∞, for alln ¯ ∈ S, then the Markov chain is said to be uniformizable and the forward and backward equations have the formal solution

∞ (Qt)n P (t) = eQt = X , t ≥ 0. n=0 n! Uniformizability can be too strong a condition for practical applications, for example if inﬁnite-server queues are part of the queueing network that is modelled. More general suﬃcient conditions can, for example, be found in [88, Section 4-3]. A detailed discussion of regularity is beyond the scope of this monograph. The behaviour of irregular Markov chains is, for example, discussed in [62], [63]. The following theorem summarizes the results on regularity and the forward and backward equations stated above.

Theorem 1.1 [16, Theorem II.18.3] For a conservative, stable, regular, continuous-time Markov chain the forward equations (1.5b) and the backward equations (1.5a) have the same unique solution {P (¯n, n¯0; t), n,¯ n¯0 ∈ S, t ≥ 0}. Moreover, this unique solution is the transition matrix of the Markov chain. 16 Preliminaries

In particular, Theorem 1.1 states that either the forward or the backward equations can be solved to find the transition matrix. Usually the forward equations are easier to use in practical cases. For any initial distribution {P0(¯n), n¯ ∈ S} defined as X P0(¯n) = P {N(0) =n ¯} , P0(¯n) = 1, n¯∈S the time-dependent or transient distribution {P (¯n, t), n¯ ∈ S} defined to be P (¯n, t) = P {N(t) =n ¯} , X P (¯n, t) = 1, n¯∈S can be obtained from the forward equations (1.5b). Premultiplication of the forward equations with P0 gives for the transient distribution the following version of the Kolmogorov forward equations forn ¯ ∈ S, t ≥ 0,  dP (¯n, t) X  = {P (¯n0, t)q(¯n0, n¯) − P (¯n, t)q(¯n, n¯0)},  dt n¯06=¯n (1.6)    P (¯n, 0) = P0(¯n). From the interpretation of the transition rates obtained from (1.4), for n¯ 6=n ¯0, the probability that the process jumps fromn ¯ ton ¯0 in the interval (t, t+h) is P (¯n, t)q(¯n, n¯0)h+o(h). Therefore, P (¯n, t)q(¯n, n¯0) may be called the probability flux or probability flow from staten ¯ to staten ¯0. The forward equations now express that the rate of change of the probability mass of dP (¯n,t) staten ¯, dt , equals the net probability flux from S \{n¯} ton ¯. Thus the Kolmogorov forward equations express an intuitively obvious relation for the time-dependent probabilities. A similar straightforward interpretation of the backward equations is not available. The remaining part of this section considers the stationary or equilibrium behaviour of Markov chains. Throughout it will be assumed that the rate matrix is stable, conservative and regular. Although these assumptions are not necessary for a large part of the discussion below, the discussion particularizes to stable, conservative, regular Markov chains when the stationary distribution is related to the invariant distribution (the equilibrium distribution of the Kolmogorov forward equations). When the assumptions are crucial to the theory they will be explicitly repeated. If P (t) is a transition matrix then the following limit exists for all n,¯ n¯0 ∈ S lim P (¯n, n¯0; t) = π(¯n, n¯0). t→∞ 2.1 Basic results on Markov chains 17

The matrix Π = (π(¯n, n¯0), n,¯ n¯0 ∈ S) satisﬁes for alln, ¯ n¯00 ∈ S, s > 0,

π(¯n, n¯00) = X π(¯n, n¯0)P (¯n0, n¯00; s) n¯0∈S = X P (¯n, n¯0; s)π(¯n0, n¯00) = X π(¯n, n¯0)π(¯n0, n¯00). n¯0∈S n¯0∈S Furthermore, π(¯n, n¯0) ≥ 0 for alln, ¯ n¯0 ∈ S, and if π(¯n, n¯) 6= 0 then P 0 n¯0∈S π(¯n, n¯ ) = 1. Therefore, Π characterizes the stationary behaviour, but cannot be immediately associated with the stationary distribution. For Π to be the stationary distribution additional assumptions guaranteeing that π(¯n, n¯) 6= 0 must be made. A staten ¯ is absorbing if the process cannot leave staten ¯, that is P (¯n, n¯0; t) = 1 for all t. For a non-absorbing staten ¯ the recurrence-time τ(¯n) is deﬁned as

τ(¯n) = inf{t : t > ρ(¯n),N(t) =n ¯ if N(0) =n ¯}, where ρ(¯n) is the exit-time from staten ¯. τ(¯n) is the time it takes the process to return to staten ¯ if it starts atn ¯. A staten ¯ is called recurrent if recurrence ton ¯ is almost certain, i.e. if P{τ(¯n) < ∞} = 1. Otherwise it is transient. A recurrent state is positive-recurrent if E{τ(¯n)} < ∞, that is if the expected return-time to staten ¯ is ﬁnite. Otherwise it is null-recurrent. Staten ¯ communicates with staten ¯0 if passage fromn ¯ ton ¯0 is possible, that is if P (¯n, n¯0; t) > 0 for some positive t. Two states intercommunicate if each one communicates with the other. A set V ⊂ S is closed if the process cannot leave V , so that q(¯n, n¯0) = 0 forn ¯ ∈ V, n¯0 ∈ S \ V . A set V ⊂ S is irreducible if it is closed and all its states intercommunicate. Two irreducible sets are disjoint, so the state space S can be decomposed into disjoint irreducible sets V1,V2,..., and a non-irreducible set W . For the equilibrium behaviour of {N(t)} the process may be analysed at each irreducible set separately. Therefore, without loss of generality, for equilibrium analysis the Markov chain may be assumed irreducible at S, that is S is an irreducible set. In this case all statesn ¯ ∈ S are of the same type (transient, null-recurrent, positive-recurrent). A measure m = (m(¯n), n¯ ∈ S) such that 0 ≤ m(¯n) < ∞ for alln ¯ ∈ S and m(¯n) > 0 for somen ¯ ∈ S is called a stationary measure if for all n¯ ∈ S, t ≥ 0, m(¯n) = X m(¯n0)P (¯n0, n¯; t), (1.7) n¯0∈S 18 Preliminaries and is called an invariant measure if for alln ¯ ∈ S,

X {m(¯n)q(¯n, n¯0) − m(¯n0)q(¯n0, n¯)} = 0. (1.8) n¯06=¯n

The relation between stationary and invariant measures is rather complicated [63]. Based on regularity of the rate matrix a simple relation between these measures can be obtained. If the Markov chain is irreducible and positive-recurrent at S then there exists a unique (up to a multiplicative factor) stationary measure m which is positive (m(¯n) > 0 for alln ¯ ∈ S). From this result, for a regular and irreducible pure-jump process, if a ﬁ- P nite mass ( n¯∈S m(¯n) < ∞) invariant measure m exists then the process is positive-recurrent and m is the unique stationary measure. In the literature, an irreducible positive-recurrent process with stationary measure having ﬁnite mass is called ergodic. Ergodicity is an important property of a process as it guarantees the existence of a unique stationary distribution π, that is a stationary measure summing to unity. Furthermore, if {N(t)} is ergodic and π is the stationary distribution then P (¯n, n¯0; t) → π(¯n0)(t → ∞) for alln, ¯ n¯0 ∈ S, or equivalently, P (¯n, t) → π(¯n)(t → ∞) for alln ¯ ∈ S for any initial distribution P0. As a consequence π may be called equilibrium distribution. Moreover, if {N(t)} is ergodic then for any f : S → [0, ∞) such that P n¯∈S f(¯n)π(¯n) < ∞, with probability 1

1 Z T X lim f(N(t))dt = Eπ {f(N)} ≡ f(¯n)π(¯n). T →∞ T 0 n¯∈S

In particular, for f(N(t)) = 1{N(t) =n ¯}, the indicator of the event {N(t) =n ¯}, i.e. 1{A} = 1 if A occurs and 0 otherwise,

1 Z T lim 1{N(t) =n ¯}dt = π(¯n). T →∞ T 0 Thus π(¯n) is the long-run fraction of time the process spends in staten ¯. Conditions for the process to be ergodic can be found, for example, in [17], [34]. The following theorem summarizes the relation between stationary, invariant and equilibrium distributions, and describes the method to determine the stationary or equilibrium distribution used throughout this monograph. 2.2 Queueing network model 19

Theorem 1.2 (Equilibrium distribution) Let {N(t), t ≥ 0} be a conservative, stable, regular, irreducible continuous-time Markov chain.

(i) If a positive ﬁnite mass invariant measure m exists then the Markov chain is positive-recurrent (ergodic). In this case π = (π(¯n), n¯ ∈ P −1 S) deﬁned as π(¯n) = m(¯n)[ n¯∈S m(¯n)] , n¯ ∈ S, is the unique stationary distribution and π is the equilibrium distribution, i.e. for all n,¯ n¯0 ∈ S, lim P (¯n, n¯0; t) = π(¯n), t→∞ independent of the initial distribution.

(ii) If a positive ﬁnite mass invariant measure does not exist then for all n,¯ n¯0 ∈ S, lim P (¯n, n¯0; t) = 0. t→∞

The main result of Theorem 1.2 is that the stationary or equilibrium distribution can be obtained as the unique probability solution to (1.8). The equations (1.8) for m = π, the invariant distribution, can be obtained from the Kolmogorov forward equations. To this end note that the transition matrix P (t) is the unique solution to (1.5b). Furthermore, for a dP (¯n,n¯0;t) 0 standard transition matrix dt → 0 (t → ∞) for alln, ¯ n¯ ∈ S. Thus for t → ∞ (1.5b) reduces to (1.8). Similar to the interpretation of (1.6), the equations (1.8) for m = π can immediately be interpreted as balancing the flow of probability mass on S. To this end π(¯n) is interpreted as the probability mass at staten ¯ and q(¯n, n¯0) as the conductance of the direct path fromn ¯ ton ¯0. Then π(¯n)q(¯n, n¯0) is the flux of probability mass along the path fromn ¯ ton ¯0 and (1.8) states that the total flow of probability mass leavingn ¯ is balanced by the flow of probability mass enteringn ¯. Therefore, (1.8) is usually referred to as global balance equations.

2 Queueing network model

Consider a queueing network consisting of N stations labelled 1, 2 ...,N. In this queueing network a single type of customer routes among the stations to receive a desired service. Assume that only one customer is allowed to move between the stations of the queueing network at a time. At station i a customer requires an amount of service that is negative- exponentially distributed with rate µi, i = 1,...,N, that is if the required 20 Preliminaries amount of service is worked oﬀ at rate 1 then the service-time at station i is negative-exponentially distributed with rate µi, i = 1,...,N. The rate at which service is provided by the service-centres (stations) depends on the occupation-numbers of the stations (the number of customers re- siding at the stations of the queueing network), for example due to a number of servers shared by stations. If ni, i = 1,...,N, customers are present at the stations, service at station i is provided at rate fi(¯n), where n¯ = (n1, . . . , nN ). As a consequence customers depart from station i in staten ¯ at rate (departure rate)

µi(¯n) = µifi(¯n), i.e., in staten ¯ customers are expelled from station i at rate µi(¯n). In addition, customers can enter the queueing network according to a Pois- son process with state-dependent arrival rate µ0(¯n). Typically, fi(¯n) is a function of nj for only a few stations j that have a direct relation to station i. Simple examples illustrating possible choices for fi(¯n) are:

• fi(¯n) = min {ni, si} station i behaves as a queue with si servers and inﬁnite waiting-room: each arriving customer is assigned a server as long as a free server is available, otherwise the customer waits until a server becomes available (completes service on a previous customer);

• fi(¯n) = ni station i is an inﬁnite-server queue: each arriving customer is assigned a separate server;

hP i−1 • fi(¯n) = j∈Ci nj a single server is shared by the stations in the group Ci, this server equally distributes its service-capacity over the stations in Ci;

• fi(¯n) = ni1 {nj < Mj} station i is an inﬁnite-server queue, but service at station i is stopped when the number of customers at station j reaches some bound Mj.

1 • fi(¯n) = ni1 {nj ≤ Nj} + 2 ni1 {nj > Nj} station i is an inﬁnite- server queue, but to help avoid congestion at station j, when the number of customers at station j exceeds Nj the service-speed at station i is reduced. Routing of customers is allowed to be state-dependent to help avoiding congestion and to allow blocking of transitions due to capacity constraints. Upon departure in staten ¯ from station i a customer routes to station j 2.2 Queueing network model 21 according to the routing probabilities p(i, j;n ¯), j = 1,...,N, or leaves the queueing network with probability p(i, 0;n ¯). Customers entering the queueing network route to station j according to the routing probabilities p(0, j;n ¯), and are rejected with probability p(0, 0;n ¯). Typical examples illustrating possible state-dependencies for the routing probabilities are:

• p(i, j;n ¯) = pij1{nj < Nj} capacity constraint at station j: customers route among the stations of the queueing network according to the routing probabilities pij, but customers routing to station j if nj = Nj are blocked;

• p(i, j;n ¯) = pij1{ni > Ni} minimal workload at i: customers cannot leave station i if ni = Ni. At the stations customer positions are not taken into account. As a consequence, the occupation-numbersn ¯ = (n1, . . . , nN ) give a full description of the state of the queueing network. The total time the queueing network remains in staten ¯ is negative-exponentially distributed with rate PN µ(¯n) = i=0 µi(¯n) and the probability that in staten ¯ a customer leaves station i is, from an elementary property of the negative-exponential distribution, given by µi(¯n)/µ(¯n), i = 1,...,N. The evolution of the queueing network described above can be represented by a continuous-time Markov chain {N(t), t ≥ 0} that records the number of customers at the stations of the queueing network. The state space S of the Markov chain is the set of non-negative integer-valued vectorsn ¯ = (n1, . . . , nN ), where ni denotes the number of customers at N station i. Thus S = IN0 . Transitions of {N(t)} occur simultaneously with the movement of a customer in the queueing network. More speciﬁc, if {N(t)} is in staten ¯ and a customer routes from station i to station j in the queueing network then the next state of {N(t)} isn ¯ − ei + ej, where ei is the i-th unit vector, i = 1,...,N, and e0 = 0. Here station 0 is introduced to represent the outside. If a customer routes from station i to station 0 then this customer leaves the queueing network and if a customer routes from station 0 to station i then this customers enters the queueing network at station i, i = 1,...,N. The transition rates of the Markov 0 N chain are forn, ¯ n¯ ∈ S = IN0  µ (¯n)p(i, j;n ¯), ifn ¯0 =n ¯ − e + e ,  i i j q(¯n, n¯0) = i, j = 0,...,N, (2.1)  0, otherwise. 22 Preliminaries

Formally, a continuous-time Markov chain representing a queueing network is a conservative continuous-time Markov chain {N(t), t ≥ 0} with N 0 0 state space S = IN0 and transition rates Q = (q(¯n, n¯ ), n,¯ n¯ ∈ S) such that 0 0 q(¯n, n¯ ) = 0 unlessn ¯ =n ¯ − ei + ej, i, j = 0,...,N. (2.2) Since q(¯n, n¯0) < ∞ for alln ¯ 6=n ¯0 (cf. (1.3a)) and the number of possible transitionsn ¯ → n¯0 is finite by (2.2), the rate matrix Q is stable. If the queueing network is closed, that is arrivals to the queueing network and departures from the queueing network are not allowed (q(¯n, n¯ − ei) = q(¯n, n¯ + ej) = 0 for alln ¯, i, j = 1,...,N) then S consists of mutually N PN exclusive closed sets Vr, r ∈ IN0 such that Vr = {n¯ ∈ IN0 : j=1 nj = r} 0 0 0 and q(¯n, n¯ ) = 0 ifn ¯ ∈ Vr,n ¯ ∈ Vr0 , r 6= r . For each r ∈ IN0 the Markov chain at Vr represents a closed queueing network in which r customers are present. For fixed r, the Markov chain at Vr is uniformizable since q(¯n) < max {q(¯n) :n ¯ ∈ Vr} < ∞. Therefore, the Markov chain representing a closed queueing network with a fixed number of customers is regular. A queueing network is called open if arrivals to the queueing network and departures from the queueing network are possible. In this case the state space can be infinite, and the number of customers in the queueing network is not constant. If the number of customers at all stations of the queueing network is constrained not to exceed a fixed (station-dependent) number or if the total number of customers allowed at the queueing network is bounded from above, then the set of admissible states is finite and therefore the Markov chain representing the queueing network is regular. Otherwise, regularity of the Markov chain is not guaranteed. Note that the Markov chain is regular if for every initial state N(0) =n ¯, it will visit a recurrent state in a finite number of jumps with probability 1. In particular, if for a closed set V ⊂ S the Markov chain is irreducible and positive recurrent at V then it is regular at V . Throughout this monograph it is assumed that the Markov chain representing a queueing network is stable and regular. The transition rates of the stable, regular, continuous-time Markov chain representing the queueing network are given by (2.1). The specific form used in most chapters is   ψ(¯n − ei) 0  µi p(i, j;n ¯ − ei), ifn ¯ =n ¯ − ei + ej, 0  φ(¯n) q(¯n, n¯ ) = i, j = 0,...,N, (2.3)   0, otherwise, 2.2 Queueing network model 23

where 1/µi ∈ (0, ∞), average service-requirement at station i, N φ : IN0 → (0, ∞), service function, N ψ : IN0 → [0, ∞), service function, N p(i, j; ·) : IN0 → [0, ∞), routing function. Note that it must be that φ(¯n) > 0 for alln ¯ ∈ S. This is essential to the theory since q(¯n) < ∞ is required. For ﬁxed values of the transition rates q(¯n, n¯0), n,¯ n¯0 ∈ S, there remains some freedom in choosing the functions φ, ψ and p. To this end, ﬁrst note that the parameter µi can N be incorporated into the service functions by setting form ¯ ∈ IN0

PN m PN m j=1 j j=1 j ψ(m ¯ ) := ψ(m ¯ )µi , φ(m ¯ ) := φ(m ¯ )µi . Second, the service functions ψ and φ are determined up to a common constant. This constant can, for example, be fixed by setting φ(0) = 1. ψ(¯n−ei) Third, also for fixed φ the service rate µi(¯n) = µi φ(¯n) and the routing function p(i, j;n ¯ − ei) are determined up to a common constant. This constant can be fixed by assuming that the routing function is a probability distribution, however, this is not assumed at this moment. Fourth, note that the argument of the routing function in (2.3) isn ¯ − ei and notn ¯ as given in (2.1). The argumentn ¯ − ei expresses that a transition n¯ → n¯ − ei + ej virtually passes through staten ¯ − ei as is extensively discussed in Chapter 5. In Chapter 3 canonical forms for the transition rates are introduced. These forms remove all freedom in the parameters of the transition rates. Chapter 5 uses the form (2.3) with µi incorporated into ψ and φ, whereas, for notational convenience, Chapter 6 uses the form (2.3) with p(i, j;n ¯ − ei) replaced by p(i, j;n ¯). Note that these forms are equivalent since the routing function is a function of i as well. The transition rates can be generalized to also include simultane- 0 ous jumps of customers. In this case, for allg ¯ = (g1, . . . , gN ),g ¯ = 0 0 N (g1, . . . , gN ) ∈ IN0 a transition in which gi, i = 1,...,N, customers leave 0 the stations of the queueing network simultaneously and gi, i = 1,...,N, customers enter the stations of the queueing network simultaneously is allowed. Therefore assumption (2.2) must be ignored, and stability of the rate matrix must explicitly be assumed. The transition rates can be obtained from (2.3) and read for a transition in whichg ¯ customers leave,g ¯0 customers enter andm ¯ customers remain at the stations of the queueing network: ψ(m ¯ ) q(¯g, g¯0;m ¯ ) = p(¯g, g¯0;m ¯ ). (2.4) φ(m ¯ +g ¯) 24 Preliminaries

Note that for given groupsg, ¯ g¯0 the decomposition ofn ¯,n ¯0 inn ¯ =m ¯ +g ¯, n¯0 =m ¯ +g ¯0 is unique, but that a transitionn ¯ → n¯0 might also occur due to other groups. The total transition rate from staten ¯ to staten ¯0 is then given by q(¯n, n¯0) = X q(¯g, g¯0;m ¯ ). (2.5) {g,¯ g¯0,m¯ :m ¯ +¯g=¯n, m¯ +¯g0=¯n0} In Chapter 9 the transition rates as speciﬁed by (2.4) and (2.5) are used. In Chapter 8, for notational convenience, a diﬀerent form is used for the transition rates. This form can be obtained from (2.4) by setting

p(¯g, g¯0;m ¯ ) = λ(¯g, g¯0)ψ(¯g, g¯0;m ¯ ), ψ(m ¯ ) = 1.

ψ(¯n−ei) The service-rate µi(¯n) = φ(¯n) introduced in (2.3) is an immediate generalization of the well-known form with ψ = φ as used in the literature on product-form queueing networks (cf. [55], [83], [87] and the references ψ(¯n−g¯) therein). The form µ(¯g;n ¯) = φ(¯n) for the service-rate is introduced in [40] for batch routing queueing networks and is the most general form still implying a product-form equilibrium distribution. The transition rates (2.3) with µi incorporated into ψ and φ, and (2.4) can be interpreted probabilistically. To this end, deﬁne

0 0 p(¯g, g¯ ;m ¯ ) X 0 pb(¯g, g¯ ;m ¯ ) = , p(¯g;m ¯ ) = p(¯g, g¯ ;m ¯ ). p(¯g;m ¯ ) 0 N g¯ ∈IN0

0 N 0 Forg, ¯ g¯ , m¯ ∈ IN0 , such thatm ¯ +g, ¯ m¯ +g ¯ ∈ S, (2.4) can be written X ψ(m ¯ )p(¯g;m ¯ ) {g,¯ m¯ :m ¯ +¯g=¯n} ψ(m ¯ )p(¯g;m ¯ ) q(¯g, g¯0;m ¯ ) = p(¯g, g¯0;m ¯ ). φ(¯n) X ψ(m ¯ )p(¯g;m ¯ ) b {g,¯ m¯ :m ¯ +¯g=¯n} (2.6) As the total rate out of staten ¯ is given by ψ(m ¯ ) X q(¯n, n¯0) = X p(¯g, g¯0;m ¯ ) n¯06=¯n {g,¯ g¯0, m¯ :m ¯ +¯g=¯n, m¯ +¯g06=¯n} φ(¯n) P ψ(m ¯ )p(¯g;m ¯ ) = {g,¯ m¯ :m ¯ +¯g=¯n} , φ(¯n) the ﬁrst term in (2.6) can be interpreted as the total state-dependent service-rate in staten ¯. The second term may now be interpreted as the 2.3 Balance equations 25 probability that a groupg ¯ of customers is released and pb is the state- dependent routing probability. The form (2.6) immediately shows that the transition rates used in [8], [9], [40], [41], [76] are equivalent. Both the service-rate and the routing function are discussed in more detail in Section 4 below.

3 Balance equations

This section discusses various types of local balance or partial balance appearing in the literature and looks ahead to diﬀerent types of local balance introduced in this monograph. As is discussed in Section 1 for a stable, regular, irreducible continuous-time Markov chain representing a queueing network the equilibrium distribution can be obtained as the unique solution to the global balance equations (3.1) also called full balance equations or total balance equations as they express balance of the total probability ﬂow in and out of each staten ¯. A probability distribution π = (π(¯n), n¯ ∈ S) is called an equilibrium distribution if it is an invariant measure summing to unity, i.e. a set of non-negative numbers m = (m(¯n), n¯ ∈ S) satisfying for alln ¯ ∈ S

X {m(¯n)q(¯n, n¯0) − m(¯n0)q(¯n0, n¯)} = 0. (3.1) n¯06=¯n

Solving the global balance equations is often very hard. Almost all solutions available at the moment satisfy more stringent balance relations. The most stringent balance relation is transition balance. A Markov chain satisﬁes transition balance if for alln, ¯ n¯0 ∈ S the transition rate from n¯ ton ¯0 equals the transition rate fromn ¯0 ton ¯, that is for alln, ¯ n¯0 ∈ S

q(¯n, n¯0) = q(¯n0, n¯).

If a Markov chain satisfies transition balance then m(¯n) = 1 for alln ¯ ∈ S satisfies the global balance equations (3.1). The equilibrium distribution π exists only if S is finite. A less restrictive form of balance often encountered in physical systems is detailed balance [50], [55], [87]. A Markov chain is reversible if a measure m exists that satisfies the detailed balance equations (3.2), that is if a measure m exists such that for alln, ¯ n¯0 ∈ S

m(¯n)q(¯n, n¯0) − m(¯n0)q(¯n0, n¯) = 0. (3.2) 26 Preliminaries

Reversibility of Markov chains is related to time-reversal. A stochastic process {N(t), − ∞ < t < ∞} is reversible if (N(t1),N(t2),...,N(tn)) has the same distribution as (N(τ − t1),N(τ − t2),...,N(τ − tn)) for all n ∈ IN, t1, t2, . . . , tn ∈ IR, τ ∈ IR. If a stochastic process is reversible and the direction of time is reversed then the probabilistic behaviour of the process remains the same. This interpretation of reversibility is not explored in this monograph. Reversibility, characterized by (3.2), is an important equilibrium concept. If m satisfies the detailed balance equations then m is an invariant measure as can immediately be seen by summing (3.2) over alln ¯0 ∈ S. If m is a probability distribution then the detailed balance equations state that the probability flow between each pair of states is balanced. Reversibility of a Markov chain can be characterized directly from the transition rates. Kolmogorov’s criteria provide this characterization and give a useful insight into the nature of reversible Markov chains. Kolmogorov’s criterium 3.1 A Markov chain is reversible if and only if its transition rates satisfy for all r ∈ IN and any finite sequence of states n¯1, n¯2,..., n¯r ∈ S, n¯r =n ¯1

r−1 r−1 Y Y q(¯ni, n¯i+1) = q(¯nr−i, n¯r−i−1). (3.3) i=1 i=1 (3.3) states that any ﬁnite path in the state space which returns to its initial pointn ¯1 has the same probability whether this path is traced in one direction or the other. This implies that a reversible Markov chain shows no net circulation in the state space. In practice relations (3.3) usually have to be established for a small number of simple paths only. (3.3) for general paths then follows by decomposition of these paths into simple paths. The following result is an immediate consequence of Kolmogorov’s criterion (3.3) and provides a construction method for the equilibrium distribution. Kolmogorov’s criterium 3.2 For an irreducible, positive-recurrent, reversible Markov chain the equilibrium distribution π is given by q(¯n , n¯ )q(¯n , n¯ ) q(¯n , n¯ ) π(¯n) = π(¯n0) 1 2 2 3 ··· r−1 r , (3.4) q(¯n2, n¯1)q(¯n3, n¯2) q(¯nr, n¯r−1) 0 for arbitrary n¯ ∈ S for all r ∈ IN and any path n¯1, n¯2,..., n¯r ∈ S such 0 that n¯1 =n ¯ , n¯r =n ¯ for which the denominator is positive. 2.3 Balance equations 27

Most Markov chains representing a queueing network are not reversible in the sense that a positive measure satisfying (3.2) cannot be obtained. For example, a queueing network in which customers may route from station 1 to station 2 but cannot return from station 2 to station 1 is not reversible since for alln ¯ ∈ S it must be the case that q(¯n, n¯ − e2 + e1) = 0 whereas q(¯n, n¯ − e1 + e2) > 0 for somen ¯ ∈ S. Nevertheless, the concept of reversibility will also play a major role in the analysis of non-reversible processes as presented in this monograph. For general queueing networks balance equations less restrictive than detailed balance are required. To this end, deﬁne forn ¯ ∈ S a collection of mutually exclusive sets {Ak(¯n), k ∈ I(¯n)} (I(¯n) ⊂ IN), such that S k∈I(¯n) Ak(¯n) = S. A Markov chain is partially balanced over {Ak(¯n), k ∈ I(¯n)} if a measure m exists such that for alln ¯ ∈ S, k ∈ I(¯n) X {m(¯n)q(¯n, n¯0) − m(¯n0)q(¯n0, n¯)} = 0. (3.5) 0 n¯ ∈Ak(¯n) Summation of (3.5) over k ∈ I(¯n) shows that a measure m satisfying (3.5) is an invariant measure. Various forms of partial balance discussed above are used for Markov chains representing queueing networks. These forms explore the special transition structure of queueing networks. For a Markov chain representing a single-changes queueing network, that is a Markov chain with transition rates satisfying (2.2), the global balance equations read forn ¯ ∈ S

N N X X {m(¯n)q(¯n, n¯ − ei + ej) − m(¯n − ei + ej)q(¯n − ei + ej, n¯)} = 0. i=0 j=0 (3.6) 0 0 For Ai(¯n) = {n¯ ∈ S :n ¯ =n ¯ −ei +ej, j = 0, . . . , N, j 6= i}, i = 0,...,N, SN AN+1 = S \ i=0 Ai(¯n), (3.5) reads forn ¯ ∈ S, i = 0,...,N,

N X {m(¯n)q(¯n, n¯ − ei + ej) − m(¯n − ei + ej)q(¯n − ei + ej, n¯)} = 0, (3.7) j=0

0 0 0 and q(¯n, n¯ ) = q(¯n , n¯) = 0 forn ¯ ∈ AN+1(¯n). Clearly, if a measure m satisfies (3.7) then it is an invariant measure since summing (3.7) over all i, i = 1,...,N + 1, immediately shows that m satisfies the global balance equations. If m is a probability distribution then (3.7) states that for all n¯ ∈ S the probability flow out ofn ¯ due to customers leaving station i is balanced by the probability flow inton ¯ due to customers entering station i. This is a natural equilibrium concept for single changes queueing 28 Preliminaries networks. The equation (3.7) is often encountered in queueing networks and expresses station balance or local balance [14], [84]. 0 0 From the global balance equations, for Aj(¯n) = {n¯ ∈ S :n ¯ = SN n¯ − ei + ej, i = 0, . . . , N, i 6= j}, j = 0,...,N, AN+1 = S \ j=0 Aj(¯n), the following form of partial balance can be obtained too, forn ¯ ∈ S, j = 0,...,N,

N X {m(¯n)q(¯n, n¯ − ei + ej) − m(¯n − ei + ej)q(¯n − ei + ej, n¯)} = 0. i=0 This form of partial balance is introduced in Chapter 5 to obtain the equilibrium distribution of the dual process. For single changes queueing networks a transitionn ¯ → n¯0 is uniquely determined by the customer leaving and entering a station. If batch routing is allowed too, then a transitionn ¯ → n¯0 may occur due to diﬀerent groupsg, ¯ g¯0 routing among the stations (cf. (2.4), (2.5)). As a consequence the transition rate fromn ¯ ton ¯0 possesses more structure than the transition rates for a single-changes queueing network. This structure is explored in Chapters 8 and 9. Chapter 8 introduces strong reversibility. A Markov chain representing a batch routing queueing network is strongly reversible if a measure m 0 N 0 0 exists such that for allg, ¯ g¯ , m¯ ∈ IN0 ,n ¯ =m ¯ +g ¯,n ¯ =m ¯ +g ¯ ∈ S

m(m ¯ +g ¯)q(¯g, g¯0;m ¯ ) = m(m ¯ +g ¯0)q(¯g0, g¯;m ¯ ). (3.8)

Strong detailed balance (3.8) states that for each possible transitionm ¯ + g¯ ↔ m¯ +g ¯0 due tog ¯ ↔ g¯0 the probability ﬂows balance. Summation of (3.8) over {g,¯ g¯0, m¯ :n ¯ =m ¯ +g, ¯ n¯0 =m ¯ +g ¯0} yields the detailed balance equations. If the transition between each pair of statesn ¯,n ¯0 is unique, or, if single-changes are allowed only (3.8) reduces to the detailed balance equations (3.2). Thus (3.8) expresses a form of balance stronger than detailed balance but weaker than transition balance. Chapter 9 further explores the structure of the transition rates. To this end note that local balance (3.7) can be obtained from detailed balance by summation over j. Similarly, summation overg ¯0 in (3.8) gives the group- local-balance equations. A Markov chain satisﬁes group-local-balance if a measure m exists such that for allg, ¯ g¯0, m¯ such thatm ¯ +g ¯ ∈ S

X {m(m ¯ +g ¯)q(¯g, g¯0;m ¯ ) − m(m ¯ +g ¯0)q(¯g0, g¯;m ¯ )} = 0. (3.9) g¯06=¯g 2.4 Product-form distributions; literature 29

Summation of (3.9) over {g,¯ m¯ :n ¯ =m ¯ +g ¯} implies that a measure m satisfying the group-local-balance equations is an invariant measure. If m is a probability distribution then the group-local-balance equations state that for alln ¯ ∈ S the probability flow out ofn ¯ due to gi, i = 1,...,N, customers leaving the stations is balanced by the probability flow inton ¯ due to gi, i = 1,...,N, customers entering the stations, that is for a group g¯ the probability flow into and out of staten ¯ balances. If single changes are allowed only group-local-balance reduces to local balance. The specific form (3.9) is not related to and cannot be concluded from any notion of partial balance for single changes queueing networks (3.5).

4 Product-form distributions; literature

The most encountered equilibrium distribution for queueing networks is the so-called product-form distribution. Except for a few very special in- stances, the product-form distribution is the only equilibrium distribution that is analytically tractable. The product-form distribution is introduced by J. R. Jackson in 1957. Jackson’s product-form equilibrium distribution states that the queue-lengths at the various stations are independent: the equilibrium distribution of the queueing network is the product of the marginal distributions at the stations of the queueing network. Over the last decades the product-form distribution has evolved from the product of marginal distributions to a closed-form distribution that can be obtained from the transition rates. This section gives a definition of the product- form aimed for in this monograph and gives an overview of the evolution of product-forms and the underlying local balance structure. Here emphasis will be on the literature considering occupation-numbers only. Queueing theory originated from teletraffic and is initialized in 1909 by A. K. Erlang [11]. In the early years stations in isolation were extensively studied. Both the equilibrium queue-length distribution and the time-dependent queue-length distribution for various queues, such as M|M|s|N-queues and M|M|∞-queues, were derived (for example, cf.[68]). In 1957 queueing networks were introduced by J. R. Jackson. The model and results obtained in [48] have been the basis for the development of queueing network theory. The queueing network studied in [48] consists of N stations, the i-th station containing si servers and infinite waiting-room. Customers arrive from outside the system in station i according to a Poisson process with rate λi. The service-time at station i is negative-exponentially distributed 30 Preliminaries

−1 with mean µi and service is in order of arrival. Once served at station i a customer instantly routes to station j with probability pij or leaves the PN network with probability pi0 = 1 − j=1 pij. The total arrival rate ci at station i is the sum of the external arrival rate λi and the internal N arrival rates cjpji from station j, j = 1,...,N, that is {ci}i=1 satisﬁes the traﬃc equations:

N X ci = λi + cjpji, i = 1,...,N. (4.1) j=1

If ci < siµi for i = 1,...,N, then the joint queue-length process has an equilibrium distribution given by

N Y N π(¯n) = πi(ni), n¯ ∈ IN0 , (4.2) i=1 where πi is the equilibrium distribution for station i with arrival rate ci, i.e. the equilibrium distribution of an M|M|si-queue with arrival rate ci. N Thus, if a positive solution {ci}i=1 such that ci < siµi for i = 1,...,N, exists to the traffic equations (4.1) then the queue-length processes at the stations are independent. The equilibrium distribution (4.2) is called product-form equilibrium distribution since it factorizes into the marginal distributions of the stations and thus expresses independence of the queue- length processes. In [48] this result is proven by substitution of the equilibrium distribution and transition rates into the global balance equations (3.1) and was not surprising in view of the Output Theorem proven in [12]: in steady state the output process of an M|M|s-queue is a Poisson process with the same rate as the arrival process to the queue. In [49] the queueing network discussed in [48] is generalized to include state-dependent service-rates µi(ni) at station i if ni customers are present at station i, where µi(ni) > 0 except µi(0) = 0. In addition, the Poisson arrival process is state-dependent with rate λ(K) if a total of K customers is present at the queueing network, λ(K) > 0 for K ≤ K+ and λ(K) = 0 for K > K+ for some K+ ∈ IN ∪ {∞}. This allows blocking of arrivals to the queueing network. Furthermore, [49] includes triggered arrivals: when the total number of customers falls below a specified limit, say K−, a new customer is immediately injected into the queueing network, and service deletions: a customer is instantaneously ejected from a station if the total number of customers at that station exceeds a specified limit, say Ni at station i. Thus, [49] includes elementary blocking results. By insertion 2.4 Product-form distributions; literature 31 into the global balance equations [49] obtains a product-form equilibrium distribution:

PN nj −1 j=1 N ni N Y Y Y ci − X + π(¯n) = B λ(r) 1{K ≤ nj ≤ K , ni ≤ Ni ∀i}, r=0 i=1 k=1 µi(k) j=1 (4.3) N where B is a normalizing constant and {ci}i=1 a solution to the traﬃc equations (4.1). In [38] a model very similar to that of [48] is studied. While the model in [48] is open, the model in [38] is closed: a ﬁxed number of customers, M, is present in the network. Although [38] studies a closed network of M|M|s-queues, the proof of the product-form equilibrium distribution also contains the more general case with service-rate µi(ni) at station i. By substitution into the global balance equations the equilibrium distribution is shown to be of product-form:

N ni N Y Y ci X π(¯n) = B 1{ nj = M}, (4.4) i=1 k=1 µi(k) j=1

N where B is a normalizing constant and {ci}i=1 is a solution to the traﬃc equations (4.1) for λi ≡ 0, i = 1,...,N. [84], [85] consider closed respectively open queueing networks with transition rates q(¯n, n¯ − ei + ej) = µi(ni)pij, identical to the model of [38] and [49] respectively. In addition, [84], [85] introduce the key-notion of local balance (3.7) to prove that the proposed distribution is the equilibrium distribution of the queueing network. The product-form results described above characterize the early product-form distributions. Due to restrictions on the state space, the equilibrium distribution does not factorize into the marginal distributions in the sense of (4.2), expressing independence of the queue-length processes at the stations, but the invariant measure for the joint queue-length pro- Qni cess factorizes into the invariant measures mi(ni) = k=1{ci/µi(k)} for the queue-length process at station i with arrival rate ci determined by the traﬃc equations (4.1). The stochastic dependence between the queue- length processes at the stations is included into the normalizing constant only. The equilibrium distribution is of classical product-form if

N PN Y π(¯n) = B d j=1 nj mi(ni)1 {n¯ ∈ S} , (4.5) i=1 32 Preliminaries

where mi(ni) is the invariant measure for the queue-length process at station i with arrival rate ci determined by the traffic equations (4.1), B a normalizing constant characterizing stochastic dependence between the queue-length processes, and d(·) is a blocking function completely determined by the state-dependent arrival process (cf. (4.3)). Except for blocking of arrivals the classical product-form queueing networks reviewed above do not contain blocking of transitions. [56] introduces blocking of transitions for queueing networks with reversible routing, N that is a solution {ci}i=1 exists to cipij = cjpji, i, j = 0,...,N. To this end [56] notes “that reversibility is preserved if the chain is modified by choosing two statesn ¯ andn ¯0 and setting q(¯n, n¯0) = q(¯n0, n¯) = 0.” In particular, upper bounds on the number of customers at the stations are included. In addition, for reversible processes [56] generalizes the state-dependence in the transition rates to q(¯n, n¯−ei +ej) = pijφi(ni)ψj(nj), that is explicit dependence on the state of the station a customer is routing to is included in the transition rates. For transition rates of this form [56] obtains a classical ni Qni product-form distribution, where mi(ni) = ci r=1 {ψi(r − 1)/φi(r)}. In the references discussed so far, the state of the queueing network is determined by the occupation numbers only, that is internal structure (ordering of customers and service-disciplines) is ignored. Although internal structure is ignored in this monograph too, queueing networks with multiple types or classes of customers and ordering of customers at the stations have played an important role in the development of queueing theory. Therefore, some references on queueing networks with multiple customer classes and ordering at the stations are reviewed below. In [52], [54] queueing networks in which customers may be of different types are studied. The routing of customers is type-dependent, but the service-requirements at the stations cannot depend on the customer-class. Within each station the customers are ordered, thus station j contains customers in positions 1, 2, . . . , nj. Station j operates as follows: – each customer requires a random amount of service with mean 1;

– a single server supplies service-eﬀort at rate φj(nj);

– a proportion γj(l, nj) of this eﬀort is directed to the customer in position l, l = 1, . . . , nj; – when a new customer arrives at station j it moves to position l with probability δj(l, nj + 1), l = 1, . . . , nj + 1. 2.4 Product-form distributions; literature 33

By varying φj, γj, and δj various queueing-disciplines can be modelled. Note that the (φ, γ, δ)-protocol does not allow the service-contribution to depend on the type of a customer. If the required amount of service is negative-exponentially distributed then [52] shows that the equilibrium distribution is of classical product-form, where, of course, the customer types and positions at the stations are included in the equilibrium distribution. For symmetrical stations, that is γj(l, nj) = δj(l, nj), [54] shows that the equilibrium queue-length distribution is insensitive to the distribution of the service-requirement. In the famous paper [4], the so-called BCMP-networks are introduced. In [4] a queueing network with multiple customer-types and ordering at the stations is discussed. Four service-protocols are considered: FCFS (first-come-first-served), where all customers have the same negative-exponentially distributed service-requirement; PS (processor-sharing); IS (infinite-server); and LCFS (last-come-first-served). The last three protocols (PS, IS, LCFS) are symmetrical and a classical product-form equilibrium distribution, which is insensitive to the distribution of the service- requirement at PS, IS, and LCFS stations, is concluded. In contrast with [54] the service-requirement at the symmetrical stations is allowed to be type-dependent. An important observation from the result of [4] is that the equilibrium distribution is of insensitive product-form

N N !ni Y Y 1 π(¯n) = B m˜ i(ni) , (4.6) i=1 i=1 µi

−1 where µi is the average service-requirement at station i and m˜ i(ni) is determined by the rate at which service is provided and the routing of customers through the network, i.e. π(¯n) depends on the service-requirement only through its mean which “implies” insensitivity. Therefore, also for queueing networks with negative-exponentially distributed service- requirements an equilibrium distribution of the form (4.6) is desired. [60] considers a closed queueing network with multiple types of customers and exponential service-times with blocking. As multiple types of customers are involved a capacity constraint at a station cannot be secured by simply increasing the service-rate to infinity, i.e. set µi(Ni + 1) = ∞ as in [49], since this may expel from the station a customer of the wrong class, that is a customer of a class different from the class of the arriving customer is rejected from the station. Therefore, in [60] customers are blocked upon entrance into the station. To this end, [60] considers two 34 Preliminaries models. In the first model a customer chooses the next station according to the routing probabilities and returns to the station it departed from if the chosen station is not available. This type of blocking is called communication-blocking. Under the assumption of reversible routing the equilibrium distribution is shown to be of classical product-form (4.5), similar to the result obtained in [56]. In the second model a customer chooses the next station according to the routing probabilities, but if the chosen station, say j, is not available the customer selects a new station according to the routing probabilities pjk, that is as if it jumps over station j. This type of blocking is called jump-over blocking [24]. For the second model reversible routing need not be assumed and the equilibrium distribution is shown to be of classical product-form. From the product-form results discussed above, a factorizing form (classical product-form) for the equilibrium distribution seems to be required to obtain a closed-form equilibrium distribution. [53] shows that the notion of a “service-potential” is more significant than that of product- form. [53] considers a Markov chain with transition rates

φ(¯n − ei) ψ(¯n) q(¯n, n¯ − ei + ej) = pij , i, j = 0,...,N, (4.7) φ(¯n) ψ(¯n + ej) N and shows that, for ψ ≡ 1 and {ck}k=1 a solution to the traﬃc equations (4.1), the Markov chain has an equilibrium distribution

N Y nk π(¯n) = Bφ(¯n) ck . (4.8) k=1

In addition, if cipij = cjpji, i, j = 0,...,N, then for arbitrary ψ N φ(¯n) Y nk π(¯n) = B ck . ψ(¯n) k=1 Thus a closed-form equilibrium distribution which is not of factorizing form is obtained. Similar to the classical product-form, the terms appearing in (4.8) can immediately be obtained form the transition rates and the traﬃc equations. The equilibrium distribution (4.8) is called product-form too, since it is a direct generalization of the classical product-form. The average service-requirement at station i can explicitly be visualized in the transition rates (4.7) and the equilibrium distribution (4.8). To this end, assume that the transition rates have the form φ(¯n − e ) q(¯n, n¯ − e + e ) = µ i p , (4.9) i j i φ(¯n) ij 2.4 Product-form distributions; literature 35

φ(¯n−ei) where 1/µi is the average service-requirement at station i and φ(¯n) represents the rate at which service is worked oﬀ. Then the equilibrium distribution is N c !nk π(¯n) = Bφ(¯n) Y k . (4.10) k=1 µk For queueing networks with negative-exponential service-rate the forms (4.8) and (4.10) are equivalent. For insensitivity analysis, however, the form (4.10) is more valuable than the form (4.8) as it immediately allows the introduction of supplementary variables to prove that the equilibrium distribution is insensitive to the service-time distribution. In [15] it is shown that for the class of queueing networks with state-independent Pois- son arrival process, the equilibrium distribution is the product-form (4.10) if and only if the transition rates have the form (4.9). This is a very powerful result as it not only characterizes the equilibrium distribution, but also characterizes the form of the transition rates. The product-form results discussed above can be summarized as follows. Consider a queueing network with state-independent Poisson arrival process and state-independent routing (pij). Then every two of the following set of three properties implies the third:

N (p1) {ck}k=1 is a solution to the traﬃc equations (4.1); (p2) the transition rates have the form (4.9);

(p3) the equilibrium distribution is given by (4.10).

Blocking results for product-form queueing networks have been obtained by various authors (for example, cf. [33], [44], [58], [60]). A general framework incorporating state-dependent routing into the theory on product-form queueing networks is introduced in [45], [46], [47]. These references introduce a general formalism for local balance in stochastic queueing networks, related to the local balance and insensitivity results obtained in [70], [71], [72], to provide a general formalism for stochastic queueing networks making explicit use of the special structure of the transition rates of queueing networks to obtain an insensitive closed-form equilibrium distribution if blocking of transitions is allowed. It is still the most general framework for standard continuous-time queueing networks. However, often it is not at all clear how to apply these theoretical results to practical systems, or how to ﬁt a practical system into this formalism. In [46] job-local-balance and the adjoint process, a process related to the 36 Preliminaries local solution of the job-local-balance equations, are introduced. In the setting of this monograph job-local-balance is equivalent to station balance (3.7). In [46] these equations are written form ¯ , i such thatm ¯ +ei ∈ S

N N X X π(m ¯ + ei) q(m ¯ + ei, m¯ + ej) = π(m ¯ + ej)q(m ¯ + ej, m¯ + ei). (4.11) j=0 j=0

For ﬁxedm ¯ a local solution {π(i;m ¯ ), i = 1,...,N} of (4.11) can often be obtained. From these local solutions the adjoint process with transition ratesq ¯ such that

 q¯(m ¯ + e , m¯ + e ) π(j;m ¯ )  i j  = , q¯(m ¯ + ej, m¯ + ei) π(i;m ¯ )   q¯(m ¯ + ei, m¯ + ej) > 0 ⇔ q(m ¯ + ei, m¯ + ej) > 0, can be defined. [46] shows that a solution {π(¯n), n¯ ∈ S} exists to the job- local-balance equations if and only if the adjoint process is reversible with equilibrium distribution {π(¯n), n¯ ∈ S}. Then, from Kolmogorov’s criterion 3.2 for the adjoint process, the equilibrium distribution for the original process can be obtained. Based on these results blocking of customers is analysed under the communications-blocking-protocol [26]. In [47] it is shown that a queueing network is insensitive with respect to the service- time distributions if and only if the equilibrium distribution satisfies job- local-balance (4.11). In [86] it is shown that partial balance is equivalent to insensitivity. To this end, subsets Ai of the state space are considered. It is shown that if these subsets are such that there is no single transition of positive intensity in which more than one Ai is vacated or more than one Ai is entered. Then the equilibrium distribution is insensitive to the nominal sojourn-time in the Ai if and only if the Markov process shows partial balance in all Ai. At first glance, the result of [86] is more general than the result of [47]. However, in [5] it is shown that the insensitivity results of [47] and [86] are equivalent. These references show that the a general notion of partial or local balance is more important for queueing networks than the notion of product-form. Local or partial balance as expressed by (3.7) and (4.11) has been studied extensively by numerous authors. Various examples for the routing characteristics have been provided. These examples involve blocking of transitions or grouping of stations into clusters of stations [27]. From these 2.4 Product-form distributions; literature 37 results , in [40] it is observed that the service-function can be generalized ψ(¯n−ei) to fi(¯n) = φ(¯n) , that is φ(¯n − ei) in (4.7) or (4.9) can be replaced by an arbitrary function ψ 6= φ. In the local balance equations (3.7) the function ψ(¯n−ei) appears both in q(¯n, n¯−ei +ej) and q(¯n−ei +ej, n¯), j = 0,...,N. Therefore, ψ(·) does not affect the equilibrium distribution of the queueing network. In [76] a general queueing network is discussed. This queueing network unifies various known results on product-form queueing networks. At first glance, the model discussed in this reference seems to further generalize the service-function. However, a close examination of the results presented in this reference shows that the service-function used in this ψ(¯n−ei) reference can be written in the form fi(¯n) = φ(¯n) . Extensions of product-form queueing networks discussed above have been reported in the literature. Most notable in this respect are discrete- time queueing networks and batch routing queueing networks. In these models the assumption that one customer can route among the stations in each transition is relaxed. Product-form equilibrium distributions for discrete-time queueing networks are obtained in [65], [81]. [65] studies a queueing networks with Bernoulli service, that is in each time-slot a cus- gi tomer is served with probability pi at station i (and with probability pi a group consisting of gi customers is served at station i). In [81] discrete- time queueing networks with a simple form of state-dependent service- probabilities is discussed. In this reference the probability of serving gi out of ni customers present at station i has the form ci(ni)αi(ni)αi(ni − 1) ··· αi(ni − gi + 1)/gi!, where ci(ni) is a normalizing constant. Note that these service-probabilities basically express that the gi customers are served one by one, as would be the case in a continuous-time queueing network. Product-form equilibrium distributions for batch routing queueing networks are obtained in [40] and [41]. In these references the service- ψ(¯n−g¯) function for serving a groupg ¯ in staten ¯ has the form f(¯g, n¯) = φ(¯n) . In the model considered in [40] customers route independently, that is a customer in a batchg ¯ routes from station i to station j with probability pij independent of the other routing customers and independent of the state of the queueing network. In [41] this model is generalized to also include batch-dependent routing probabilities. These transition rates are generalized in [8] and [9] (Chapters 8 and 9 of this monograph), where state-dependent routing probabilities are included too. 38 Preliminaries

5 Review of assumptions

For convenience, this section summarizes the basic assumptions made on the queueing networks discussed in this monograph. These assumptions will be given for batch routing queueing networks. For single changes queueing networks, in A6 simply replace the vectorsg ¯ by unit-vectors.

A1 A single type of customers is present in the queueing network.

A2 At the stations of the queueing network customer-positions are not taken into account.

A3 The service-time is negative-exponentially distributed.

A4 The queueing network can be represented by a stable, regular, conservative, continuous-time Markov chain. As a consequence, the Markov chain is uniquely determined by the transition rates, and the invariant measure can be obtained as a solution to the Kolmogorov forward equations.

A5 The Markov chain representing a queueing network is irreducible and positive-recurrent. As a consequence, the Markov chain possesses a unique equilibrium distribution.

A6 The transition rate for a transition in whichg ¯ customers are served and depart from the stations, andg ¯0 customers enter the stations, whilem ¯ customers remain at the stations, has the form

ψ(m ¯ ) q(¯g, g¯0;m ¯ ) = p(¯g, g¯0;m ¯ ). φ(m ¯ +g ¯) Chapter 3

Transient product-form distributions

1 Introduction

As is illustrated in Section 2.4, there is a very large literature which studies product-forms in queueing networks. However, most of these authors consider equilibrium behaviour only. Although the time-dependent or transient behaviour of single stations has been studied extensively (see, for example [17], [68]), there seem to be very few analytical results for the transient behaviour of queueing networks. To achieve transient results in queueing systems, the time-dependent probability distribution is usually expanded in terms of the eigenvalues and eigenvectors of the transition matrix. For networks of queues, however, this method seems to be of lit- tle value. In this case the number of eigenvectors and eigenvalues would be impossible to handle. Therefore, it seems of great interest to give closed-form (e.g. product-form) expressions for the transient probability distribution. The transient behaviour of open networks of M/G/∞-queues with homogeneous Poisson input is studied in [56]. In [32] these results are extended to a tandem network of M/G/∞-queues with non-homogeneous Poisson input, that is the arrival process is a Poisson process with time- dependent arrival rate. Reference [39] studies the transient behaviour of closed queueing networks of M/G/∞-queues and open networks of M/G/∞-queues with non-homogeneous Poisson input. It is shown in these references that a network of M/G/∞-queues has a transient product-

39 40 Transient behaviour: product-form distributions form distribution. In particular it is shown that

N Y nk P (¯n, t) = B(t)φ(¯n) ck(t) , (1.1) k=1 wheren ¯ = (n1, . . . , nN ) denotes the state with nk customers at station k, QN nk φ(¯n) = 1/( k=1 µk nk! ), ck(t) is a function associated with station k, µk is the service-rate at station k, and B(t) is a time-dependent normalizing constant. The method used in these references to derive the transient product-form distribution (1.1) is an independence argument. It relies on the fact that the sample paths for individual customers in a queueing network consisting of M/G/∞-queues only are independent and therefore the transient probability distribution has product-form. It is assumed in these references that the network is initially empty in the open network case and that all customers start at the same station in the closed network case. Up to now, however, networks consisting only of M/G/∞-queues seem to be the only networks for which the transient distribution is studied in great detail. Based on the experience with equilibrium product-form distributions (cf. Section 2.4) one would expect that networks in which non M/G/∞- queues appear may also possess a product-form transient distribution. However, this chapter shows that for a queueing network to have a transient distribution of the form (1.1) all stations must be M/M/∞-queues. The form (1.1) will be referred to as “transient product-form distribution”. The justiﬁcation for this is that such a distribution serves as an analogue to the well-known product-form equilibrium distributions which exist in many networks of queues. Note that a distribution of the form (1.1) is not necessary for a network of queues to have independent marginal distributions. An obvious example is a network of independent M/M/1-queues (or any other type of queues) which clearly possesses a transient distribution with independent marginal distributions, but which does not have a transient distribution of the form (1.1).

2 Model and canonical representation

Consider a continuous-time queueing network consisting of N stations, labelled 1, 2,...,N. A state of the queueing network is a vectorn ¯ = (n1, . . . , nN ), where ni denotes the number of customers at station i, i = 1,...,N. Assume that the queueing network can be represented by a 3.2 Model and canonical representation 41

N stable, regular, continuous-time Markov chain with state space S ⊂ IN0 . The transition rate from staten ¯ to staten ¯0 is denoted by q(¯n, n¯0) and is assumed to be of the form ψ(¯n − e ) q(¯n, n¯ − e + e ) = p i , i, j = 0,...,N, n¯ ∈ S, (2.1) i j ij φ(¯n) where pij denotes the probability that a customer leaving station i routes to station j, i, j = 1,...,N, p0j denotes the probability that an entering customer routes to station j, j = 1,...,N, and pi0 denotes the probability that a customer leaving station i leaves the network, i = 1,...,N, and ψ(·), φ(·) are arbitrary functions such that φ(·) > 0, and φ(¯n) = 0 if ni < 0 for some i, ψ(·) ≥ 0, and ψ(m ¯ ) = 0 if mi < 0 for some i. It is known (cf. [9], [41], [76]) that for transition rates of the form (2.1) a necessary and sufficient condition for the process to have product-form equilibrium distribution N Y nk π(¯n) = Bφ(¯n) ck , n¯ ∈ S, (2.2) k=1 N is that the coefficients {ck}k=1 satisfy the traffic equations N X cj = cipij + p0j, j = 1,...,N. (2.3) i=1 The form (2.3) for the traffic equations varies slightly from that given by many authors, in which the traffic equations have p0j replaced by λp0j with λ the parameter of the Poisson arrival process. The fact that p0j appears here is a consequence of the fact that all transitions (including arrivals) are written in the form (2.1). This point is discussed in more detail in Section 2.1. Also, note that the function ψ(·) does not appear in the equilibrium distribution. This is a direct consequence of the form of the global balance equations. This chapter shows that a necessary and sufficient condition for the time-dependent distribution P (¯n, t) of a queueing network with homogeneous transition rates given in (2.1) and initial conditions

N Y nk P (¯n, 0) = B0φ(¯n) ξk k=1 to be of product-form, defined by N Y nk P (¯n, t) = B(t)φ(¯n) ck(t) , n¯ ∈ S, (2.4) k=1 42 Transient behaviour: product-form distributions is that the network consists of M/M/∞-queues only and that the coeffi- N cients {ck(t)}k=1 satisfy the time-dependent traffic equations

N 1 dck(t) X = {ci(t)pik − ck(t)pki} + p0k − ckpk0, k = 1,...,N, (2.5) µk dt i=1 with initial conditions ck(0) = ξk, where for a closed queueing network p0k = pk0 = 0, k = 1,...,N, in the N relations above, and where {µk}k=1 is a set of coefficients defined in terms of the transition rates between specified states. For a given physical model there is freedom in setting the parameters in the description above. This can be immediately seen by observing (2.1), where both φ(·) and ψ(·) can be multiplied by an arbitrary constant without affecting the transition rates, or by observing (2.5) for a closed queueing network where the ck(t) are determined up to an arbitrary constant only. Therefore, since the present chapter aims for necessary and sufficient conditions for the time- dependent distribution to be of product-form, the model is transformed such that all freedom in the coefficients appearing in the model is removed. This gives slightly different formulations for the open and closed model and also changes the form of the time-dependent version of the traffic equations for the open queueing network. Therefore, in the discussion below, the open and closed network are considered separately.

2.1 Canonical form for the open network This chapter considers an open queueing network with state-independent Poisson input with parameter λ. The transition rate q(¯n, n¯ + ej) for an arriving customer is then given by

q(¯n, n¯ + ej) = λp0j, j = 1,...,N. Equation (2.1) then implies for alln ¯ ∈ S

ψ(¯n) = λφ(¯n).

The following lemma summarizes all changes this implies for P (¯n, t). This lemma provides a unique expression for the transition rates. This form is obtained in [15] and [41] too. However, since it also determines the parameters of the time-dependent queueing network and gives a transformation for the coeﬃcients ck(t) it is inserted here. 3.2 Model and canonical representation 43

Lemma 2.1 (Canonical form - open network) Let P (¯n, t) be a time- dependent probability distribution on S = {n¯ :n ¯ = (n1, . . . , nN ), ni ≥ 0, i = 1,...,N} of the form

N Y nk P (¯n, t) = Bb(t)φb(¯n) cbk(t) , n¯ ∈ S, t ≥ 0, k=1 and let f(¯n, i) be a function on the set S × {0,...,N} of the form

ψb(¯n − e ) f(¯n, i) = i , n¯ ∈ S, i = 0,...,N, (2.6) φb(¯n) satisfying f(¯n, 0) = λ, n¯ ∈ S. (2.7) Then by deﬁning

B(t) = Bb(t)φb(0), (2.8) " N #−1 φb(¯n) Y φ(¯n) = λnk , (2.9) φb(0) k=1 ck(t) = λcbk(t), (2.10) P (¯n, t) and f(¯n, i) can be written in the canonical form

N Y nk P (¯n, t) = B(t)φ(¯n) ck(t) , n¯ ∈ S, t ≥ 0, (2.11) k=1 φ(¯n − e ) f(¯n, i) = i , n¯ ∈ S, i = 1,...,N, (2.12) φ(¯n) where φ(0) = 1. (2.13) Proof Comparison of (2.6) and (2.7) gives for alln ¯ ∈ S

ψb(¯n) = λφb(¯n). (2.14) Substitution of (2.8), (2.9) and (2.10) into the right-hand side of (2.11) gives " #−1 N φb(¯n) N N Y nk Y nk Y nk B(t)φ(¯n) ck(t) = Bb(t)φb(0) λ (λcbk(t)) k=1 φb(0) k=1 k=1 N Y nk = Bb(t)φb(¯n) cbk(t) . k=1 44 Transient behaviour: product-form distributions

Substitution of (2.14) and (2.9) into the right-hand side of (2.6) immediately gives the right-hand side of (2.12) and from (2.9) it immediately follows that φ(0) = 1. 2

Inserting the transformation (2.10) in the time-dependent traﬃc equations (2.5) gives

N 1 dck(t) X = {ci(t)pik − ck(t)pki}+λp0k −ckpk0, k = 1,...,N, (2.15) µk dt i=1 with initial conditions ck(0) = λξk. This is the time-dependent version for the equilibrium traffic equations used by many authors when the arrival process is a state-independent Poisson process. For the original process with transition rates (2.1) and equilibrium distribution (2.2) this is not the appropriate form. However, for the case of the transformed model, the traffic equations (2.15) are equivalent to (2.5). The condition (2.7) imposed on f(·) implies that the arrival process to the open queueing network is a state-independent Poisson process with parameter λ. With the transformation defined in the lemma above the transition rates for the open network are given by φ(¯n − e ) q(¯n, n¯ − e + e ) = p i , i = 1, . . . , N, j = 0,...,N, i j ij φ(¯n)

q(¯n, n¯ + ej) = λp0j, j = 1,...,N, which is in agreement with the transition rates obtained in [15] and [41].

2.2 Canonical form for the closed network In the closed network case there are also some degrees of freedom in the parameters of the process. The following lemma gives a transformation which removes all freedom from the model. Lemma 2.2 (Canonical form) Let P (¯n, t) be a time-dependent proba- PN bility distribution on S = {n¯ :n ¯ = (n1, . . . , nN ), j=1 nj = M, ni ≥ 0, i = 1,...,N}, with M > 0, of the form

N Y nk P (¯n, t) = Bb(t)φb(¯n) cbk(t) , n¯ ∈ S, t ≥ 0, k=1 3.2 Model and canonical representation 45 and let f(¯n, i) be a function on the set S × {1,...,N} of the form

ψb(¯n − e ) f(¯n, i) = i , n¯ ∈ S, i = 1,...,N. φb(¯n)

Let µi, for i = 1,...,N, be deﬁned by f(Me , i) µ = i , i M then by deﬁning

" N #−1 X cbi(t) ck(t) = cbk(t) , k = 1, . . . , N, t ≥ 0, (2.16) i=1 µi " N #M M X cbi(t) B(t) = Bb(t)φb(Me1)M!(µ1) , t ≥ 0, (2.17) i=1 µi φb(¯n) φ(¯n) = , (2.18) M φb(Me1)M!(µ1) ψb(¯n) ψ(¯n) = , (2.19) M−1 ψb((M − 1)e1)(M − 1)! (µ1) P (¯n, t) and f(¯n, i) can be written in the canonical form

N Y nk P (¯n, t) = B(t)φ(¯n) ck(t) , n¯ ∈ S, t ≥ 0, (2.20) k=1 ψ(¯n − e ) f(¯n, i) = i , n¯ ∈ S, i = 1,...,N, (2.21) φ(¯n) where h M i φ(Me1) = 1/ M!(µ1) , (2.22)

h M−1i ψ((M − 1)e1) = 1/ (M − 1)! (µ1) , (2.23) and for all t ≥ 0 N c (t) X i = 1. (2.24) i=1 µi Proof For fixed t, P (¯n, t) is a probability distribution over S. Therefore 0 ≤ cbk(t) < ∞ for all k for all t, and for all t there exists a k such that cbk(t) > 0. By assumption, φ(·) > 0, which implies that N c (t) 0 < X bi < ∞, i=1 µi 46 Transient behaviour: product-form distributions and that (2.16) is well-defined. The form (2.21) follows from the definition of f(¯n, i), (2.18), (2.19) and the fact that f(Me1, 1) = µ1M. The normalization (2.22) and (2.23) follows from (2.18) and (2.19). Insertion of (2.16), (2.17) and (2.18) into PN the right-hand side of (2.20) gives by using that i=1 ni = M

M N " N c (t)# Y nk M X bi B(t)φ(¯n) ck(t) = Bb(t)φb(Me1)M!(µ1) k=1 i=1 µi φb(¯n) × M φb(Me1)M!(µ1)  n −1 " N # N N c (t)! k Y nk Y X bi × cbk(t)   k=1 k=1 i=1 µi N Y nk = Bb(t)φb(¯n) cbk(t) . k=1 Inserting (2.16) into the left-hand side of (2.24) gives

−1 N c (t) N c (t) " N c (t)# X i = X bi X bi = 1, i=1 µi i=1 µi i=1 µi which completes the proof. 2

Substitution of the transformation (2.16) into the time-dependent traf- ﬁc equations (2.5) gives the following form

N 1 dck(t) X = ci(t)pik − ck(t), k = 1,...,N. µk dt i=1

3 Suﬃcient conditions

This section considers suﬃcient conditions for queueing networks to have transient product-form. Theorem 3.1 below deals explicitly with the queueing network discussed in Section 2 and covers both the open and the closed network case. From the canonical representation, the transition rates for the queueing network are given by ψ(¯n − e ) q(¯n, n¯ − e + e ) = p i , i, j = 1,...,N, (3.1a) i j ij φ(¯n) 3.3 Suﬃcient conditions 47

q(¯n, n¯ + ej) = λp0j, j = 1,...,N, (3.1b) φ(¯n − e ) q(¯n, n¯ − e ) = p i , i = 1,...,N, (3.1c) i i0 φ(¯n) where ψ(¯n − ei) ≡ φ(¯n − ei) if the network is open. Thus q(¯n, n¯ − ei + ej) can be written as pijf(¯n, i), where the f(¯n, i) are given for the open model by (2.12), (2.13) and for the closed model by (2.21), (2.22), (2.23). Note N that it is assumed that φ(¯n) > 0 for alln ¯ ∈ IN0 and φ(¯n) = 0 if ni < 0 for some i. A probability distribution P (¯n, t) is said to be the time-dependent probability distribution for the process with transition rates q(¯n, n¯0) if P (¯n, t) satisﬁes the Kolmogorov forward equations (cf. Theorem 2.1.1)

dP (¯n, t) = X {P (¯n0, t)q(¯n0, n¯) − P (¯n, t)q(¯n, n¯0)} , (3.2) dt n¯0 with initial conditions P (¯n, 0) = P0(¯n). The product-form transient distribution (2.4) satisﬁes the Kolmogorov forward equations if and only if the following relation holds for alln ¯ ∈ S and for all t > 0

N N 1 dB(t) X ni dci(t) X ψ(¯n − ei) + = {cj(t)pji − ci(t)pij} B(t) dt i=1 ci(t) dt i,j=1 ci(t)φ(¯n) N X + {ci(t)pi0 − λp0i} (3.3) i=1 N X φ(¯n − ei) + {λp0i − ci(t)pi0} , i=1 ci(t)φ(¯n) as can easily be verified by substituting (2.4) and (3.1) into (3.2). If the network is open then ψ ≡ φ and if the network is closed the first term in the right-hand side contributes only. The derivation of (3.3) from (3.2) implicitly uses that P (¯n, t) > 0 for alln ¯ ∈ S for all t > 0. This can be justified as follows. If P (¯n, t) = 0 for some t, then, from Lévy’sdichotomy (cf. [16, Theorem II.5.2]), P (¯n, t) = 0 for all t > 0. This implies that for some k0 for which nk0 > 0, ck0 (t) = 0 for all t > 0. However, if this is the case, then P (¯n, t) = 0 for alln ¯ such that nk0 > 0. Therefore, for all t > 0 station k0 cannot contain any jobs 48 Transient behaviour: product-form distributions and may be removed from the network. Hence it may be assumed that P (¯n, t) > 0. The following theorem gives a sufficient condition for the transient probability distribution to be of product-form. For a network of M/G/∞- queues, with non-homogeneous Poisson input in the open network case, this product-form was obtained in [39]. However, the formulation used in that paper is rather different from the formulation used in the theorem below. The transient form for the traffic equations does not appear in [39]. Furthermore, the initial conditions in the theorem below are more general than the initial conditions in [39]. Theorem 3.1 (Sufficient conditions) Consider a queueing network with transition rates given by (3.1) and product-form initial distribution

N Y nk P0(¯n) = B0φ(¯n) ξk , n¯ ∈ S, k=1 where φ(·) satisﬁes

ψ(¯n − e ) k = µ n , k = 1,...,N, (3.4) φ(¯n) k k and ψ = φ if the network is open. Then

N Y nk P (¯n, t) = B(t)φ(¯n) ck(t) , (3.5) k=1

N where the {ck(t)}k=1 satisfy the traﬃc equations

N 1 dck(t) X = ci(t)pik + λp0k − ck(t), k = 1,...,N, (3.6) µk dt i=1 with initial conditions ck(0) = ξk, (3.7) and the normalization constant B(t) satisﬁes

N 1 dB(t) X = {ci(t)pi0 − λp0i} , (3.8) B(t) dt i=1 with initial conditions B(0) = B0. (3.9) 3.3 Suﬃcient conditions 49

Proof Insertion of (3.4) into (3.3) gives

N 1 dB(t) X − {ci(t)pi0 − λp0i} B(t) dt i=1 N  N  X niµi X 1 dci(t) = {cj(t)pji − ci(t)pij} + λp0i − ci(t)pi0 − , i=1 ci(t) j=1 µi dt  which is obviously satisﬁed if (3.6) and (3.8) hold. The initial conditions (3.7) and (3.9) give P (¯n, 0) = P0(¯n), which implies that (3.5) gives the transient probability distribution of the process with transition rates satisfying (3.4). 2

The above theorem applies to both the open and the closed network case. In the closed case interpret pk0 = p0k = 0 in (3.6) and (3.8). From (3.4), (together with (2.22) and (2.23) in the closed case),

N 1 1 !nk φ(¯n) = Y . (3.10) k=1 nk! µk

For the closed network case, (3.8) and (3.9) imply that B(t) = B0 for all t ≥ 0. From (2.24), (3.5) and (3.10):

N 1 c (t)!nk P (¯n, t) = M! Y k , n¯ ∈ S, t ≥ 0. k=1 nk! µk For the open network, (3.8) implies that

N d log B(t) X = {ci(t)pi0 − λp0i} dt i=1 N   N   X  X  = ci(t) 1 − pij − λp0i i=1  j=1  N 1 dc (t) = − X i , i=1 µi dt which implies

N " # N " # Y ck(t) − ξk Y ck(t) B(t) = B0 exp − = exp − . k=1 µk k=1 µk 50 Transient behaviour: product-form distributions

From (3.10):

N " c (t)# 1 c (t)!nk P (¯n, t) = Y exp − k k , n¯ ∈ S, t ≥ 0. k=1 µk nk! µk Theorem 3.1 applies to open and closed networks separately, but it also applies to mixed queueing networks where, for example, part of the network is closed and part of the network is open. This is a direct consequence of the product-form initial distribution which guarantees that separate parts of the network are independent at t = 0. If these parts do not interact then they remain independent and the product-form holds for all t ≥ 0. Furthermore, in the proof of Theorem 3.1 λ may be replaced by λ(t). This implies that the results from [39] are generalized in the theorem above to product-form initial conditions. In this case, in the traffic equations (3.6) λ must be replaced by λ(t) too, yielding the traffic equations for an open queueing network with time-dependent Poisson input. Note that λ(t) plays a role similar to ck(t), k = 1,...,N, and can be regarded as c0(t). The Kolmogorov forward equations are a set of linear differential equations. Therefore, if P (i)(¯n, t) is a solution of the Kolmogorov forward equa- (i) PI (i) tions with initial conditions P0 (¯n), i = 1,...,I, then i=1 kiP (¯n, t) is a solution of the Kolmogorov forward equations with initial conditions PI (i) i=1 kiP0 (¯n). This allows further extension of the possible initial distributions to non product-form initial conditions.

4 Necessary conditions for the open network

This section considers open queueing networks and shows that a necessary condition for the network to have a transient product-form distribution is that all stations are M/M/∞-queues. First a technical result considering the solution of the traﬃc equations is given in Lemma 4.1. This lemma will be used in the proof of the necessity result stated in Theorem 4.3.

N N N Lemma 4.1 Let {µi}i=1 be a set of positive numbers, {λi}i=1 and {ξi}i=1 be sets of non-negative numbers and P = [pij], i, j = 1,...,N, be a stochastic matrix whose essential submatrices are strictly substochastic and such that for all j, j = 1,...,N

∃ a sequence i , . . . , i such that λ p p ··· p > 0. (4.1) j1 jk ij1 ij1 ij2 ij2 ij3 ijk j 3.4 Necessary conditions for the open network 51

N Let {ci(t)}i=1 be a solution to the set of diﬀerential equations

N 1 dci(t) X = cj(t)pji − ci(t) + λi, i = 1,...,N, (4.2) µi dt j=1 such that ci(0) = ξi, i = 1,...,N. (4.3) N Then if there exists an n¯ ∈ IN0 \{0} such that

N Y ni ci(t) , (4.4) i=1 is independent of t, it must be the case that the initial conditions are such that the ci(t) are in equilibrium. That is, the ξi satisfy

N X ξi = ξjpji + λi, i = 1,...,N. j=1 ¯ ¯ ¯ Proof Deﬁne the vectorsc ¯(t), λ, and ξ, such that [¯c(t)]i = ci(t), [λ]i = λi ¯ and [ξ]i = ξi and the matrix M such that

( µ , if i = j, [M] = i ij 0, otherwise.

The system of equations (4.2) with initial conditions (4.3) can now be written dc¯(t) =c ¯(t)[P − I]M + λ¯M, (4.5) dt such that c¯(0) = ξ.¯ (4.6) The assumption that the essential submatrices of P are strictly substochastic implies that all the eigenvalues of (P − I)M have negative real parts (see [75]) and so the solution of (4.5) can be written in the form (see, e.g. [69])

J m(j) ¯ −1 X X l −αj t c¯(t) = λ(I − P ) + Ajl t e w¯jl, (4.7) j=1 l=0

J where the set {−αj}j=1 contains the distinct eigenvalues of (P − I)M, which have algebraic multiplicity m(j) + 1, ordered so that Re(αm+1) ≥ 52 Transient behaviour: product-form distributions

Re(αm) and Im(αm+1) > Im(αm) if Re(αm) = Re(αm+1), and thew ¯jl are linearly independent vectors. The Ajl are constants determined by the initial conditions (4.6), and the factor λ¯(I − P )−1 arises as a particular solution to (4.5). The matrix (I −P )−1 contains only non-negative entries (see Theorem 2.6 of [75]). Moreover, assumption (4.1) implies that λ¯(I − P )−1 contains only positive entries. The individual components of (4.7) can be written

J m(j) (i) −α0t X X (i) l −αj t ci(t) = A00 e + Ajl t e , (4.8) j=1 l=0

(i) ¯ −1 where, for notational convenience, α0 = 0 and A00 = [λ(I − P ) ]i are introduced to represent the first term in the right-hand side. For fixed i let (i) J(i) = max{j ≥ 0 : ∃l with Ajl 6= 0}, and for fixed i and j ≤ J(i) define

( max{l ≥ 0 : A(i) 6= 0}, if such an l exists, L = jl ji 0, otherwise.

J(i), the maximum value of j for which there is a non-zero term with e−αj t (i) in (4.8), is well deﬁned because A00 6= 0. Using (4.8) and the ordering of the αi it follows that the coeﬃcient of

h ini tLJ(i)i e−αJ(i)t

ni ni in ci(t) is B(i) ≡ (AJ(i)LJ(i)i ) which is non-zero. The coeﬃcient of

P n L " # i i J(i)i X t exp − niαJ(i)t (4.9) i in the expansion of (4.4) is N Y B(i). i=1 N ¯ Now forn ¯ ∈ IN0 ,n ¯ 6= 0, (4.9) depends on t unless αJ(i) = 0 for all i. Also (4.9) is linearly independent of the other terms in the expansion of (4.4). It follows that, for (4.4) to be independent of t, αJ(i) = 0 for all i which in turn implies J(i) = 0 for all i. 3.4 Necessary conditions for the open network 53

Thus, from (4.8)

(i) ¯ −1 ci(t) = A00 = [λ(I − P ) ]i and the system is in equilibrium. 2 Remark 4.2 (Assumption (4.1); limitations of the result) In the following theorem ξi, λi and pij will be interpreted as initial conditions, arrival parameters and routing parameters for an open network of queues. In this context the ij-th entry of (I − P )−1 gives the expected number of visits of a customer to station j per sojourn time in the network conditional on it entering the network in station i. Thus the j-th entry of

1 ¯ −1 PN λ(I − P ) i=1 λi is the expected number of visits of a customer to station j per sojourn in the network. By Assumption (4.1) this is non-zero. If this assumption is not satisﬁed there exist stations which cannot be reached from outside the (i) network. Assume station i is such a station. Then in (4.8) above A00 = 0. There are now two cases to be considered. (i) 1. There exist j, l such that Ajl > 0, in which case B(i) > 0 and an argument similar to that used above leads to the conclusion that (4.4) is dependent of t.

2. ci(t) = 0 ∀t, which implies ξi = 0 and station i started oﬀ empty, as did all other stations j with pji > 0. In this case these stations never have any customers and can be removed from the model. Conversely, if station i is such that it can be reached from outside the network but customers can never depart the network having reached station i the assumption that the essential submatrices of (I − P ) are strictly substochastic no longer holds, and 0 is an eigenvalue of (I − P ). In this case the i-th component of the particular solution of (4.5) has the form (i) j A00 t , j > 0,

(i) ni QN ni where A00 6= 0. It follows that ci(t) and i=1 ci(t) can never be independent of t. This is to be expected since, in this case, there exists no equilibrium distribution for the queueing network. The last case to be considered is that in which there exist stations to which customers can neither arrive from outside the network nor depart 54 Transient behaviour: product-form distributions to outside the network. In such a situation there must exist an irreducible closed network which is isolated from any other stations in the network. This case needs a result slightly diﬀerent to Lemma 4.1. This is given in Lemma 5.1 in Section 5. In the proof of Theorem 4.3 below, the Kolmogorov forward equations in the representation given in (3.3) is used. Note that the form (3.3) could only be derived from the Kolmogorov forward equations under the assumption P (¯n, t) > 0 for alln ¯ ∈ S, for all t > 0. 2 Theorem 4.3 Assume the network has an initial distribution of the form

N Y nk P (¯n, 0) = B0φ(¯n) ξk . (4.10) k=1 Then it is necessary for the time-dependent probability distribution to be of product-form N Y nk P (¯n, t) = B(t)φ(¯n) ck(t) , (4.11) k=1 N that for the set of numbers {µk}k=1 deﬁned as 1 µk = (4.12) φ(ek) N the {ck(t)}k=1 satisfy N 1 dck(t) X = ci(t)pik + λp0k − ck(t), k = 1,...,N, (4.13) µk dt i=1 with initial condition

ck(0) = ξk, k = 1,...,N, (4.14) and that the normalization constant B(t) satisﬁes

N 1 dB(t) X = {ci(t)pi0 − λp0i} , (4.15) B(t) dt i=1 with initial condition B(0) = B0. (4.16) Furthermore, if the network is not in equilibrium, then it is necessary for the existence of a product-form transient distribution that for all n¯ ∈ S, k = 1,...,N, φ(¯n − e ) k = µ n . (4.17) φ(¯n) k k 3.4 Necessary conditions for the open network 55

Proof Relation (3.3) must hold for alln ¯ ∈ S, therefore insertion ofn ¯ = 0 is allowed. Using the fact that φ(¯n) = 0 if ni < 0 for some i this gives (4.15). Comparison of (4.10) and (4.11) forn ¯ = 0 yields B(0) = B0 which proves (4.16). Insertion of (4.15) into (3.3) gives

N N  N X ni dci(t) X 1 φ(¯n − ei) X = {cj(t)pji − ci(t)pij} i=1 ci(t) dt i=1 ci(t) φ(¯n) j=1   − ci(t)pi0 + λp0i . (4.18) 

Insertion ofn ¯ = ei into this relation and use of (2.13) and (4.12) gives the traﬃc equations (4.13). (4.10) and (4.11) for t = 0 andn ¯ = ei imply that the initial condition for ck(t) is given by ck(0) = ξk. Insertion of (4.13) in (4.18) gives N ( ) X 1 dci(t) φ(¯n − ei) 0 = ni − , (4.19) i=1 ci(t) dt µiφ(¯n) which must hold for alln ¯. The remainder of the proof shows by an inductive argument that, unless the network is in equilibrium, for alln ¯, N 1 1 !nk φ(¯n) = Y . (4.20) k=1 nk! µk Ifn ¯ = 0 then, by (2.13), φ(0) = 1 which satisﬁes (4.20). Now assume that PN (4.20) holds for alln ¯ such that i=1 ni ≤ M − 1. Then forn ¯ such that PN i=1 ni = M insertion of (4.20) forn ¯ − ei into (4.19) gives   N N !nk−δki X 1 dci(t)  1 Y 1 1  0 = ni − i=1 ci(t) dt  µiφ(¯n) k=1 (nk − δki)! µk  N n dc (t) ( 1 N 1 1 !nk ) = X i i 1 − Y . (4.21) i=1 ci(t) dt φ(¯n) k=1 nk! µk Lemma 4.1 and Remark 4.2 imply that, unless the process is in equilibrium, that for at least some t > 0 N n dc (t) d " N # X i i Y nk = log ck(t) 6= 0. i=1 ci(t) dt dt k=1 Therefore, from (4.21) it follows that (4.20) holds forn ¯, which completes the induction step. Taking the quotient of (4.20) forn ¯ −ei andn ¯ immediately implies that (4.17) holds. 2 56 Transient behaviour: product-form distributions

5 Necessary conditions for the closed network

This section turns to closed queueing networks. Because the state space is PN restricted to S = {n¯ : i=1 ni = M} the induction argument used for open queueing networks cannot be applied. This section proves an analogue of Theorem 4.3 but, in the ﬁnal part, is able only to prove that if one station of a network with time-dependent product-form is an M/M/∞- queue independent of the other stations then, unless the network is in equilibrium, all the stations must be M/M/∞-queues. The stations can always be re-labelled so that this “special” station is station number 1. In the notation of Section 2 this independence is modelled by writing

ψe(¯v) ψ(¯n) = n1 , (5.1) µ1 n1! and φe(¯v) φ(¯n) = n1 , (5.2) µ1 n1! wherev ¯ is the vector which is obtained fromn ¯ by removing the ﬁrst component and φe (ψe) is the function which is obtained from φ (ψ) by removing the ﬁrst component from its argument. Since station 1 is an independent M/M/∞-queue, f(¯n, 1) = µ1n1. This gives

ψ(¯n − e1) ψe(¯v)µ1n1 f(¯n, 1) = = = µ1n1, (5.3) φ(¯n) φe(¯v) and so ψe(¯v) = φe(¯v). (5.4) Lemma 5.1, below, gives a preliminary result needed for the theorem. In the analysis below it is assumed that the network is irreducible. Note that this is not a restriction since the irreducible subnetworks can be analyzed separately if the network is reducible.

N N Lemma 5.1 Let {µi}i=1 be a set of positive numbers and {ξi}i=1 be a set of non-negative numbers, P = [pij] be an irreducible stochastic matrix, N and {ci(t)}i=1 be a solution to the set of diﬀerential equations

N 1 dci(t) X = cj(t)pji − ci(t), i = 1,...,N, (5.5) µi dt j=1 3.5 Necessary conditions for the closed network 57 such that ci(0) = ξi, i = 1,...,N, and for all t N c (t) X i = 1. (5.6) i=1 µi N PN Then if there exist n¯ ∈ S = {n¯ ∈ IN0 : i=1 ni = M} other than Me1 such that N Y ni ci(t) , (5.7) i=2 is independent of t, it must be the case that the initial conditions are such that the ci(t) are in equilibrium, i.e. the ξi satisfy

N X ξj = ξipij, j = 1,...,N. i=1 Proof With notation introduced in the proof of Lemma 3.1, (5.5) can be written dc¯(t) =c ¯(t)(P − I)M, dt such that c¯(0) = ξ,¯ and the individual components ci(t) still have the form (4.8) except that (i) −α0t the term A00 e arises not as a particular solution to the inhomogeneous diﬀerential equation, but because α0 = 0 is now an eigenvalue (of multi- (i) plicity 1) of (P − I)M. It follows that the A00 are the equilibrium values for the closed network (up to a multiplicative constant). The ci(t) can be chosen such that (5.6) holds by the lemma on canonical form, Lemma 2.2. Using similar arguments to the proof of Lemma 4.1, it can be concluded that the coeﬃcient of

N P n L " N # i=2 i J(i)i X t exp − niαJ(i)t i=2

QN in the expansion of (5.7) is i=2 B(i) which is non-zero. Thus, again by arguments similar to those in Lemma 4.1, (5.7) is (i) independent of t only if Ajl = 0 in (4.8) for all j > 0. In this case 58 Transient behaviour: product-form distributions

(i) ci(t) = A00 = ξi, i = 2,...,N. However, by (5.6), for all t

N ! X ci(t) c1(t) = µ1 1 − i=2 µi = ξ1.

The network is thus in equilibrium. 2

Theorem 5.2 Assume the network has an initial distribution of the form

N Y nk P (¯n, 0) = B0φ(¯n) ξk . k=1 Then it is necessary for the time-dependent probability distribution to be of product-form N Y nk P (¯n, t) = B(t)φ(¯n) ck(t) , (5.8) k=1 N that for the set of numbers {µk}k=1 deﬁned as

ψ((M − 1)ek) µk = Mφ(Mek)

N the {ck(t)}k=1 satisfy

N 1 dck(t) X = {ci(t)pik − ck(t)pki} , k = 1,...,N, (5.9) µk dt i=1 with initial condition

ck(0) = ξk, k = 1,...,N, (5.10) and that B(t) = B0, t ≥ 0. (5.11) Furthermore, if station 1 is an independent M/M/∞-queue as described in (5.1), (5.2), and (5.3), and if the network is not in equilibrium, then it is necessary for the existence of a product-form transient distribution that

ψ(¯n − e ) k = µ n . (5.12) φ(¯n) k k 3.5 Necessary conditions for the closed network 59

Proof Insertion ofn ¯ = Mei into (3.3) gives after rearranging the terms

N 1 dB(t) ci(t) 1 dci(t) X + M = M {cj(t)pji − ci(t)pij} . (5.13) B(t) dt µi µi dt j=1 Summation of (5.13) over all i gives by using (5.6) dB(t) = 0, dt which implies (5.11). Insertion of (5.11) into (5.13) implies that the traﬃc equations (5.9) hold. Insertion ofn ¯ = Mei into (5.8) for t = 0 gives (5.10). Insertion of (5.9) and (5.11) into (3.3) gives

N ( ) X 1 dci(t) ψ(¯n − ei) 0 = ni − . (5.14) i=1 ci(t) dt µiφ(¯n) Now assume that station 1 is an independent M/M/∞-queue with service- rate n1µ1 if n1 jobs are at station 1. Then from (5.4)

ψ(¯n − ei) φe(¯v − ei) = 1{i 6= 1} + niµi1{i = 1}. φ(¯n) φe(¯v) Insertion of this form into (5.14) gives

N ( ) X 1 dci(t) φe(¯v − ei) 0 = vi − , i=2 ci(t) dt µiφe(¯v)

n PN o which must hold for allv ¯ in the set Ve = v¯ : vi ≥ 0, i=2 vi ≤ M . By an inductive argument, using Lemma 5.1 it can be shown that, unless the process is in equilibrium,

φe(¯v − ei) = viµi. φe(¯v) 2

6 Discussion and general remarks

This chapter has shown that unlike queueing networks in equilibrium, a queueing network which is not in equilibrium can have the transient product-form distribution

N Y nk P (¯n, t) = B(t)φ(¯n) ck(t) (6.1) k=1 60 Transient behaviour: product-form distributions only if it is a network of inﬁnite-server queues. In addition it is proven that a network of M/M/∞-queues with an initial product-form distribution

N Y nk P (¯n, 0) = B0φ(¯n) ξk k=1 has the transient product-form distribution (6.1), where the ck(t) satisfy the time-dependent traﬃc equations

N 1 dck(t) X = ci(t)pik + λp0k − ck(t), k = 1,...,N, (6.2) µk dt i=1 subject to ck(0) = ξk, k = 1,...,N. These two results can be combined to show that it is necessary and sufficient for a network of queues, not in equilibrium, to have a product-form transient distribution that it be a network of infinite-server queues. For more specific initial conditions, but more general arrival characteristics and service-time distributions the sufficiency part of this result has been previously established in [32], [39] and [56]. However, the time-dependent traffic equations (6.2) appear to be new, as is the expression of the open and closed models in canonical form. The results presented here have the advantage of reducing the derivation of the transient distribution of the network to the solution of a set of linear differential equations with a number of variables equal to the number of stations in the network. There are several observations that can immediately be made. It is clear from the form of (4.7) that as t → ∞ P (¯n, t) approaches the equilibrium distribution π(¯n). For open networks it is of interest to also consider the case where the arrival rate to station i is a function of time (say λp0i = λi(t)). This was considered in [32] and [39], where it was shown that networks of queues with non-homogeneous Poisson input have a product-form transient distribution. It is easy to show, using methods similar to those given in this chapter, that if a Markovian network of queues with non-homogeneous Poisson input has product-form then (4.13) - (4.16) must be satisfied. However, it is unclear how to argue, in general, results analogous to Lemma 4.1 and Remark 4.2. These results are used in the proof of Theorem 4.3 only to show that if a network is not in equilibrium then for some t > 0 N n dc (t) X i i 6= 0. (6.3) i=1 ci(t) dt 3.6 Discussion 61

If (6.3) can be assumed then (4.17) also holds for networks with non- homogeneous Poisson input. A similar observation can be made with respect to the proof of Theorem 5.2. Statements (5.9) - (5.11) can be shown to follow from product-form even if it is not assumed that one station is an infinite-server queue. These assumptions are used only in the proof of (5.12). If a network of queues is to have a product-form transient distribution for arbitrary service-time distributions then it follows from the analysis presented here that it must be a network of infinite-server queues merely by observing that exponential service-times are special cases of general service-times. Then, from [39] and Theorems 4.3 and 5.2 it follows that a necessary and sufficient condition for a queueing network to have transient product-form (6.1) is that it is a network of infinite-server queues (M/G/∞-queues). 62 Chapter 4

Transient behaviour of the Engset loss model

1 Introduction

This chapter studies the transient behaviour of the Engset model [30]. The Engset loss model consists of a service-system with s servers and quasirandom input. This input process is generated by N sources. Each source generates requests at negative-exponential rate γ when the source is idle. If a server at the service-system is free, the request is served at negative-exponential rate µ and the source becomes busy until its request completes service. Otherwise, the request is lost (blocked) and the source remains idle. It is well-known that the equilibrium distribution of the number of requests at the service-system is of product-form. If the number of sources does not exceed the number of servers, i.e. if N ≤ s, this product-form expression can be extended to the transient distribution of the number of requests at the service-system. This is intuitively obvious since requests cannot be blocked and thus requests are independent. This chapter proves that the transient distribution is of product-form if and only if the number of sources does not exceed the number of servers, i.e. if and only if N ≤ s. This result is similar to the result obtained in Chapter 3, where it is proven that a necessary and suﬃcient condition for the transient distribution of the number of customers at the stations in a queueing network to be of product-form is that all stations are inﬁnite- server queues. An essential assumption in Chapter 3 is that customers cannot be blocked, i.e. each customer arriving at a station is accepted to receive service, and the method of proof used in Chapter 3 depends heavily

63 64 Transient behaviour: Engset loss model on this assumption. In the Engset loss model customers arriving at the service-system when all servers are busy are lost. Thus, the Engset loss model does not satisfy the essential assumption that all arriving customers are accepted to receive service and the result for the Engset loss model cannot be concluded from Chapter 3. The method in proving that the transient queue-length distribution is not of product-form is completely diﬀerent from the proof given in Chapter 3. The present chapter shows that the proof in Chapter 3 cannot be extended to include blocking, but the result of Chapter 3 extends to queueing networks with blocking, that is queueing networks with blocking do not possess a product-form transient distribution. A generalization of product-form distributions are distributions that are a sum of product-forms. As a second result this chapter shows that the transient distribution is a sum of two product-form distributions if and only if N = 2s+1 and γ = µ. The product-form distributions used in this result are the product-form distributions for the Engset model with N servers, i.e. for the Engset model in which each source has a reserved server. This chapter considers a very special model only. Moreover, for this special model only two cases are discussed. If N ≤ s it is shown that the transient distribution is of product-form and if N = 2s + 1 and γ = µ it is shown that the transient distribution is a sum of two product-forms. As analytical results of a simple form for the transient distribution are very diﬃcult to obtain, each result is of (theoretical) interest in itself.

2 Engset loss model

Consider a service-system consisting of s servers. The input into the service-system is generated by N sources. Each source generates a request at negative-exponential rate γ when the source is idle, independent of the state of the other sources. If a server is free, the request is served by the service-system at negative-exponential rate µ and the source becomes busy until its request completes service. If all servers are busy, the request is lost and the source remains idle. Let N(t) denote the number of requests at the service-system. The stochastic process, {N(t), t ≥ 0}, at state space S = {0, 1, 2, . . . , s0}, s0 = min(s, N), describing the number of requests in the system, can be modelled by a birth-and-death process with transition 4.2 Model 65 rates, Q = (q(n, n0), n, n0 ∈ S), given by ([21])

 0 0  nµ, n = n − 1, 0 < n ≤ s ,   −nµ − (N − n)γ, n0 = n, 0 ≤ n < s0 − 1,  q(n, n0) = −s0µ, n0 = n, n = s0,  0 0  (N − n)γ, n = n + 1, 0 ≤ n < s − 1,   0, otherwise. The probability that the process with s servers at the service-system is (s) in state n at time t, Pn (t) ≡ P{N(t) = n}, n ∈ S, t ≥ 0, satisﬁes the Kolmogorov forward equations [21]

dP (s)(t) 0 = −NγP (s)(t) + µP (s)(t), dt 0 1 dP (s)(t) n = (N − n + 1)γP (s) (t) − (nµ + (N − n)γ)P (s)(t) dt n−1 n (s) 0 +(n + 1)µPn+1(t), 0 < n < s , (s) dPs0 (t) 0 (s) 0 (s) = (N − s + 1)γP 0 (t) − s µP 0 (t), dt s −1 s with initial conditions

(s) Pn (0) = pn, n ∈ S. Equivalently, the Engset loss model can be modelled by a closed queueing network consisting of two stations in which N customers are present. Here, station 1 is an M|M|∞-queue with servers working at negative- exponential rate γ and station 2 is an M|M|s|s-queue with servers working at negative-exponential rate µ. As the number of customers at station 1 is completely determined by the number of customers at station 2, the queueing network can be modelled by considering the number of customers at station 2 only. Thus, the queueing network is described by {N(t)}, denoting the number of customers at station 2. As {N(t)} is a birth-and-death process, the equilibrium distribution π(s) = lim P (s)(t), n ∈ S, n t→∞ n exists, is independent of the initial conditions, and is given by

N−n n  0 N−k k−1 N ! 1 ! 1 ! s N ! 1 ! 1 ! π(s) = X , n ∈ S. n n  k  γ µ k=0 γ µ (2.1) 66 Transient behaviour: Engset loss model

3 Transient queue-length distribution

In contrast to the equilibrium distribution, in general, the transient distri- (s) bution, Pn (t), is not of product-form. This section shows that a necessary and suﬃcient condition for the transient distribution to be of product- form is that N ≤ s. Furthermore, this section shows that a necessary and suﬃcient condition for the transient distribution to be a sum of two product-form distributions is that N = 2s + 1 and γ = µ. Throughout this section, for simplicity it is assumed that at t = 0 all servers at the service-system are free, i.e. that the initial distribution of {N(t)} is

P{N(0) = 0} = 1. (3.1)

Furthermore, without loss of generality as this inﬂuences the value of the normalizing constant only (Lemma 3.2.2), γ, µ can be chosen such that 1 1 + = 1. (3.2) γ µ First, consider the case where the number of sources does not exceed the number of servers at the service-system, i.e.

Case 1: N ≤ s. Then, in the queueing network description, both stations are essentially equivalent to M|M|∞-queues. Thus, from Theorem 3.3.1, the transient (s) distribution of the number of customers at station 2, Pn (t), is given by

N−n n N ! c (t)! c (t)! P (s)(t) = 1 2 , n ∈ S = {0, 1,...,N}, t ≥ 0, n n γ µ (3.3) where c1(t), c2(t) satisfy 1 dc (t) 1 = c (t) − c (t), γ dt 2 1 1 dc (t) (3.4) 2 = c (t) − c (t), µ dt 1 2 with initial conditions

c1(0) = γ, c2(0) = 0. (3.5) 4.3 Queue-length distribution 67

As (3.4) with initial conditions (3.5) possesses a positive solution for all t ≥ 0, given by γ c (t) = 1 + e−(γ+µ)t, c (t) = 1 − e−(γ+µ)t, t ≥ 0, (3.6) 1 µ 2 for N ≤ s, the transient distribution is of product-form. Note that this is not a surprising result, since in this case requests cannot be blocked, i.e. requests are independent. The following theorem shows that this result cannot be generalized to the case N > s.

Theorem 3.1Unless N ≤ s there do not exist B(t), c1(t), c2(t) : [0, ∞) → (0, ∞), and φ(n): S → (0, ∞) such that limt→∞ B(t), limt→∞ c1(t) and 0 limt→∞ c2(t) exist, and for all t ≥ 0, n ∈ S = {0, 1, . . . , s = min(s, N)}

c (t)!N−n c (t)!n P (s)(t) = B(t)φ(n) 1 2 . (3.7) n γ µ

Note that if N ≤ s then B(t), φ(n) can be chosen as

N ! B(t) = 1, φ(n) = n and c1(t), c2(t) can be chosen as given in (3.6) above.

(s) Proof Assume Pn (t) has the form given in (3.7) with B(t), c1(t), c2(t) such that the limits for t → ∞ exist. Then, as {N(t)} is a birth-and-death (s) (s) process, limt→∞ Pn (t) exists and is given by πn . Taking limits in (3.7) gives for all n ∈ S

lim c (t)!N−n lim c (t)!n π(s) = lim P (s)(t) = lim B(t)φ(n) t→∞ 1 t→∞ 2 . n t→∞ n t→∞ γ µ

(s) As {N(t)} is a birth-and-death process, Pn (t) > 0 for all t > 0, for all n ∈ S, thus taking quotients for n, n0 is allowed. By insertion of (2.1) this gives for n, n0 ∈ S

N−n n N−n n lim c (t)! lim c (t)! N ! 1 ! 1 ! φ(n) t→∞ 1 t→∞ 2 γ µ n γ µ 0 0 = N−n0 n0 . lim c (t)!N−n lim c (t)!n N ! 1 ! 1 ! φ(n0) t→∞ 1 t→∞ 2 γ µ n0 γ µ 68 Transient behaviour: Engset loss model

Cancelling terms gives for n, n0 ∈ S

N ! !n−n0 φ(n) limt→∞ c2(t) n 0 = !, φ(n ) limt→∞ c1(t) N n0 which immediately gives for n0 = 0

−n N ! lim c (t)! φ(n) = t→∞ 2 φ(0), n ∈ S. n limt→∞ c1(t)

(s) Thus Pn (t) can be written

N−n n N ! cˆ (t)! cˆ (t)! P (s)(t) = Bˆ(t) 1 2 , (3.8) n n γ µ where

N ˆ B(t) = B(t)φ(0) lim c1(t) , t→∞

cˆ1(t) = c1(t)/ lim c1(t), t→∞ cˆ2(t) = c2(t)/ lim c2(t). t→∞ Inserting (3.8) into the Kolmogorov forward equations gives for n = 0:

dBˆ(t) dcˆ (t) cˆ (t) + N 1 = −Nγ + Nγ 2 , (3.9a) Bˆ(t)dt cˆ1(t)dt cˆ1(t) for 0 < n < s0:

dBˆ(t) dcˆ (t) dcˆ (t) + (N − n) 1 + n 2 Bˆ(t)dt cˆ1(t)dt cˆ2(t)dt cˆ (t) cˆ (t) = nµ 1 − (nµ + (N − n)γ) + (N − n)γ 2 , (3.9b) cˆ2(t) cˆ1(t) and for n = s0:

dBˆ(t) dcˆ (t) dcˆ (t) cˆ (t) + (N − s0) 1 + s0 2 = s0µ 1 − s0µ. (3.9c) Bˆ(t)dt cˆ1(t)dt cˆ2(t)dt cˆ2(t) 4.3 Queue-length distribution 69

Subtraction of (3.9a) from (3.9c) gives

( 0 ) dcˆ2(t) dcˆ1(t) 1 s µ Nγ − = 0 + (ˆc1(t) − cˆ2(t)). (3.9d) cˆ2(t)dt cˆ1(t)dt s cˆ2(t) cˆ1(t)

Insertion of (3.9a) and (3.9d) into (3.9b) gives for 0 < n < s0

Nγ n ( s0µ Nγ ) (−cˆ1(t) +c ˆ2(t)) + 0 + (ˆc1(t) − cˆ2(t)) cˆ1(t) s cˆ2(t) cˆ1(t)

nµ (N − n)γ = (ˆc1(t) − cˆ2(t)) + (−cˆ1(t) +c ˆ2(t)), cˆ2(t) cˆ1(t) 0 which is satisﬁed ifc ˆ1(t) =c ˆ2(t) or if for 0 < n < s Nγ nµ Nγ N nµ Nγ nγ − − 0 = − + − , cˆ1(t) cˆ2(t) cˆ1(t) s cˆ2(t) cˆ1(t) cˆ1(t)

0 (s) i.e. if N = s = min(s, N). By assumption N > s, thus for Pn (t) to satisfy the Kolmogorov forward equations, it must be the case thatc ˆ1(t) =c ˆ2(t) for all t which is in contrast to (3.5). 2

The assumption (3.1) on the initial distribution can be relaxed too: Pn(0) is of product-form, i.e.

N−n n N ! c (0)! c (0)! P (s)(0) = 1 2 , n ∈ S = {0, 1, . . . , s0}, n n γ µ

c1(0) c2(0) where c1(0), c2(0) are such that γ + µ = 1. As can immediately be seen from the proof above, Theorem 3.1 holds true if it is assumed that the initial conditions are such that the process is not in equilibrium. To this end, note thatc ˆ1(t) =c ˆ2(t) for all t if and only if the process is in equilibrium as can immediately be seen from (3.9a) – (3.9c) since the right- (s) dPn (t) hand sides in these equations equal 0. Thus, for all n ∈ S: dt = 0 for all t which is the case if and only if the process is in equilibrium, in contrast to the assumptions. The result of Theorem 3.1 is similar to the result obtained in The- orem 3.5.2. There it is shown that for a closed queueing network the transient distribution of the number of customers at the stations is of product-form if and only if all stations are M|M|∞-queues. An essential assumption in Chapter 3, however, is that Pn(t) > 0 for all n ∈ {0,...,N}. 70 Transient behaviour: Engset loss model

For the Engset loss model with N > s this assumption is not satisfied as Pn(t) = 0 if n ∈ {s+1,...,N}. The proof of Theorem 3.5.2 depends heavily on this assumption. To show that the result of the present paper cannot be concluded in Chapter 3, consider an outline of the proof in Chapter 3 in the setting of this chapter. First, as an immediate consequence of the assumption that Pn(t) > 0 for all n ∈ {0,...,N}, it is shown that if the transient distribution is of product-form, then the coefficients c1(t), c2(t) satisfy (3.4) by considering the Kolmogorov forward equations for n = 1 and n = N − 1. Based on this result, by induction to n it is shown that N φ(n) = n for all n ∈ {0,...,N}, which implies that both stations must be M|M|∞-queues. In the case of the Engset loss model for N > s the assumption that Pn(t) > 0 for all n ∈ {0,...,N} is not satisfied. There- fore, it cannot be shown that the coefficients c1(t), c2(t) satisfy (3.4) by considering the Kolmogorov forward equations for n = N − 1 as these are not defined. Then the second step in the proof of Theorem 3.5.2 cannot be made. Therefore, as both the essential assumption is not satisfied and the method of proof of Theorem 3.5.2 cannot be applied, Theorem 3.1 is not a special case of the result in Chapter 3. In contrast to the equilibrium distribution, the transient distribution for N > s is not of product-form. The transient distribution can be expressed using the eigenvalue-eigenvector-expansion of the rate matrix. However, this expression involves Bessel functions and eigenvalues for Q and is very tedious and not transparent (cf. [21], [68]). In the special case

Case 2: N = 2s + 1, γ = µ, it will be shown that the transient distribution can be expressed as a sum of two product-form terms. This is stated in the following theorem, where as a consequence of (3.2) it is assumed that γ = µ = 2.

(s) Theorem 3.2 If N = 2s+1, γ = µ = 2, the transient distribution Pn (t), t > 0 can be written

(s) (2s+1) (2s+1) Pn (t) = Pn (t) + PN−n (t), n ∈ S = {0, 1, . . . , s}, (3.10)

(2s+1) where Pk (t), 0 ≤ k ≤ 2s + 1, is given in (3.3), i.e.

2s+1−k k 2s + 1 ! c (t)! c (t)! P (2s+1)(t) = 1 2 , t ≥ 0. k k γ µ 4.3 Queue-length distribution 71

Proof Insertion of (3.10) into the Kolmogorov forward equations gives for n = 0:

(s) dP (t) d n o 0 = P (2s+1)(t) + P (2s+1)(t) dt dt 0 N (2s+1) (2s+1) (2s+1) (2s+1) = −2NP0 (t) + 2P1 (t) − 2NPN (t) + 2PN−1 (t) n (2s+1) (2s+1) o n (2s+1) (2s+1) o = −2N P0 (t) + PN (t) + 2 P1 (t) + PN−1 (t) (s) (s) = −2NP0 (t) + 2P1 (t), for 0 < n < s: dP (s)(t) d n o n = P (2s+1)(t) + P (2s+1)(t) dt dt n N−n (2s+1) (2s+1) (2s+1) = 2(N − n + 1)Pn−1 (t) − 2NPn (t) + 2(n + 1)Pn+1 (t) (2s+1) (2s+1) (2s+1) +2(n + 1)PN−n−1 − 2NPN−n (t) + 2(N − n + 1)PN−n+1(t) n (2s+1) (2s+1) o = 2(N − n + 1) Pn−1 (t) + PN−(n−1)(t) n (2s+1) (2s+1) o −2N Pn (t) + PN−n (t) n (2s+1) (2s+1) o +2(n + 1) Pn+1 (t) + PN−(n+1)(t) (s) (s) (s) = 2(N − n + 1)Pn−1(t) − 2NPn (t) + 2(n + 1)Pn+1(t), and ﬁnally, for n = s: dP (s)(t) d n o s = P (2s+1)(t) + P (2s+1)(t) dt dt s N−s (2s+1) (2s+1) (2s+1) = 2(N − s + 1)Ps−1 (t) − 2NPs (t) + 2(s + 1)Ps+1 (t) (2s+1) (2s+1) (2s+1) +2(s + 1)PN−s−1 − 2NPN−s (t) + 2(N − s + 1)PN−s+1(t) n (2s+1) (2s+1) o = 2(N − s + 1) Ps−1 (t) + PN−(s−1)(t) n (2s+1) (2s+1) o −2N Ps (t) + PN−s) (t) n (2s+1) (2s+1) o +2(s + 1) Ps+1 (t) + PN−(s+1)(t) (s) (s) = 2(N − s + 1)Ps−1(t) − 2sPs (t) n (2s+1) (2s+1) o −2(N − s) Ps (t) + PN−s (t) n (2s+1) (2s+1) o +2(s + 1) Ps+1 (t) + PN−(s+1)(t) (s) (s) = 2(N − s + 1)Ps−1(t) − 2sPs (t), 72 Transient behaviour: Engset loss model where the last step is obtained by inserting N = 2s + 1. (2s+1) As Pn (t), 0 ≤ n ≤ 2s + 1, is a probability distribution for all t and N = 2s + 1, it immediately follows that for all t ≥ 0

s s X (s) X n (2s+1) (2s+1) o Pn (t) = Pn (t) + PN−n (t) n=0 n=0 s N X (2s+1) X (2s+1) = Pn (t) + Pn (t) = 1. n=0 n=s+1

(s) Thus Pn (t) is a probability distribution which completes the proof. 2

If N = 2s + 1, γ = µ, the assumption (3.1) on the initial distribution can be relaxed to Pn(0) is a sum of two product-form distributions:

N−n n N ! c (0)! c (0! P (s)(0) = 1 2 n n γ µ n N−n N ! c (0)! c (0)! + 1 2 , N − n γ µ as can immediately be seen from the proof above. The transient probability of state n for the Engset loss model with N = 2s + 1 sources, s servers, and γ = µ, is the sum of the transient distribution of state n and of state N − n for the Engset loss model with N = 2s+1 sources, 2s+1 servers, and γ = µ. This result can be explained as follows. If the number of sources N = 2k + 1, the number of servers s = 2k + 1, and γ = µ, then the transition rates are symmetrical around 1 k + 2 in the sense that q(n, n0) = q(N − n, N − n0), n = 0,...,N = 2k + 1.

Due to this symmetry, when the process reaches n = k it cannot distinguish between passing on to {k + 1,..., 2k + 1}, which is the process described above, or bouncing back into {0, . . . , k}, which is the process with k servers at the service-centre. Thus the probability of state n for (k) (2k+1) (2k+1) the process with k servers is given by Pn (t) = Pn (t) + PN−n (t), 0 ≤ n ≤ k. The result of Theorem 3.2 cannot be generalized to the case N = 2s+1, γ 6= µ nor to the case N 6= 2s+1. Intuitively, these generalizations destroy the symmetry of the process. Formally, this can be seen by observing the equilibrium distribution as is shown in the following theorem. 4.3 Queue-length distribution 73

Theorem 3.3 For the Engset model,

(s) (N) (N) πn = πn + πN−n, 0 ≤ n ≤ s, (3.11) if and only if N = 2s + 1 and γ = µ. Proof If N = 2s + 1 and γ = µ then (3.11) follows from Theorem 3.2. To prove the reversed statement, ﬁrst assume that N = 2s + 1, γ 6= µ, and (3.11) holds. Insertion of (2.1) into (3.11) gives that (3.11) holds iﬀ for n = 0, . . . , s

N−n n N ! 1 ! 1 ! n γ µ s ! !N−k !k X N 1 1 k k=0 γ µ N−n n n N−n N ! 1 ! 1 ! N ! 1 ! 1 ! = + . n γ µ n γ µ

(N) Since N = 2s + 1 and πn is a probability distribution

N−k k N−k k s N ! 1 ! 1 ! N N ! 1 ! 1 ! X + X = 1. (3.12) k k k=0 γ µ k=s+1 γ µ Algebraic manipulations using (3.12) give that (3.11) is satisﬁed iﬀ for n = 0, . . . , s

N−n n n N−n N ! 1 ! 1 ! N ! 1 ! 1 ! n γ µ n γ µ = . (3.13) s ! !N−k !k s ! !k !N−k X N 1 1 X N 1 1 k k k=0 γ µ k=0 γ µ Cancelling terms in (3.13) gives that (3.11) holds iﬀ for n = 0, . . . , s

k−n k−n s N ! µ! s N ! γ ! X = X . (3.14) k k k=0 γ k=0 µ As γ 6= µ it must be the case that either γ < µ or γ > µ. If γ < µ then

k k s N ! µ! s N ! s N ! γ ! X > X > X . k k k k=0 γ k=0 k=0 µ 74 Transient behaviour: Engset loss model

Thus, for n = 0, (3.14) cannot be satisﬁed which is in contrast to the assumption that (3.11) holds. A similar argument shows that (3.14) cannot be satisﬁed if γ > µ. Now assume that N 6= 2s + 1 and (3.11) holds. Summation of (3.11) over n gives if N > 2s + 1

s s X (s) X n (N) (N) o πn = πn + πN−n n=0 n=0 s N X (N) X (N) = πn + πn n=0 n=N−s N−s−1 X (N) = 1 − πn < 1, n=s+1 which implies that (3.11) cannot be satisﬁed. A similar argument shows that (3.11) cannot be satisﬁed if N < 2s + 1 which completes the proof. 2

A direct consequence of Theorems 3.2 and 3.3 is the following corollary.

Corollary 3.4 (Main result) A necessary and suﬃcient condition for (s) the transient distribution Pn (t), t ≥ 0, to be of the form

(s) (2s+1) (2s+1) Pn (t) = Pn (t) + PN−n (t), n ∈ S = {0, 1, . . . , s},

(2s+1) where Pk (t), 0 ≤ k ≤ 2s + 1, is given in (3.3), i.e.

2s+1−k k 2s + 1 ! c (t)! c (t)! P (2s+1)(t) = 1 2 , t ≥ 0, k k γ µ is that N = 2s + 1 and γ = µ.

(k) Proof As {N(t)} is a birth-and-death process limt→∞ Pn (t) exists for all k and for all n, n = 0,..., min(k, N). 2 Chapter 5

Dual processes

The equilibrium behaviour of queueing networks that can be modelled as a continuous-time Markov chain has been studied extensively. Based on a notion of local balance product-form equilibrium distributions have been obtained for a wide class of queueing networks. Chapters 3 and 4 show that these results cannot be extended to the transient behaviour of queueing networks. This chapter analyses product-form queueing networks with transition rates that are a generalization of th processes discussed in Chap- ters 3 and 4 and includes state-dependent routing. The transition structure of these queueing networks is analysed in detail. Based on this structure the dual process is introduced. This process can be chosen such that it describes the state of the queueing network as observed by a customer in transit between the stations of the queueing network.

1 Introduction

The physical model for a transition of a standard queueing network assumes that in a transitionn ¯ → n¯ − ei + ej ﬁrst a customer leaves station i and subsequently this customer is routed to station j. Thus, the transition virtually passes through staten ¯ − ei, the state in which a customer has left station i and did not yet arrive at station j. In some cases, a queueing network can also be modelled considering vacancies at the stations, i.e. positions at the stations where no customers are present. If a customer leaves station i a vacancy is created at station i, and if a customer enters station j a vacancy is deleted at station j. In a transition n¯ → n¯ − ei + ej ﬁrst a vacancy is created at station i and subsequently a vacancy is deleted from station j. Thus, in contrast to the customer

75 76 Dual processes

n¯ − ei + ej − ek n¯ − ei %@ @ % %% % r @ primal process % r @ % @@R % @@R% % rn¯ − ei + ej − ek + el rn¯ − ei + ej rn¯

m¯ + ei m¯ + ej e m¯ + ej − ek + el r e e r e e r dual process eRe e m¯ =n ¯ − e Ree i rm¯ rm¯ + ej − ek Figure 5.1. Sequence of states for the primal and the dual process

process first a unit is created at station i. Letm ¯ = (m1, . . . , mN ) denote the number of vacancies at the stations, then a transitionm ¯ → m¯ +ei −ej, where first a vacancy is created at station i and subsequently a vacancy is deleted at station j virtually passes through statem ¯ + ei. A second process with transitionsm ¯ → m¯ + ej − ek that virtually pass throughm ¯ + ej is the process describing the evolution of states as observed by a customer in transit between two stations. To this end, consider the evolution of the states of a queueing network. In a transitionn ¯ → n¯ − ei + ej a customer leaves station i. In transit, this customer observes staten ¯ − ei. Then the customer is absorbed in station j. Subsequently a customer may leave station k. In transit this customer observes staten ¯ −ei +ej −ek. Thus, as depicted in Figure 5.1, a transitionn ¯ − ei → n¯ − ei + ej − ek is induced in which first a unit is created at station j and subsequently a unit is deleted at station k. The stochastic process describing transitions in which first a unit is created and subsequently a unit is deleted is called the dual process. The transition rates for this dual process are obtained from the transition rates of the primal process, the stochastic process describing the original queueing network, based on a potential interpretation of the transition rates partly due to [55], [87]. This approach leads to a generalization of the arrival theorem. In this generalization the process describing the evolution of states observed by customers in transit is given. Moreover, it is argued that a moving customer may influence the equilibrium distribution of the 5.2 The primal process 77 queueing network. Thus, a moving customer will not see the network as if the customer is not present, but this moving customer sees the network that is influenced by its own presence. The equilibrium distribution observed by a moving customer is shown to be the equilibrium distribution of the dual process. In the examples, the symmetrical relation between the primal an the dual process is shown to be similar to the relation between the primal and dual problem in linear programming thus justifying the name dual process. Furthermore the relation between the transition rates for the primal process and the transition rates for the dual process is discussed. This chapter is organized as follows. In Section 2 the primal process is studied in detail. In particular, the state space and transition rates are discussed. Furthermore, some elementary results on product-form queueing networks are given. These results hint at the form of a dual process. Finally, the primal process is interpreted using potentials. This interpretation will form the basis of the introduction of the dual process in Section 3. Here the transition rates of the dual process are defined based on the potential interpretation of the transition rates of the primal process. Furthermore, a sufficient condition for the dual equilibrium distribution to be of product-form is given. In Section 4, for a general class of primal processes, the dual routing function is given. Section 5 relates the transition rates and equilibrium distribution of the primal process to those of the dual process. In this section the Palm probabilities associated with the jumps of the primal process are studied. It is shown that moving customers see the dual equilibrium distribution thus generalizing the arrival theorem. Section 6 gives some examples and general remarks on the relation between the primal and the dual process.

2 The primal process

Consider a continuous-time queueing network consisting of N stations, labelled 1,...,N. Assume that the queueing network can be represented by a stable, regular, continuous-time Markov chain N = {N(t), t ≥ 0} at N state space S ⊆ IN0 , with statesn ¯ = (n1, . . . , nN ), where ni denotes the number of customers at station i, i = 1,...,N. The transition rate from staten ¯ ∈ S to staten ¯0 ∈ S is denoted by q(¯n, n¯0). As the Markov chain N describes a queueing network, it is natural to assume that transitions of N correspond to routing of customers, i.e. it is natural to assume that, 78 Dual processes forn, ¯ n¯0 ∈ S

0 0 q(¯n, n¯ ) = 0 unlessn ¯ =n ¯ − ei + ej, i, j ∈ N , i 6= j, (2.1) where N = {0,...,N} if the queueing network is open and if the queueing network is closed N = {1,...,N}. In a transitionn ¯ → n¯ − ei + ej first a customer leaves station i and subsequently this customer is routed among the stations. In this transition the customer-configurationn ¯ − ei remains unchanged. Therefore, the transition can be written as (¯n − ei) + ei → (¯n − ei) + ej and can be regarded as virtually passing throughm ¯ =n ¯−ei ∈ N IN0 , a so-called dual state. The set of dual states plays an important role in the definition of the transition rates and in the dual process. Therefore this set is explicitly defined here. Definition 2.1 (Dual state space) For a stochastic process N at state space S with transition rates satisfying (2.1), the dual state space, Sd, is the set

d N S = {m¯ ∈ IN0 |∃i, j ∈ N , i 6= j :m ¯ +ei, m¯ +ej ∈ S, q(m ¯ +ei, m¯ +ej) > 0}. (2.2) The transition rates for N can now be deﬁned. Assume that, for ψ : Sd → IR+, φ : S → IR+, p : S × S → IR+ ∪ {0}, d pij(¯n − ei) ≡ p(¯n, n¯ − ei + ej),n ¯ − ei ∈ S , i, j ∈ N , the transition rates for the primal process have the form ψ(¯n − e ) q(¯n, n¯−e +e ) = i p (¯n−e ), n,¯ n¯−e +e ∈ S, i, j ∈ N . (2.3) i j φ(¯n) ij i i j

The general formulation of the transition rates, (2.3), does not give rise to restrictions additional to (2.1) on the transition rates. The form (2.3) is chosen in agreement with the literature (cf. Chapter 2) and expresses a decomposition of the transition rates into a service part, ψ/φ, and a routing part, pij. Here pij(m ¯ ) is an arbitrary function and the argument, m¯ , of pij(m ¯ ) explicitly states that a transitionm ¯ + ei → m¯ + ej virtually passes through the dual statem ¯ . In the sequel, strong restrictions will be imposed on pij (cf. (2.9)). The functions ψ and φ are not subject to any further restrictions. 5.2 The primal process 79

The following theorem provides a suﬃcient condition for N to have a product-form invariant measure, that is a non-negative solution, m = (m(¯n), n¯ ∈ S), to the global balance equations

X {m(¯n)q(¯n, n¯0) − m(¯n0)q(¯n0, n¯)} = 0, n¯ ∈ S. (2.4) n¯06=¯n

As transitionsn ¯ → n¯−ei+ej are allowed only, the global balance equations can be written forn ¯ ∈ S

X {Φ(¯n)q(¯n, n¯ − ei + ej) − Φ(¯n − ei + ej)q(¯n − ei + ej, n¯)} = 0. (2.5) i,j∈N

If a solution, Φ, exists to the local balance equations for all i ∈ N , n¯ ∈ S,

X {Φ(¯n)q(¯n, n¯ − ei + ej) − Φ(¯n − ei + ej)q(¯n − ei + ej, n¯)} = 0, (2.6) j∈N then Φ is an invariant measure as can easily be seen by summing (2.6) over i. In Theorem 2.2 below, local balance of this form is expressed by the state-dependent traﬃc equations (2.9). However, local balance can also be expressed as follows by considering (2.5) for each j ∈ N separately. If for all j ∈ N , n¯ ∈ S, a solution Ψ exists to

X {Ψ(¯n)q(¯n, n¯ − ei + ej) − Ψ(¯n − ei + ej)q(¯n − ei + ej, n¯)} = 0, (2.7) i∈N then Ψ is an invariant measure as can be seen by summing (2.7) over j. A solution to (2.6), however, does not necessarily correspond to a solution to (2.7). In Section 3, (2.7) expresses local balance for the dual process, whereas in this section, (2.6) expresses local balance for the primal process. The result of Theorem 2.2 is well-known. For example, for slightly diﬀerent forms of the transition rates this result is obtained [9], [40], [55], [76], [87]. However, as terms appearing in the proof of this theorem give a ﬁrst glance at the dual process and the proof shows the symmetry between N and the dual process introduced in Section 3, both Theorem 2.2 and its proof are given here explicitly.

Theorem 2.2 N allows an invariant measure Φ at S, given by

N Y nk Φ(¯n) = φ(¯n) ck , n¯ ∈ S, (2.8) k=1 80 Dual processes

N if for all n¯ ∈ S the coeﬃcients {ci}i=1 are a non-negative (ci ≥ 0 for all i) solution of

X {γipij(¯n − ei) − γjpji(¯n − ei)} = 0, i ∈ N , γ0 = 1. (2.9) j∈N

Proof Insertion of (2.3) and (2.8) into the global balance equations at S gives forn ¯ ∈ S

X {Φ(¯n)q(¯n, n¯0) − Φ(¯n0)q(¯n0, n¯)} n¯06=¯n ( N X Y nk ψ(¯n − ei) = φ(¯n) ck pij(¯n − ei) i,j∈N k=1 φ(¯n) N ) Y nk−δki+δkj ψ(¯n − ei) − φ(¯n − ei + ej) ck pji(¯n − ei) k=1 φ(¯n − ei + ej) N X Y nk−δki X = ψ(¯n − ei) ck {cipij(¯n − ei) − cjpji(¯n − ei)} (2.10) i∈N k=1 j∈N = 0, where the last equality is obtained from (2.9). 2

At ﬁrst glance, (2.9) does not give rise to major restrictions on the routing function pij. However, note that pij(¯n − ei) ≡ p(¯n, n¯ − ei + ej) and p : S × S → IR+ ∪ {0}. Therefore, (2.9) implicitly assumes that, for all j N in the summation,n ¯ − ei + ej ∈ S. Moreover, the coeﬃcients {ci}i=1 are state-independent. This restricts blocking of transitions to the following cases, covering a fairly wide class of practical blocking examples, where pij(m ¯ ) = λijbij(m ¯ ).

N Reversible blocking: the solution {ci}i=1 to (2.9) satisﬁes ciλij = cjλji, N i, j ∈ N and bij(m ¯ ) = bji(m ¯ ) for allm ¯ ∈ IN0 such that bothm ¯ + ei N andm ¯ + ej ∈ S, where S is an arbitrary subset of IN0 .

N Indicator blocking: the solution {ci}i=1 to (2.9) satisfies the state-inde- P pendent traffic equations j∈N {ciλij − cjλji} = 0, i ∈ N , and bij or S satisfies some very special conditions, such as S is coordinate N PN convex, i.e. S = {n¯ ∈ IN0 | j=1 nj ≤ M} or bij is such that the stop- N protocol is satisfied, i.e. if S = {n¯ ∈ IN0 |nj ≤ Nj, j = 1,...,N} and mk = Nk then bij(m ¯ ) = 0 for all i (cf. Example 6.2). 5.2 The primal process 81

The following theorem considers the expected rate of transitions between a pair of stations. As in the previous theorem, both Theorem 2.3 and its proof contain terms appearing in the dual process. In order to give + a proper formulation of this expected value, deﬁne qij : S → IR ∪ {0}, i, j ∈ N as qij(¯n) = q(¯n, n¯ − ei + ej), n¯ ∈ S. Theorem 2.3 If Φ as given in (2.8) satisﬁes

X Φ(¯n) = B−1 < ∞, n¯∈S then under the distribution

π(¯n) = BΦ(¯n), n¯ ∈ S, for any i, j ∈ N the expected rate of transitions from station i to station j, i 6= j, is given by X Eqij = B Ψ(m ¯ )cipij(m ¯ ), (2.11) m¯ ∈Sd where Ψ is deﬁned as

N Y mk d Ψ(m ¯ ) = ψ(m ¯ ) ck , m¯ ∈ S . (2.12) k=1 Proof Direct computation of the expectation value gives for any i, j ∈ N , i 6= j

X Eqij = q(¯n, n¯ − ei + ej)π(¯n) n¯∈S N X ψ(¯n − ei) Y nk = pij(¯n − ei)Bφ(¯n) ck n¯∈S φ(¯n) k=1 N X Y nk−δki = B ψ(¯n − ei) ck cipij(¯n − ei) n¯∈S k=1 X = B Ψ(m ¯ )cipij(m ¯ ), m¯ ∈Sij where, for i, j ∈ N , i 6= j,

N Sij = {m¯ ∈ IN0 |m¯ + ei, m¯ + ej ∈ S, pij(m ¯ ) > 0}. 82 Dual processes

Note that, since q(m ¯ + ei, m¯ + ej) > 0 if and only if pij(m ¯ ) > 0,

d [ S = Sij, i,j∈N which completes the proof. 2

The function Ψ : Sd → IR+ ∪ {0} deﬁned in (2.12) appears both in the proof of Theorem 2.2 and in Theorem 2.3. The right-hand side of (2.11) gives rise to the following interpretation of Ψ. If Ψ satisﬁes

h i−1 X Ψ(m ¯ ) = Bd < ∞, m¯ ∈Sd then under the distribution πd at Sd deﬁned by

πd(m ¯ ) = BdΨ(m ¯ ), m¯ ∈ Sd,

(2.11) can be written B Eq = X c p (m ¯ )πd(m ¯ ). ij Bd i ij m¯ ∈Sd

Eqij expresses the average probability ﬂow from station i to station j and pij(m ¯ ) is the probability that a customer routes from station i to station j through the dual statem ¯ . πd can thus be interpreted as the probability that a transition passes through the dual statem ¯ ∈ Sd, i.e. that a customer in transit observes statem ¯ ∈ Sd. This will be formalized in Section 5.2. The remaining part of this section gives a potential interpretation of the process N with transition rates (2.3) and invariant measure (2.8). This interpretation will be used when the dual process is introduced.

Potential-interpretation of the transition rates 2.4 In accordance with [55], [87], the transition rates (2.3) can be interpreted 0 as reflecting the potential difference between statesn ¯ andn ¯ =n ¯ − ei + ej. To this end, define the global or configuration-potential U representing the potential of configurations at S and configuration-potential V representing the potential of configurations at Sd. In the configurationm ¯ ∈ Sd one customer has been released into the queueing network. This customer may influence the potential of the configuration, for example, it may “encour- age” customers at the stations to leave the station, or it may “discourage” 5.3 Definition of the dual process 83 customers to leave the station. Therefore, in general, U 6= V . The intensity at which a customer is released from station i in staten ¯ ∈ S is a function of the potential difference of the configurationsn ¯ andn ¯ − ei, i.e. this intensity is a function of V (¯n − ei) − U(¯n). Once a customer is released, the routing of the customer is determined by the site-potentials of the stations and not by the potential difference of the configurations. To this end, define Dij(m ¯ ), the potential difference between station i and station j in configurationm ¯ . The potential difference between statesn ¯ andn ¯ − ei + ej is then given by V (¯n − ei) − U(¯n) − Dij(¯n − ei). Define

φ(¯n) = exp [−U(¯n)] , n¯ ∈ S,

ψ(m ¯ ) = exp [−V (m ¯ )] , m¯ ∈ Sd, d pij(m ¯ ) = exp [Dij(m ¯ )] , m¯ ∈ S , i, j ∈ N , (2.13) then the transition rates have the form (2.3). Furthermore, Φ can now be written " N # X Φ(¯n) = exp −U(¯n) − ζknk , n¯ ∈ S, k=1 where ζi = − log ci, i ∈ N , can be regarded as local or site-potentials. However, as is stated in the literature (cf. [87]), this interpretation is valid only if N is reversible since the site potentials must satisfy

d −ζi + Dij(m ¯ ) = −ζj + Dji(m ¯ ), m¯ ∈ S , i, j ∈ N .

In Sections 3 and 4 this potential-interpretation is used to construct the transition rates for the dual process.

3 Deﬁnition of the dual process

For the primal process described in Section 2, this section defines the dual process. The dual process is a stochastic process at the dual state space Sd. In a transition of the dual process first a unit is created at a station and subsequently a unit is deleted at another station of the queueing network. The transition rates of the dual process are defined based on the transition structure of the primal process. Here a transition of the dual process corresponds to a transition between intermediate states of transitions for the primal process as depicted in Figure 5.1. To this end, observe that a transitionn ¯ → n¯ − ei + ej for N virtually passes through d the dual staten ¯ − ei ∈ S . Consider two successive transitions for the 84 Dual processes

primal process, sayn ¯ → n¯ − ei + ej → n¯ − ei + ej − ek + el. As depicted in the upper half of Figure 5.1, these successive transitions virtually pass through the dual statesn ¯ − ei andn ¯ − ei + ej − ek and therefore induce a d d transition from dual staten ¯ − ei ∈ S to dual staten ¯ − ei + ej − ek ∈ S . This represents a transition for the dual process. The transition rates for the dual process (3.1) are defined by analogy with the transition rates for the primal process (2.3). In a transition n¯ → n¯ − ei + ej for the primal process N, first the potential difference V (¯n − ei) − U(¯n) must be overcome. Then, in staten ¯ − ei, a customer is routed among the stations according to the site-potential difference Dij(¯n − ei). A transitionn ¯ − ei → n¯ − ei + ej − ek for the dual process virtually passes throughn ¯ − ei + ej. In this transition, first the global potential difference U(¯n − ei + ej) − V (¯n − ei) must be overcome, then, in d staten ¯ − ei + ej, according to the site potential difference Dkj(¯n − ei + ej) the process reaches staten ¯ − ei + ej − ek. From this construction, the transition rates for the dual process are given by (3.1) below. The symmetry between the primal process and the dual process is symbolized in Figure 5.1. The lower half of this figure is a mirror-image of the upper half. Upwards transitions in both the lower and upper half correspond to the global potential difference and downwards transitions correspond to the difference in site-potential. In each transition for the dual process a single unit can change stations only. Therefore, the dual transition rates satisfy (2.1). In addition it is assumed that the dual process defined below is a stable, regular, continuous-time Markov chain. In contrast to the primal process, a transitionm ¯ → m¯ + ej − ei for the dual process virtually passes through state m¯ + ej. This is explicitly visualized in the notation by writingm ¯ + ej − ei instead ofm ¯ − ei + ej which is the usual notation for the primal process. Definition 3.1 (Dual process) For a stochastic process at S with transition rates given in (2.3), the dual process is the continuous-time Markov chain N d = {N d(t), t ≥ 0} at Sd with transition rates qd given by φ(m ¯ + e ) qd(m, ¯ m¯ +e −e ) = j pd (m ¯ +e ), m,¯ m¯ +e −e ∈ Sd, i, j ∈ N , j i ψ(m ¯ ) ij j j i (3.1) d d d d d d where pij(m ¯ +ej) ≡ p (m, ¯ m¯ +ej −ei), m¯ ∈ S , i, j ∈ N and p : S ×S → IR+ ∪ {0}. d The dual routing function pij is not directly related to the primal routing d function pij. The relation between pij and pij depends on the interpreta- 5.3 Definition of the dual process 85

d tion of the dual process as is shown in Section 4. A few remarks on pij d can be made here. Firstly, note that pij is not required to be a probability distribution. Secondly, the dual routing function, in general, cannot d satisfy pjj(¯n) = 0. This can easily be seen by observing Figure 5.1. If for the primal process a customer routes from station i to station j and next a customer routes from station j to station l, i.e. k = j in Figure 5.1, then for the dual process a transitionm ¯ → m¯ occurs. Therefore, qd(m, ¯ m¯ ) > 0 and thus pjj(¯n) > 0. Similar to Theorem 2.2, a suﬃcient condition for N d to possess a product form invariant measure can now be given. The proof of this result is a direct analogue to the proof of Theorem 2.2 and shows the symmetry between the primal and dual process.

Theorem 3.2 N d allows an invariant measure Ψ at Sd, given by

N Y mk d Ψ(m ¯ ) = ψ(m ¯ ) dk , m¯ ∈ S , (3.2) k=1

d N if for all m¯ ∈ S the coeﬃcients {di}i=1 are a positive (di > 0 for all i) solution of ( ) X 1 d 1 d pij(m ¯ + ej) − pji(m ¯ + ej) = 0, i ∈ N , γ0 = 1. (3.3) i∈N γj γi

Proof Insertion of (3.1) and (3.2) into the global balance equations at Sd gives form ¯ ∈ Sd n o X Ψ(m ¯ )qd(m, ¯ m¯ 0) − Ψ(m ¯ 0)qd(m ¯ 0, m¯ ) m¯ 06=m ¯ ( N X Y mk φ(m ¯ + ej) d = ψ(m ¯ ) dk pij(m ¯ + ej) i,j∈N k=1 ψ(m ¯ ) N ) Y mk+δkj −δki φ(m ¯ + ej) d − ψ(m ¯ + ej − ei) dk pji(m ¯ + ej) k=1 ψ(m ¯ + ej − ei) N ( ) X Y mk+δkj X 1 d 1 d = φ(m ¯ + ej) dk pij(m ¯ + ej) − pji(m ¯ + ej) j∈N k=1 i∈N dj di = 0, where the last equality is obtained from (3.3). 2 86 Dual processes

d Since the dual routing function pij is not explicitly related to the pri- N mal routing function, the coefficients {di}i=1 for the dual process are not N d related to the coefficients {ci}i=1 for the primal process. In Section 4, pij 1 will be chosen such that di = ci, i = 1,...,N, or di = , i = 1,...,N. ci In the first case the dual process is said to describe customers and in the second case the dual process describes vacancies or holes. Similar to Theorem 2.3, the expected rate of transition from station i to station j can now be computed for the dual process. To this end, define d d + qij : S → IR ∪ {0}, i, j ∈ N , as

d d d qij(m ¯ ) = q (m, ¯ m¯ + ej − ei), m¯ ∈ S . Again, the symmetry between the dual process and the primal process is illustrated by Theorem 3.3. Theorem 3.3 If Ψ as given in (3.2) satisﬁes h i−1 X Ψ(m ¯ ) = Bd < ∞, m¯ ∈Sd then under the distribution πd(m ¯ ) = BdΨ(m ¯ ), m¯ ∈ Sd, for any i, j ∈ N , i 6= j,

d d X 1 d Eqij = B Φ(¯n) pij(¯n), (3.4) n¯∈S− dj

− N where S ⊂ IN0 is deﬁned as − d d S = {n¯|∃i, j ∈ N , i 6= j :n ¯−ei, n¯−ej ∈ S , q (¯n−ej, n¯−ei) > 0}. (3.5) Proof Direct computation of the expectation value gives for any i, j ∈ N

d X d d Eqij = q (m, ¯ m¯ + ej − ei)π (m ¯ ) m¯ ∈Sd N X φ(m ¯ + ej) Y = pd (m ¯ + e )Bdψ(m ¯ ) dmk ψ(m ¯ ) ij j k m¯ ∈Sd k=1 N 1 = Bd X φ(m ¯ + e ) Y dmk+δkj pd (m ¯ + e ) j k d ij j m¯ ∈Sd k=1 j

d X 1 d = B Φ(¯n) pij(¯n), d dj n¯∈Sij 5.4 The dual routing function 87 where, for i, j ∈ N , i 6= j,

d N d d Sij = {n¯ ∈ IN0 |n¯ − ei, n¯ − ej ∈ S , pij(¯n) > 0}. Note that − [ d S = Sij i,j∈N which completes the proof. 2

The set S− defined in (3.5) is the set of intermediate states for the transitions of the dual process. From the definition of the dual process, d − q (¯n − ej, n¯ − ei) is defined only ifn ¯ ∈ S. Therefore, S ⊆ S. In general, S− is a strict subset of S, however, when the queueing network is open and unbounded, it may be the case that S− = S (cf. (4.2) for M = ∞). The forward dualizing method is the method used above to construct the dual process N d from the primal process N. The state space Sd of the dual process consists of the intermediate states for the transitions of N and the transition rates for N d are obtained via the potential interpretation. It is possible to define a backward dualizing method in several ways. The most direct method is the following first backward dualizing method, which has the same structure as the forward dualizing method. The backward dual process of N d is the process at S−, the set of intermediate states for the transitions of the dual process, with transition rates given by (2.3). In this case, the backward dual process N dd is not identical to the primal process unless S− = S. Without any knowledge of the primal process N, the first backward dual process can be constructed from N d. This backward dualizing method is discussed in Example 6.2. A different backward dualizing method is the following second backward dualizing method. A major drawback of this method is that it requires specific knowledge of the primal process. An advantage of this method is that it guarantees that the backward dual process is identical to the primal process, i.e. N dd = N. For a statem ¯ ∈ Sd, consider all states m¯ + ei, i ∈ N , such thatm ¯ + ei ∈ S. The second backward dual process N dd is the process with state space Sdd given by

dd N d S = {n¯ ∈ IN0 |∃m¯ ∈ S , i ∈ N such thatn ¯ =m ¯ + ei ∈ S}, and transition rates at Sdd given in (2.3). From the deﬁnition of the dual state space it is immediately clear that Sdd = S and therefore that N dd = N, unless S contains singletons (¯n ∈ S is called a singleton if for alln ¯0 ∈ S: q(¯n, n¯0) = q(¯n0, n¯) = 0). 88 Dual processes

4 The dual routing function

The dual process with transition rates (3.1) is related to the primal process with transition rates (2.3) through the transition structure and the functions ψ, φ only. In the definition of the dual process, the dual rout- d ing function pij is not directly related to the primal routing function pij. Therefore, Section 3 defines a collection of dual processes for a collection of primal processes. In this section, based on the potential-interpretation of the transition rates and the transition structure of the primal and dual d processes, the dual routing function pij is related to the primal routing function pij for specific site-potential-interpretations of the routing functions. In Examples 4.1 and 4.2 a state of the dual process is chosen to d represent a customer-configuration. In this case pij is obtained such that di = ci, i = 1,...,N. In Examples 4.3 and 4.4 a state of the dual process represents the configuration of vacancies or holes, and di = 1/ci, i = 1,...,N. Throughout this section, assume that the primal routing function is state-independent except for blocking effects at the boundary of the state space S, that is for allm ¯ ∈ Sd

pij(m ¯ ) = pij1{m¯ + ei, m¯ + ej ∈ S}, and, for simplicity, pii = 0 for all i ∈ N . When the queueing network is open assume that, for M ≤ ∞,Mi ∈ IN0, i = 1,...,N, N N X S = {n¯ ∈ IN0 | ni ≤ M, ni ≥ Mi, i = 1,...,N}. i=1 N This guarantees that the solution {ci}i=1 to the state-dependent traﬃc equations (2.9) can be obtained as the unique solution to the “standard” traﬃc equations N X {γipij − γjpji} = 0, i = 1, . . . , N, γ0 = 1. (4.1) j=0

For all i ∈ {1,...,N} there exists a j ∈ {0,...,N} such that pij > 0. Therefore, forn ¯ ∈ S such that ni > Mi the transitionn ¯ = (¯n − ei) + ei → (¯n − ei) + ej is possible. As a consequence the statesn ¯ − ei, i = 1,...,N, are dual states. This implies that the dual state space is N d N X S = {n¯ ∈ IN0 | ni ≤ M − 1, ni ≥ Mi, i = 1,...,N}. i=1 5.4 The dual routing function 89

d d Assume q (m, ¯ m¯ + ej − ei) > 0 for allm, ¯ m¯ + ej − ei ∈ S , i, j ∈ N . The state space S− for the ﬁrst backward dualizing method is obtained from d d P S as follows. Ifm ¯ ∈ S such that i mi < M − 1 then for some j ∈ N it d − must be thatm ¯ + ei − ej ∈ S . As a consequence,m ¯ + ei ∈ S since the d transitionm ¯ → m¯ + ei − ej passes throughm ¯ + ei. Ifm ¯ ∈ S such that P d i mi = M−1 then for some j ∈ N , j 6= 0, it must be thatm ¯ +ei−ej ∈ S . P The statesm ¯ +ei such that i mi = M−1, mi = Mi−1 cannot be obtained d as backward dual states since the transitionm ¯ → m¯ + ei would leave S . Therefore, the backward dual state space is

N − N X S = S \{n¯ ∈ IN0 | ni = M and ni = Mi for some i ∈ {1,...,N}}. i=1 (4.2) If M = ∞ then S− = S = Sd. However, as will be shown in Example 6.2, for the dual process to satisfy the dual traﬃc equations (3.3) at the lower boundary of Sd, in most examples, it must be assumed that qd(m, ¯ m¯ 0) = 0. Thus, in most cases, S− 6= S. When the queueing network is closed assume that for M < ∞,Mi ∈ IN0, i = 1,...,N,

N N X S = {n¯ ∈ IN0 | ni = M, ni ≥ Mi, i = 1,...,N} i=1

N This guarantees that the solution {ci}i=1 to (2.9) can be obtained as the unique solution to the traﬃc equations

N X {γipij − γjpji} = 0, i = 1,...,N. (4.3) j=1

The dual state space is given by

N d N X S = {n¯ ∈ IN0 | ni = M − 1, ni ≥ Mi, i = 1,...,N}. i=1

d d If q (m, ¯ m¯ + ej − ei) > 0 for allm, ¯ m¯ + ej − ei ∈ S , i, j ∈ N , then

− S = S \{n¯ ∈ S|ni = Mi for some i ∈ {1,...,N}} . (4.4)

Thus for closed queueing networks S− is a strict subset of S. 90 Dual processes

Under the assumptions made above, Theorem 2.2 implies that N allows an invariant measure Φ at S given by

N Y nk Φ(¯n) = φ(¯n) ck , n¯ ∈ S. k=1 The transition rates for the dual process will now be given for four different cases. In all of these cases, due to a refinement of the site-potential- d interpretation, the site-potential difference Dij for the dual process is related to the site-potential difference Dij for the primal process. This d relation is then used to express pij as function of pij. To this end, recall (2.13) and the definition ζi = − log ci, i = 1,...,N, as given in the potential-interpretation 2.4.

4.1 Similar transitions

This example compares a transitionn ¯ → n¯ − ei → n¯ − ei + ej for the primal process to a transitionm ¯ → m¯ + ej → m¯ + ej − ei for the dual process, as depicted in Figure 5.2, where in both cases a unit is transferred from station i to station j. Assume the following transition mechanism for the primal process. First, a customer is released from station i according to the global potential difference, but this customer remains in the vicinity of station i. The global state is changed inton ¯ − ei, representing the customer-configuration of the customers remaining at the stations, with one customer released into the queueing network. Second, the customer is released from the vicinity of station i and routed to station j according to the site-potential difference Dij(¯n − ei). From the transition mechanism above, it is natural that a state for the dual process represents the customer-configuration at the stations of the network, but with one customer that is not attached to a specific station, i.e. a customer in transit from one station to another. This gives the following transition mechanism for the dual process. Firstly, according to the global potential difference, the free moving customer is captured in the vicinity of sta- d tion j. Secondly, according to the site-potential difference Dij(m ¯ + ej) a customer is released from station i and the customer near station j ac- tually enters the station. In the new state,m ¯ + ej − ei, one customer is moving freely among the stations. Note that, for both the primal and the dual process, a customer is routed from station i to station j. However, for the primal process a customer must overcome the site-potential ζi for leaving the vicinity of station i to become a free moving customer that 5.4 The dual routing function 91

n¯ − 2ei + ej n¯ − ei %%@ r primal process % r @ % @@R % rn¯ − 2ei + 2ej rn¯ − ei + ej rn¯

m¯ + ei m¯ + ej m¯ + 2ej − ei r e r e r dual process e m¯ =n ¯ − e Ree i rm¯ rm¯ + ej − ei Figure 5.2. Similar transitions for the primal and the dual process can be routed to station j, whereas for the dual process a customer must overcome the site-potential −ζj for entering station j and thus allowing a customer to be released from station i. Therefore, the following relation for the site-potential diﬀerence for the primal and dual process can be deduced: d Dij(m ¯ + ej) = −ζi − ζj + Dij(m ¯ ), i, j ∈ N . (4.5) For the routing function this implies

d − pij(¯n) = cicjpij, n¯ ∈ S , i, j ∈ N . (4.6)

N If the primal process is reversible, i.e. the solution {ci}i=1 to the traﬃc equations (4.1), (4.3) satisﬁes

cipij = cjpji, i, j ∈ N , then for the dual routing function defined in (4.6), where S− is given N in (4.2), (4.4), the coefficients {ci}i=1 solve (3.3). Therefore, Theorem 3.2 implies that the dual process allows an invariant measure Ψ at Sd given by N Y mk d Ψ(m ¯ ) = ψ(m ¯ ) ck , m¯ ∈ S . (4.7) k=1 However, if the primal process is non-reversible, for the dual process to possess a positive solution to (3.3), the dual routing function (4.6) must 92 Dual processes be modified to ( c c p , if n > M , k = 1, . . . , N, i, j ∈ N , pd (¯n) = i j ij k k (4.8) ij 0, otherwise. When the primal process represents a closed queueing network (4.8) is identical to (4.6) and S− is given in (4.4). However, when the primal process represents an open queueing network (4.8) corresponds to (4.6) with S− as given in (4.2) replaced by

N − N X S = {n¯ ∈ IN0 | ni ≤ M, nk > Mk, k = 1,...,N}. (4.9) i=1 d N For pij as given in (4.8) the coefficients {ci}i=1 solve (3.3) and the dual process allows an invariant measure Ψ as given in (4.7) above. Note that (4.7) is in agreement with the interpretation of the states of the dual process given above since (4.7) expresses the potential of the customer- configurationm ¯ . d The modification (4.8) of the dual routing function pij as given in (4.6) for a non-reversible primal process is not in conflict with the derivation of (4.6) since the potential-interpretation of the transition rates is valid if the process is reversible only. The blocking protocol introduced in (4.8) is the protocol dual to the stop-protocol for primal processes and will be discussed in more detail in Example 6.2. An interesting observation for routing functions related through (4.6), i.e. for a reversible primal process, is stated in the following lemma, where the probability flow from station i to station j for the primal and the dual process is compared. As both the primal and the dual process describe the evolution of customer-configurations in a queueing network, it seems natural to assume that the probability flow between stations for the primal process and the dual process must match. In many examples, however, this may not be the case. Lemma 4.1 If S− = S and (4.6) holds, then

d Eqij = Eqij, i, j ∈ N , if and only if N N X Y nk X Y mk φ(¯n) ck = ψ(m ¯ ) ck , n¯∈S k=1 m¯ ∈Sd k=1 i.e. B = Bd for the appropriate deﬁnition of the normalizing constants for the primal and the dual process. 5.4 The dual routing function 93

Proof If (4.6) holds, then comparison of (2.11) and (3.4) gives

d Eqij = Eqij, i, j ∈ N , if and only if 1 B X Ψ(m ¯ )c p (m ¯ ) = Bd X Φ(¯n) pd (¯n), i ij c ij m¯ ∈Sd n¯∈S− j with B and Bd deﬁned in Theorem 2.3 and Theorem 3.2, respectively. 2

n¯ − 2ei + ej n¯ − ei %%@I r r % @ primal process % @@ %© rn¯ − 2ei + 2ej rn¯ − ei + ej rn¯

m¯ + ei m¯ + ej m¯ + 2ej − ei r e r e r dual process e m¯ =n ¯ − e Ree i rm¯ rm¯ + ej − ei Figure 5.3. Similar subtransitions for the primal and the dual process

4.2 Similar subtransitions

In the previous example a transitionm ¯ → m¯ +ej −ei for the dual process is compared to a transitionn ¯ → n¯ −ei +ej for the primal process. Although this seems to be a natural way to obtain the routing function for the dual d process, the subtransitions corresponding to Dij and Dij do not match. In fact, in the previous example, the subtransitionn ¯ − ei → n¯ − ei + ej, where a customer for the primal process enters station j, is compared to the subtransitionm ¯ + ej → m¯ + ej − ei, where a customer for the dual process leaves station i. As can be seen from Figure 5.3, the subtransition corresponding tom ¯ + ej → m¯ + ej − ei for the dual process is the subtransitionn ¯ − ei + ej → n¯ − 2ei + ej for the primal process. However, this subtransition cannot occur as a consequence of a site-potential diﬀerence for the primal process, since upwards subtransitions correspond to a global 94 Dual processes

potential diﬀerence. Therefore, the subtransitionm ¯ + ej → m¯ + ej − ei as a part of the transitionm ¯ → m¯ + ej → m¯ + ej − ei for the dual process is compared to the subtransitionn ¯ − 2ei + ej → n¯ − ei + ej as a part of the transitionn ¯ − 2ei + 2ej → n¯ − 2ei + ej → n¯ − ei + ej for the primal process. Similar to Example 4.1, in the primal process a customer must overcome the site-potential ζj for leaving the vicinity of station j, whereas in the dual process a customers must overcome the site-potential −ζj for entering station j. This implies for the site-potential diﬀerences, that

d Dij(m ¯ + ej) = −2ζj + Dji(¯n − 2ei + ej), i, j ∈ N , which results in the following relation for the routing functions:

d 2 − pij(¯n) = cj pji, n¯ ∈ S , i, j ∈ N . (4.10) If the primal process is reversible (4.10) is equivalent to (4.6), however, if the primal process is non-reversible the modiﬁed version of (4.10)

( c2p , if n > M , k = 1, . . . , N, i, j ∈ N , pd (¯n) = j ji k k (4.11) ij 0, otherwise, cannot be identiﬁed with (4.8). Therefore, in my opinion, the derivation of both (4.8) and (4.11) via reversible processes is justiﬁed. Similar to N the previous example, {ci}i=1 solves (3.3), thus Ψ as given in (4.7) is an invariant measure for the dual process.

4.3 Similar transitions; reversed potential As a direct consequence of the transition mechanism for the primal process, the previous examples assume that a state for the dual process represents a customer-configuration. However, a state for the dual process may also be interpreted as representing the configuration of vacancies. To this end, reconsider the transition mechanism for the primal process as described in Example 4.1 and Figure 5.2. In a transition n¯ → n¯ − ei → n¯ − ei + ej first a customer leaves station i, or equivalently a vacancy is created at station i. For the dual process, in a transition m¯ → m¯ + ej → m¯ + ej − ei first a unit is created at station j. There- fore, it seems natural to identify a unit in the dual process with a vacancy or hole in the primal process. As the site-potential of a vacancy will be the reversed of the site-potential of a customer, in the site-potential interpretation the identification of a unit for the dual process to a vacancy 5.4 The dual routing function 95

corresponds to an additional site-potential ζj instead of the −ζj in the derivation of (4.5) in Example 4.1. If, when routing, a unit for the dual process behaves similar to a unit for the primal process, the site-potential relation (4.5) must be replaced by

d Dij(m ¯ + ej) = −ζi + ζj + Dij(m ¯ ), i, j ∈ N .

For the routing function this implies

d ci − pij(¯n) = pij, n¯ ∈ S , i, j ∈ N . cj Similar to Example 4.1 for a non-reversible primal process the dual routing function must be modiﬁed to

 ci d  pij, if nk > Mk, k = 1, . . . , N, i, j ∈ N , pij(¯n) = cj  0, otherwise.

d 1 By insertion of the dual routing function p into (3.3), di = , i = ij ci 1,...,N, is a solution to (3.3). Therefore, Theorem 3.2 implies that the dual process allows an invariant measure Ψ at Sd given by

N 1 mk Ψ(m ¯ ) = ψ(m ¯ ) Y , m¯ ∈ Sd. (4.12) k=1 ck

4.4 Similar subtransitions; reversed potential By analogy with Example 4.2, the subtransitions depicted in Figure 5.3 may be compared in the case described in Example 4.3. If the primal process is reversible, this implies for the dual routing function

d − pij(¯n) = pji, n¯ ∈ S , (4.13) where S− is defined in (4.2), (4.4). If the primal process is non-reversible, S− as specified above must, in the open network case, be modified to (4.9). d d For pij thus defined the dual process allows an invariant measure Ψ at S given in (4.12). Note that (4.12) is in agreement with the interpretation of the states of the dual process since (4.12) expresses the potential of a vacancy configurationm ¯ . The relation (4.13) between the primal routing function and the dual routing function may also be interpreted directly as follows. If a customer 96 Dual processes routes from station j to station i in the primal process then an additional vacancy is created at station j and a vacancy is deleted at station i, i.e. a vacancy routes from station i to station j. Therefore, the dual routing function must be related to the primal routing function as given in (4.13). A natural analogue of Lemma 4.1 is the following lemma where the probability flow for a reversible primal process is related to the probability flow for the dual process. It seems natural to assume that the probability flow for customers in the primal process and the probability flow for va- d cancies in the dual process must match, i.e. to assume that Eqij = Eqji. In many examples, however, this may not be the case.

Lemma 4.2 If S− = S and (4.13) holds, then

d Eqij = Eqji, i, j ∈ N , if and only if

N N m X Y X Y 1 k φ(¯n) cnk = ψ(m ¯ ) , k c n¯∈S k=1 m¯ ∈Sd k=1 k i.e. B = Bd for the appropriate deﬁnition of the normalizing constants for the primal and the dual process.

Proof Similar to the proof of Lemma 4.1. 2

5 Relation between the primal and the dual process

For the primal process introduced in Section 2, a collection of dual processes is introduced in Section 3. Based on a reﬁnement of the potential- interpretation of the transition rates, Section 4 gives some examples of dual processes with transition rates completely determined by the transition rates of the primal process. Although the dual process is completely determined by the primal process, in general, the primal process and corresponding dual process do not describe the same physical system (queueing network). The following deﬁnition of primal and dual processes may therefore be considered.

A primal process is a process for which a transitionn ¯ → n¯ − ei + ej passes throughn ¯ − ei, i.e., ﬁrst a unit is deleted at station i and then a unit is created at station j. 5.5 Relation between the primal and the dual process 97

A dual process is a process for which a transitionn ¯ → n¯−ei+ej passes throughn ¯ + ej, i.e., ﬁrst a unit is created at station j and then a unit is deleted at station i. A dual process corresponding to a given primal process is a dual process with transition rates completely determined by the primal process, and a primal process corresponding to a given dual process is a primal process with transition rates completely determined by the dual process.

The dual processes given in Section 4 are examples of dual processes corresponding to the primal process given in Section 2. In practical cases it may be desirable for the dual process corresponding to a given primal process to describe the same physical system. The problem of average probability ﬂow between stations being identical is considered in Lemma 4.1 and Lemma 4.2. For a dual process to correspond to a primal process the most obvious condition is that the probability ﬂow between states is identical. This problem is considered in Section 5.1 below. In Section 2 it is argued that the equilibrium distribution of the dual process represents the probability that a transition for the primal process passes through a dual state. This will be formalized in Section 5.2, where the Palm probabilities associated with the jumps of the primal process are studied. Finally, the results of Section 5.1 and Section 5.2 are combined in the last part of Section 5.2, where a theorem similar to the arrival theorem is proven. Throughout this section, assume that the primal process N is stationary and irreducible with unique stationary distribution π at S. Fur- N thermore, assume that a positive solution {ci}i=1 to (2.9) exists, where P d j∈N pij(m ¯ ) = 1, for all i ∈ N ,m ¯ ∈ S .

5.1 Equal probability flow For a dual process to correspond to a primal process, the most obvious condition on the transition rates of the dual process is that the probability flow between states of the dual process is induced by the probability flow between states of the primal process. To this end, reconsider Figure 5.1. If the dual process possesses an equilibrium distribution πd, then the probability flow from statem ¯ to statem ¯ + ej − ek for the dual process is given by d d π (m ¯ )q (m, ¯ m¯ + ej − ek). 98 Dual processes

For the primal process, this probability flow is induced by the probability flow from statem ¯ + ei to statem ¯ + ej − ek + el. Thus, the probability flow from statem ¯ to statem ¯ + ej − ek in the primal process is given by X π(m ¯ + ei)q(m ¯ + ei, m¯ + ej)q(m ¯ + ej, m¯ + ej − ek + el). i,l∈N

As the global part of the transition rates for the dual process is determined d in the deﬁnition, (3.1), the dual routing function pij must be determined such that these ﬂows match.

Theorem 5.1 The dual routing function is for m,¯ m¯ + ej − ei ∈ Sd, i, j ∈ N given by

 N mk+δkj  B ψ(m ¯ )ψ(m ¯ + ej − ei) Y ck  dj  d 2  B φ(m ¯ + ej) k=1 dk d  pij(m ¯ + ej) = if ∃r, s ∈ N   such that p (m ¯ ) > 0, p (m ¯ + e − e ) > 0,  rj is j i  0, otherwise, (5.1) where di > 0, i = 1,...,N, and

N −1 h di X Y mk 0 < B = ψ(m ¯ ) dk < ∞, (5.2) m¯ ∈Sd k=1 if and only if the dual process is an irreducible Markov chain with unique equilibrium distribution πd at Sd given by

N d d Y mk d π (m ¯ ) = B ψ(m ¯ ) dk , m¯ ∈ S , (5.3) k=1

d d and the probability ﬂow from state m¯ ∈ S to state m¯ + ej − ek ∈ S is the same for both the dual process and the primal process for all m¯ ∈ Sd, j, k ∈ N , i.e.

d d π (m ¯ )q (m, ¯ m¯ + ej − ek) X = π(m ¯ + ei)q(m ¯ + ei, m¯ + ej)q(m ¯ + ej, m¯ + ej − ek + el).(5.4) i,l∈N

0 d Proof Form, ¯ m¯ ∈ S there exist i0, i1, j0, j1 such thatm ¯ + ei0 , m¯ + 0 0 0 ei1 , m¯ + ej0 , m¯ + ej1 ∈ S, and such that q(m ¯ + ei0 , m¯ + ei1 ) > 0, q(m ¯ + 5.5 Relation between the primal and the dual process 99

0 ej0 , m¯ + ej1 ) > 0. As N is irreducible, there exists a sequence of states n¯0, n¯1, n¯2,..., n¯k−2, n¯k−1, n¯k ∈ S such that q(¯ni, n¯i+1) > 0, i = 0, . . . , k−1, 0 0 andn ¯0 =m ¯ + ei0 ,n ¯1 =m ¯ + ei1 ,n ¯k−1 =m ¯ + ej0 ,n ¯k =m ¯ + ej1 . Deﬁne + + m¯ i =n ¯i−1 − (¯ni−1 − n¯i) , i = 1, . . . , k − 1, wherev ¯ denotes the vec- d torv ¯ with all non-positive entries replaced by zero. Thenm ¯ i ∈ S , i = d d 1, . . . , k−1. From (5.1), pij(m ¯ +ej) > 0 for all i, j ∈ N , m,¯ m¯ +ej −ei ∈ S . d d This implies for the sequencem ¯ 1,..., m¯ k−1 ∈ S that q (m ¯ i, m¯ i+1) > 0, 0 d i = 1, . . . , k−1. Sincem ¯ =m ¯ 1 andm ¯ =m ¯ k−1 this implies that N is irre- N ducible. Insertion of (5.1) into (3.3) gives that {di}i=1 is a positive solution of (3.3). Therefore, from Theorem 3.2 and (5.2), πd as given in (5.3) is the unique equilibrium distribution of the dual process. From Theorem 2.2, QN nk d d π(¯n) = Bφ(¯n) k=1 ck . Insertion of π, π and the transition rates q, q d into (5.4), immediately gives in the non-trivial case (pij(m ¯ + ej) > 0) for the right-hand side of (5.4) X π(m ¯ + ei)q(m ¯ + ei, m¯ + ej)q(m ¯ + ej, m¯ + ej − ek + el) i,l∈N N (2.3),(2.8) X Y mk+δki ψ(m ¯ + ej − ek) = B ck ψ(m ¯ )pij(m ¯ ) pkl(m ¯ + ej − ek) i,l∈N k=1 φ(m ¯ + ej) N X Y mk ψ(m ¯ + ej − ek) = cipij(m ¯ )Bψ(m ¯ ) ck i∈N k=1 φ(m ¯ + ej) N (2.9) Y mk+δkj ψ(m ¯ + ej − ek) = Bψ(m ¯ ) ck , k=1 φ(m ¯ + ej) and for the left-hand side of (5.4)

d d π (m ¯ )q (m, ¯ m¯ + ej − ek) N d Y mk φ(m ¯ + ej) d = B ψ(m ¯ ) dk pkj(m ¯ + ej) k=1 ψ(m ¯ )

N N mk+δkj (5.1) Y mk ψ(m ¯ + ej − ek) Y ck = Bψ(m ¯ ) dk dj, k=1 φ(m ¯ + ej) k=1 dk which immediately establishes (5.4). To prove the reversed statement, ﬁrst note that (5.3) implies (5.2). d Second, if form, ¯ m¯ + ej − ek ∈ S it is not the case that ∃r, s ∈ N such that prj(m ¯ ) > 0 and pks(m ¯ + ej − ek) > 0 then the right-hand side of (5.4) equals zero. Thus, since πd(m ¯ ) > 0, ψ, φ > 0, it must be the case that d d pkj(m ¯ + ej) = 0. Third, if such r, s ∈ N do exist then insertion of π, π , 100 Dual processes q as given in (2.3) and qd as given in (3.1) into (5.4) gives, similar to the derivation of (5.4) above, that the dual routing function has the form given in (5.1). 2

If the dual routing function has the form (5.1), then the transition rates of the dual process are

N mk+δkj d B ψ(m ¯ + ej − ei) Y ck q (m, ¯ m¯ + ej − ei) = d dj. B φ(m ¯ + ej) k=1 dk From the viewpoint of equal probability ﬂow, it seems more logical to use this form as a deﬁnition for the dual transition rates, i.e.

d ψ(m ¯ + ej − ei) d q (m, ¯ m¯ + ej − ei) = pij(m ¯ + ej). (5.5) φ(m ¯ + ej) In Section 3 the dual process is introduced via the potential-interpretation of the transition rates. In this case, it is more natural to give the form (3.1) for the transition rates for the dual process as this form explicitly states that a dual transition from statem ¯ to statem ¯ + ej − ei passes through statem ¯ + ej. This makes the relation between the primal process and the dual process a symmetrical relation as can be seen when comparing Theo- rem 2.2 to Theorem 3.2 and Theorem 2.3 to Theorem 3.3. The form (5.5) appears as a consequence of the speciﬁc assumption (5.4) guaranteeing equal probability ﬂow between states. A theorem similar to Theorem 5.1 above in which the primal routing function is related to the dual routing function for a given form of the dual routing function can easily be formulated.

5.2 Palm probabilities Each type of transitionn ¯ → n¯0 (¯n 6=n ¯0) for N can be associated with a subset H of S × S \ diag(S × S). For example, a transition in which a customer moves from station i to station j corresponds to [ Hij = {(m ¯ + ei, m¯ + ej), m¯ + ei, m¯ + ej ∈ S} , i, j ∈ N , (5.6) m¯ ∈Sd

Let NH be the process counting the H-transitions of N. Assume that (5.2) holds for di = ci, i = 1,...,N, then for all H ⊆ S × S \ diag(S × S) 0 < X π(¯n)q(¯n, n¯0) < ∞, (¯n,n¯0)∈H 5.5 Relation between the primal and the dual process 101

and the Palm probability PH associated with NH can be deﬁned. The Palm probability of the event C given that an H-transition occurs is given by (cf. [2]) P 0 (¯n,n¯0)∈C π(¯n)q(¯n, n¯ ) PH (C) = P 0 ,C ⊆ H. (¯n,n¯0)∈H π(¯n)q(¯n, n¯ ) For example, the probability that the unmoved customers are in statem ¯ when a customer moves from station i to station j is given by Pij(m ¯ ) ≡

PHij ({m¯ + ei, m¯ + ej}). Direct computation gives the following theorem. Theorem 5.2 (Palm probabilities) 1. The probability that the unmoved customers are in state m¯ ∈ Sd when a customer moves from station i to station j, i, j ∈ N is given by

Ψ(m ¯ )pij(m ¯ ) d Pij(m ¯ ) = P , m¯ ∈ S , i, j ∈ N . (5.7) m¯ ∈Sd Ψ(m ¯ )pij(m ¯ )

2. The probability that the unmoved customers are in state m¯ ∈ Sd when a customer leaves station i or enters station i, i ∈ N , is given by Ψ(m ¯ ) d Pi(m ¯ ) = P , m¯ ∈ S , i ∈ N . (5.8) m¯ ∈Sd Ψ(m ¯ ) For convenience, the exterior is regarded as station 0, and

N Y mk Ψ(m ¯ ) = ψ(m ¯ ) ck . k=1 Proof The proof of the statements of the Theorem consists of specifying the appropriate sets H and C ⊆ H. 1. H represents the transitions in which a customer moves from station i to station j, i.e. H = Hij as given in (5.6) and C represents the transitionm ¯ +ei → m¯ +ej, i.e. C = Cij(m ¯ ) = {(m ¯ +ei, m¯ +ej)}. This gives

π(m ¯ + ei)q(m ¯ + ei, m¯ + ej) Pij(m ¯ ) = P m¯ ∈Sd π(m ¯ + ei)q(m ¯ + ei, m¯ + ej) Bψ(m ¯ ) QN cmk c p (m ¯ ) = k=1 k i ij P QN mk m¯ ∈Sd Bψ(m ¯ ) k=1 ck cipij(m ¯ ) Ψ(m ¯ )pij(m ¯ ) = P . m¯ ∈Sd Ψ(m ¯ )pij(m ¯ ) 102 Dual processes

2. If H represents the transitions in which a customer leaves station i and C represents the transitions in which a customer leaves station i S S in statem ¯ + ei, then H = j∈N Hij and C = j∈N Cij(m ¯ ). This immediately gives Pi(m ¯ ). If H represents the transitions in which a customer enters station i and C represents the transitions in which S a customer enters station i to statem ¯ + ei, then H = j∈N Hji and S C = j∈N Cji(m ¯ ). This immediately gives P j∈N π(m ¯ + ej)q(m ¯ + ej, m¯ + ei) Pi(m ¯ ) = P P j∈N m¯ ∈Sd π(m ¯ + ej)q(m ¯ + ej, m¯ + ei) ψ(m ¯ ) QN cmk P c p (m ¯ ) = k=1 k j∈N j ji P QN mk P m¯ ∈Sd ψ(m ¯ ) k=1 ck j∈N cjpji(m ¯ ) P (2.9) Ψ(m ¯ ) j∈N cipij(m ¯ ) = P P m¯ ∈Sd Ψ(m ¯ ) j∈N cipij(m ¯ ) Ψ(m ¯ ) = P . m¯ ∈Sd Ψ(m ¯ ) 2

Intuitively, there seems to be a discrepancy between (5.7) and (5.8) as (5.7) P and (5.8) should be such that j∈N Pij(m ¯ ) = Pi(m ¯ ). However, Pij(m ¯ ) expresses the probability that the unmoved customers are in statem ¯ when a customer moves from station i to station j and not the probability that the unmoved customers are in statem ¯ when a customer moves from station i to station j given that a customer leaves station i. Therefore, there is no discrepancy. For completeness, the following theorem considers the probabilities mentioned above. Theorem 5.3 (Palm probabilities) The probability that the unmoved customers are in state m¯ ∈ Sd and a transition from station i to station j, i, j ∈ N , occurs when a customer leaves station i, i ∈ N , is given by

Ψ(m ¯ )pij(m ¯ ) d Pi(m, ¯ i → j) = P , m¯ ∈ S , i, j ∈ N , m¯ ∈Sd Ψ(m ¯ ) and the probability that the unmoved customers are in state m¯ ∈ Sd and a transition from station i to station j, i, j ∈ N , occurs when a customer enters station j, j ∈ N , is given by

ci Ψ(m ¯ ) pij(m ¯ ) cj d Pj(m, ¯ i → j) = P , m¯ ∈ S , i, j ∈ N . m¯ ∈Sd Ψ(m ¯ ) 5.5 Relation between the primal and the dual process 103

Proof If H represents the transitions in which a customer leaves station i and C represents the transition from statem ¯ + ei to statem ¯ + ej then S H = j∈N Hij and C = Cij(m ¯ ). This immediately gives Pi(m, ¯ i → j). If H represents the transitions in which a customer enters station j and C represents the transition from statem ¯ + ei to statem ¯ + ej then H = S i∈N Hij and C = Cij(m ¯ ). This gives

π(m ¯ + ei)q(m ¯ + ei, m¯ + ej) Pj(m, ¯ i → j) = P P i∈N m¯ ∈Sd π(m ¯ + ei)q(m ¯ + ei, m¯ + ej) ψ(m ¯ ) QN cmk c p (m ¯ ) = k=1 k i ij P QN mk P m¯ ∈Sd ψ(m ¯ ) k=1 ck i∈N cipij(m ¯ ) (2.9) Ψ(m ¯ )cipij(m ¯ ) = P . m¯ ∈Sd Ψ(m ¯ )cj 2

The Palm probability PH (m ¯ ) is the probability that when an H-transition occurs a moving customer sees the other customers in state m¯ ∈ Sd. For example, Pij(m ¯ ) is the probability that a customer in transit from station i to station j sees the non-moving customers in statem ¯ ∈ Sd. The state observed by a customer in transit is a dual state representing the customer conﬁguration at the stations with one customer in transit. The global potential of the state observed by a customer in transit may be in- ﬂuenced by this customer. Therefore, the distribution seen by a customer in transit does not necessarily correspond to the equilibrium distribution of the queueing network with this customer removed. From Theorem 5.2 d d above, in some cases PH (m ¯ ) = π (m ¯ ),m ¯ ∈ S , the equilibrium distribution of the dual process. This distribution is independent of the customer in transit. In those cases, arrivals see time averages (ASTA) as expressed by (5.8) and moving customers (units) see time averages (MUSTA) as expressed by (5.7). This is formalized in the following variant of the arrival theorem.

Theorem 5.4 (Arrival Theorem) Consider a stationary Markov chain N N with transition rates (2.3) such that a positive solution {ci}i=1 to (2.9) exists. The distribution observed by a customer arriving at a station or d departing from a station is given by π as given in (5.3) with di = ci, i = 1,...,N. If, for i, j ∈ N , pij(m ¯ ) is independent of m¯ then the distribution observed by a customer in transit from station i to station j is given by πd as speciﬁed above. Moreover, if moving customers (arriving at a station, departing from a station or in transit between stations) observe πd, the 104 Dual processes evolution of the states observed by the moving customers is given by the dual process N d with transition rates

 B ψ(m ¯ + e − e )  j i c ,  d j  B φ(m ¯ + ej) d  q (m, ¯ m¯ +ej −ei) = if ∃r, s ∈ N   such that prj(m ¯ ) > 0, pis(m ¯ + ej − ei) > 0,   0, otherwise, (5.9)

Proof From Theorem 5.2, πd is the distribution observed by a moving customer in the cases specified in the theorem. The transition rates of the process describing the evolution of the states observed by the customers in transit must be such that the probability flow from statem ¯ ∈ Sd to statem ¯ 0 ∈ Sd is completely determined by the probability flow for N. As the equilibrium distribution of the process describing the evolution of the states observed by the customers in transit is given by πd, Theorem 5.1 gives the last statement of the theorem. 2

By definition the dual process is a Markov chain. As is shown in Theorem 5.4, the evolution of states as observed by a moving customer can be described by the dual process. However, as can immediately be seen by observing a sequence of transitionsn ¯ → n¯ − ei → n¯ − ei + ej → n¯ − ei + ej − ek → n¯ − ei + ej − ek + el, as depicted in Figure 5.1, the transition probability from staten ¯ − ei to staten ¯ − ei + ej − ek must depend onn ¯ as the routing probability pij(¯n − ei) for the first part of the transition sequence depends on staten ¯ and station i. Therefore, for the primal process, the evolution of states as observed by a moving customer depends on the history of the process and cannot be a Markov chain. Thus, the dual process is a Markov chain constructed such that it describes the evolution of the states as observed by a moving customer. The arrival theorem is discussed by various authors (cf. [55], [83] and the references therein). Recently, for queueing networks with transition rates similar to (2.3), the arrival theorem is discussed in [42] for batch routing networks, and in [76] MUSTA is defined and proven. However, all authors assume ψ = φ when considering the arrival theorem, also when this is not necessary in the proof (cf. [42]). The motivation for this specific choice, ψ = φ, is that the distribution seen by a moving customer must be identical to the equilibrium distribution of the queueing network with one customer removed. In my opinion, as I discussed in the 5.6 Examples 105 preamble of Theorem 5.4, this assumption is not necessary. This is a direct consequence of the potential-interpretation of the primal and dual processes. A moving customer may influence the configuration potential of the state it observes. This influence is taken into account in ψ. If ψ = φ, a moving customer does not influence the configuration-potential of the state it observes. In this case, the first part of Theorem 5.4 reduces to the well-known version of the arrival theorem. To the best of my knowledge, the second part of Theorem 5.4 is new, in that the process describing the evolution of the states observed by the moving customers is not described in the literature. The appearance of this process is a direct consequence of the introduction of the dual process in Section 3.

6 Examples

This section gives some theoretical examples on the relation between the primal and the dual process. In Example 6.1 the symmetrical relation between the primal and the dual process is shown to be related to the relation between the primal and dual problem in linear programming. This justiﬁes the name dual process given to the process introduced in Deﬁnition 3.1. In Examples 6.2 – 6.5, the transition rates and state spaces for the primal and dual process are discussed and Example 6.6 discusses customer-vacancy duality as reported in the literature.

6.1 Complementary slackness relations This section gives a motivation for the name dual process for the process introduced in Definition 3.1. To this end, reconsider Definition 2.1 and Theorem 2.2. In the definition of the dual state space specific information on the transition rates of the primal process is used (m ¯ ∈ Sd only if ∃i, j ∈ N such that q(m ¯ + ei, m¯ + ej) > 0). This information is not necessary to the theory, it merely makes the introduction of the dual process more elegant. Therefore, the dual state space may be replaced by S∗ defined as

∗ N S = {m¯ ∈ IN0 |∃i, j ∈ N :m ¯ + ei, m¯ + ej ∈ S}. Moreover, the assumption ψ(m ¯ ) > 0 is not necessary. ψ may be replaced ∗ + ∗ + by ψ : S → IR ∪ {0}, and pij may be deﬁned as pij : S → IR ∪ {0}. With these relaxed assumptions, Theorem 2.2 must be replaced by the following theorem. 106 Dual processes

Theorem 6.1 N allows an invariant measure Φ at S, given by

N Y nk Φ(¯n) = φ(¯n) ck , n¯ ∈ S, k=1

N if for all n¯ ∈ S the coeﬃcients {ci}i=1 are a non-negative (ci ≥ 0 for all i) solution of X {γipij(¯n − ei) − γjpji(¯n − ei)} = 0, i ∈ N , γ0 = 1, (6.1) j∈N

∗ for all n¯ − ei ∈ S such that ψ(¯n − ei) > 0. The proof of Theorem 6.1 is identical to the proof of Theorem 2.2. The motivation for the last equality in the proof of Theorem 2.2 is changed. This equality is obtained since for fixedn ¯ ∈ S, N can be divided into N+ = {i ∈ N |ψ(¯n − ei) > 0} and N0 = {i ∈ N |ψ(¯n − ei) = 0}. At N+ (6.1) holds and thus the last equality in the proof of Theorem 2.2 is obtained. ¯ For b, the zero-vector,c ¯ defined as [¯c]i = ci, P (¯n) defined as [P (¯n)]ij = QN nk−δki pij(¯n − ei), Ψ(¯n) defined as [Ψ(¯n)]i = ψ(¯n − ei) k=1 ck , i, j ∈ N , (2.10) can be written Ψ(¯n) · ¯b − cP¯ (¯n) = 0, (6.2) where · denotes inner-product. (6.2) is valid since either [Ψ(¯n)]i = 0 or ¯ [(b − cP¯ (¯n))]i = 0. Therefore, (6.2) resembles a complementary slackness relation from linear programming (cf. [74]), where Ψ(¯n) is the optimal solution of a dual problem andcP ¯ (¯n) ≤ ¯b is the feasibility condition for the primal problem. Therefore, the process with invariant measure QN mk ψ(m ¯ ) k=1 ck is called a dual process. The relaxation of the assumptions stated in Section 2 to the form stated above creates difficulties when introducing the dual process. Although it is possible to use the relaxed assumptions, I have chosen to include all the assumptions of Section 2 on the transition rates, i.e. on Sd, ψ, φ and pij. This makes the theory and the introduction of the dual process more transparent.

6.2 Blocking examples This example shows that the dual state space depends on the blocking protocol. To this end, consider an open two station queueing network with 5.6 Examples 107

ﬁnite capacity constraints on the number of customers in the stations and in the system. If M1, M2 customers are allowed at station 1, 2, respectively and M < M1 + M2 customers are allowed in the system, the state space S for this process is given by

2 S = {n¯ ∈ IN0|n1 ≤ M1, n2 ≤ M2, n1 + n2 ≤ M}.

If all transitionsn ¯ → n¯ − ei + ej within S are allowed, the dual state space Sd is

d 2 S = {m¯ ∈ IN0|m1 ≤ M1, m2 ≤ M2, m1 + m2 ≤ M − 1}.

Dualizing back using the ﬁrst backward dualizing method gives under the d assumption that all transitionsm ¯ → m¯ + ej − ei within S are allowed

S− = S, as can easily be seen from the deﬁnition (3.5) of S−. Assume that the primal routing function pij(m ¯ ) is state-independent, d i.e. pij(m ¯ ) = pij for allm ¯ ∈ S . Then, unless N is reversible, N does not posses a product-form equilibrium distribution. For a product-form equilibrium distribution to exist, a special blocking-protocol must be used for blocking certain transitions near the boundary of S. To this end, consider the stop-protocol, also referred to as communication-blocking (cf. [26]). In the stop-protocol, if one station reaches its limit all other stations are stopped and customers are not allowed to enter the queueing network. Graphically, this protocol can easily be explained. To this end, consider the primal traﬃc equations (2.9):

X {γipij(m ¯ ) − γjpji(m ¯ )} = 0, i ∈ N . (6.3) j∈N

For each i ∈ N ,m ¯ ∈ Sd, (6.3) states that the flow for transitions in P which a customer leaves station i ( j cipij) balances with the flow for P transitions in which a customer enters station i ( j cjpji). For fixedm ¯ this is visualized in Figure 5.4, where for i ∈ N these flows are depicted. As pij(m ¯ ) ≡ p(m ¯ + ei, m¯ + ej) is independent ofm ¯ , for all statesm ¯ flows are balanced in exactly the same way. But ifm, ¯ m¯ + e1 ∈ S,m ¯ + e2 6∈ S, i.e. if m2 = M2, transitionsm ¯ ↔ m¯ + e2, m¯ + e1 ↔ m¯ + e2 are prohibited. Therefore, for the flow balance not to be disturbed, also the transitions m¯ ↔ m¯ + e1 must be blocked. A similar argument holds if m1 = M1. The 108 Dual processes

m¯ + e2 m¯ + e2 m¯ + e2

@ @ r r @ r @ @ @ @ @ vm¯ rm¯ + e1 vm¯ rm¯ + e1 m v¯ rm¯ + e1 i = 0 i = 1 i = 2 flow out ofm ¯ flow out ofm ¯ + e1 flow out ofm ¯ + e2 = flow intom ¯ = flow intom ¯ + e1 = flow intom ¯ + e2

Figure 5.4. Balance of ﬂows; primal process.

n¯ − e1 n¯ n¯ − e1 n¯ n¯ − e1 n¯

@ r v r @ v r v @@ nr¯ − e2 nr¯ − e2 nr¯ − e2 j = 0 j = 1 j = 2 flow out ofn ¯ flow out ofn ¯ − e1 flow out ofn ¯ − e2 = flow inton ¯ = flow inton ¯ − e1 = flow inton ¯ − e2

Figure 5.5. Balance of ﬂows; dual process.

blocking-protocol prohibiting the transitionsm ¯ ↔ m¯ + e1 if m2 = M2 and m¯ ↔ m¯ + e2 if m1 = M1 is called the stop-protocol. If the stop protocol is used the dual state space Sd is given by

d 2 Ss = {m¯ ∈ IN0|m1 ≤ M1 − 1, m2 ≤ M2 − 1, m1 + m2 ≤ M − 1}.

Similar to the assumption on the primal routing function, assume that the d d − dual routing function is state-independent, i.e. pij(¯n) = pij for alln ¯ ∈ S . Then, for the dual process to posses a product-form equilibrium distribution, similar to the blocking-protocol for the primal process, transitions d near the boundary of Ss must be blocked. In contrast with the primal process, where transitions near the upper boundary of S are blocked, for the dual process to allow a product-form equilibrium distribution transitions near the lower boundary of Sd must be blocked. To realize this, reconsider the dual traﬃc equations (3.3): 5.6 Examples 109

( ) X 1 d 1 d pij(¯n) − pji(¯n) = 0, j ∈ N . i∈N γj γi

d In Figure 5.5, for fixedn ¯, the balanced flows are depicted. As pij(¯n) is independent ofn ¯, for alln ¯ flows are balanced in exactly the same way. d d Therefore, ifn, ¯ n¯ − e1 ∈ Ss , n¯ − e2 6∈ Ss , i.e. if n2 = 0, transitions n¯ ↔ n¯−e2, n¯−e1 ↔ n¯−e2 are prohibited. Therefore, for the flow balance not to be disturbed, the transitionsn ¯ ↔ n¯ −e1 must be blocked. A similar argument shows that if n1 = 0 transitionsn ¯ ↔ n¯−e2 must be blocked. The protocol stopping arrivals to and departures from the non-starved stations if a station is starved is called the dual-stop-protocol. Under the dual-stop- protocol, dualizing back according to the first backward dualizing method gives

− 2 Ss = {n¯ ∈ IN0|0 < n1 ≤ M1 − 1, 0 < n2 ≤ M2 − 1, n1 + n2 ≤ M}. d d − − AsSs ⊂ S and Ss ⊂ S , this example shows that the dual state space depends on the speciﬁc blocking protocol.

6.3 Alternative transition rates; dual states In many practical examples, the form of the transitions (e.g.m ¯ → m¯ − ei + ej) and the equilibrium distribution are observed only. In these cases the transition rates must be determined from these observations. This example gives two alternatives for these rates. Moreover, this example gives an alternative justiﬁcation of Example 4.1. Consider a dual process at state space Sd. Then a transitionm ¯ → m¯ + ej − ei for this process passes throughm ¯ + ej and the transition rates qd have the form φ(m ¯ + e ) qd(m, ¯ m¯ + e − e ) = j pd (m ¯ + e ). (6.4) j i ψ(m ¯ ) ij j

In practical situations, however, the intermediate state,m ¯ + ej, is not d observed. Therefore, a transitionm ¯ → m¯ + ej − ei for a process at S could pass throughm ¯ − ei resulting in transition rates φ(m ¯ − e ) qd(m, ¯ m¯ − e + e ) = i p (m ¯ − e ), (6.5) i j ψ(m ¯ ) ij i

? + ? + where φ : S → R , pij : S → R ∪ {0} and

? N d d S = {n¯ ∈ IN0 |∃i, j ∈ N , i 6= j :n ¯+ei, n¯+ej ∈ S , q (¯n+ei, n¯+ej) > 0}. 110 Dual processes

Assume that the process at Sd has an equilibrium distribution πd given by N d d Y mk d π (m ¯ ) = B ψ(m ¯ ) ck , m¯ ∈ S , k=1 d d then the routing function pij or pij must be determined such that π satisfies the global balance equations (2.4) at Sd. As the routing functions reflect the site-potential difference and statem ¯ +ej contains one customer extra at both station j and station i when compared tom ¯ − ei, it is not unnatural to assume d pij(m ¯ + ej) = cicjpij(m ¯ − ei).

Then, if the routing is state-independent (e.g. pij(m ¯ − ei) = pij1{m¯ − ei ∈ ? N S }), from Example 4.1, {ci}i=1 is a solution to (2.9) if and only if it is a solution to (3.3). This example shows that, unless speciﬁc information on the transition rates is available (e.g. speeds, blocking), (6.4) and (6.5) are indistinguishable descriptions of the process at Sd.

6.4 Alternative transition rates; traffic equations Based on the intermediate states, the previous example gives a theoretical argument for the identification of a primal and a dual process. However, a more practical observation may be the basis of this identification. To N PN this end, consider a process N at state space S = {n¯ ∈ IN0 | i=1 ni = M} with transition rates

q(¯n, n¯ − ei + ej) = µipij, i, j ∈ N . (6.6) N Furthermore, assume a positive solution {ci}i=1 is obtained to X {γjµipij − γiµjpji} = 0, j ∈ N . i∈N Consider the dual formulation for N with dual routing function d pij = µipij, i, j ∈ N and φ ≡ 1, ψ ≡ 1. Since (6.6) is independent of the intermediate state n¯ − ei orn ¯ + ej, the transition rates in the dual formulation are given by (6.6) too. From Theorem 3.2, N allows an invariant measure

N 1 nk ψ(¯n) = Y , n¯ ∈ S. k=1 ck 5.6 Examples 111

6.5 Self-dual process A primal process is called self-dual if a dual process corresponding to this primal process exists such that the primal process and the dual process are statistically indistinguishable. Consider a primal process at state space S with transition rates (2.3) and equilibrium distribution π. Then for the dual process at state space Sd deﬁned in (2.2) with transition rates (3.1) and equilibrium distribution πd, the process is self-dual if and only if (i) Sd = S, d (ii) q (¯n, n¯ + ej − ei) = q(¯n, n¯ − ei + ej), n,¯ n¯ − ei + ej ∈ S, (iii) πd(¯n) = π(¯n), n¯ ∈ S. Note that the intermediate states for a dual transition and a primal transition are not the same. However, as these dual states are not observed during a transition this is not a problem. As an illustration consider the following simple example. Consider an open queueing network consisting of N single-server queues. The service- rate at station i is µi. Upon release from station i a customer is routed to station j according to the routing probabilityp ˆij. Assume that a positive N solution {ci}i=1 exists to

γipˆij = γjpˆji, i, j ∈ N , γ0 = µ0. This queueing network can be modelled as a primal process at state space N S = IN0 with transition rates q(¯n, n¯ −ei +ej) = µipˆij, where µ0 is the rate of the Poisson arrival process to the queueing network. For φ ≡ 1, ψ ≡ 1, and primal routing function

pij = µipˆij, i, j ∈ N ,

N the transition rates have the form (2.3). As {ci/µi}i=1 is a positive solution to (2.9), for µi > ci, i = 1,...,N, the primal process possesses an equilibrium distribution π given by N c !nk π(¯n) = B Y i . (6.7) k=1 µi The dual state space for this primal process is Sd = S. Consider the dual process with transition rates (3.1) with dual routing function

d pij = pij, i, j ∈ N . 112 Dual processes

N Then {µi/ci}i=1 is a positive solution to (3.3). Thus the dual process allows an invariant measure πd given in (6.7) and the process is self-dual. A few remarks on self-duality are to be made. First, note that, in general, for (i) and (iii) to be valid, it need not be the case that (ii) is satisfied. Moreover, (ii) will, in general, not be satisfied. Second, from the above example, it seems to be the case that the transition rates for the dual process depend on the specific choice for φ, ψ and pij. This is not the case. To this end, consider the obvious choice for these functions:

N !nk Y 1 φ(¯n) = ψ(¯n) = , pij =p ˆij. k=1 µk In this case, for the process to be self-dual, the dual routing function has to be deﬁned as d pij(¯n) = µiµjpij, n¯ ∈ S. (6.8)

6.6 Customer-vacancy duality Consider a closed queueing network consisting of single-server queues only. The server at station i works at rate µi, i = 1,...,N, and the number of customers at station i is not to exceed a given capacity constraint Mi, i = 1,...,N. The state space of the primal process describing this queueing network when M customers are present in the network is given by

N N X SM,M¯ = {n¯ ∈ IN0 | ni = M, ni ≤ Mi, i = 1,...,N}, i=1 ¯ where M = (M1,...,MN ). The transition rates for the primal process are

q(¯n, n¯ − ei + ej) = µipˆij, n,¯ n¯ − ei + ej ∈ SM,M¯ , i, j ∈ N ,

N wherep ˆij is the routing probability. Assume a positive solution {ci}i=1 exists to the traﬃc equations

N X {cipˆij − cjpˆji} = 0, i ∈ N . j=1

Similar to Example 6.5, deﬁne the primal routing function pij as

pij(m ¯ ) = µipˆij1{m¯ + ei, m¯ + ej ∈ SM,M¯ }, 5.6 Examples 113

then for M ≤ mini∈N Mi, from Theorem 2.2, the primal process has an equilibrium distribution π at SM,M¯ given by

N !nk Y ck π(¯n) = B , n¯ ∈ SM,M¯ . k=1 µk

If M > mini∈N Mi the equilibrium distribution cannot immediately be obtained in closed form, since the normalizing constant cannot be directly PN obtained. However, for M ≥ i=1 Mi − mini∈N Mi the equilibrium distribution, including its normalizing constant, can be obtained by considering the vacancies at the stations. To this end, note that when ni customers are present at station i, mi = Mi − ni vacancies are present at station i. Furthermore, when a customer routes from station i to station j a vacancy is routed from station j to station i. Moreover, in a transition n¯ → n¯ −ei +ej ﬁrst a customer leaves station i and is subsequently routed to station j. This corresponds to the creation of a vacancy at station i and subsequently the removal of a vacancy at station j. Thus the queueing network can be described using vacancies instead of customers and the process describing the evolution of the states in the vacancy description is a dual process. The state space of this dual process is

N N d N X X SM,M¯ = {m¯ ∈ IN0 | mi = Mi − M, mi ≤ Mi, i = 1,...,N}, i=1 i=1 and the transition rates are

d d q (m, ¯ m¯ + ej − ei) = µjpˆji, m,¯ m¯ + ej − ei ∈ SM,M¯ , i, j ∈ N . For the dual process to posses an equilibrium distribution πd based on the dual traﬃc equations the dual stop-protocol must be used (cf. Exam- ple 6.2). However, as is argued in Example 6.4, the intermediate state for a transitionm ¯ → m¯ + ej − ei is not observed. Thus this transition can be considered as passing throughm ¯ − ei. In this primal description blocking N will not occur. Then, if {di}i=1 solves X {diµjpji − djµipij} = 0, i ∈ N , (6.9) j∈N the vacancy process possesses an equilibrium distribution πd given by

N d d Y mk d π (m ¯ ) = B dk , m¯ ∈ SM,M¯ . (6.10) k=1 114 Dual processes

From (6.10) the equilibrium distribution for customers can immediately be obtained by inserting ni = Mi − mi, i = 1,...,N. This gives

N n Y 1 k π(¯n) = B , n¯ ∈ SM,M¯ . k=1 dk In the special case that the queueing network is a cyclic network a method considering vacancies at the stations is used in [38]. If the queueing network is cyclic, then 1 1 di = , i = 2, . . . , N, d1 = , µi−1 µN is a solution to (6.9) and the equilibrium distribution π is given by

N Y nk−1 π(¯n) = B µk , n¯ ∈ SM,M¯ , k=1 as obtained in [38]. This example generalizes the approach used in [38] to non-cyclic queueing networks, however, this example is not a straightforward application of the dual process presented in this paper. Note that the dual process presented here gives a justiﬁcation for the dual routing function presented in Example 4.4. Chapter 6

Norton’s theorem

1 Introduction

Queueing networks are widely used in computer performance evaluation, telecommunications and manufacturing. For a large class of queueing a closed-form (e.g. product-form) equilibrium distribution can be obtained. This equilibrium distribution specifies the behaviour of the queueing network, and various characteristics of interest can be determined from the equilibrium distribution. Frequently, one is interested merely in global characteristics such as the total throughput of a group of stations, sojourn- times at parts of the queueing network, transition flows from one part to another or just a global occupancy distribution for clusters of stations rather than a detailed distribution for all stations. Such characteristics can be obtained from the equilibrium distribution of the queueing network. However, the size of the queueing network often prohibits efficient calculation of this equilibrium distribution even if a product-form expression for the equilibrium distribution is available. If global characteristics are of interest only, one would preferably be able to compute these global characteristics by evaluating or measuring global behaviour only. Intuitively this would be achieved by aggregating parts of the queueing network as single stations so that global characteristics can be calculated without calculating the local characteristics. Furthermore, if one is interested in local characteristics of a part of the queueing network only, computation of these characteristics would be simplified if the rest of the queueing network could be aggregated as a single station or as a small number of single stations. Conversely, as conditions for product-form expressions can be complicated to verify when state-

115 116 Norton’s theorem dependent routing and service are involved, it would also be appealing when these conditions can be decomposed into global and local conditions. In particular, detailed product-form expressions can then be concluded by merely verifying global and local traffic conditions. Unfortunately, such aggregation and decomposition results are not generally justified as local and global network behaviour is usually integrated by state-dependent routing and service-mechanisms. In [13] an efficient aggregation method similar to Norton’s theorem from electrical circuit theory was introduced. Norton’s theorem for queueing networks states that

under certain conditions on the structure of the queueing network it is possible to replace a subset of the queueing network by a single station such that the equilibrium distribution of the rest of the network remains unchanged.

[13] prove the aggregation method to be correct for queueing networks of the BCMP-type [4] consisting of two subnetworks of which the subnetwork of interest is a single station. This can easily be extended to subnetworks of interest consisting of several stations such that customers enter the subnetwork through a single input node and leave the subnetwork through a single output node. [3], [57] and [79] further extend Norton’s theorem to BCMP-networks consisting of two arbitrary subnetworks. Related work on decomposition and aggregation is reported in [22]. In [82] the aggregation results for BCMP-networks consisting of two subnetworks are generalized to queueing networks consisting of two quasireversible subnetworks. An approach related to the one based on Norton’s theorem is reported in [19], [20], called “Parametric analysis by chain.” This method constructs a reduced network around a particular routing chain of interest. Detailed contrasting with the related literature will be given in Section 8. From these results on Norton’s theorem, the general impression seems to have grown that aggregation and decomposition results are generally valid under product-form conditions. However, no formal justification for such results is available when state-dependent routing and service such as due to blocking or common resource sharing are involved. This chapter aims to give this justification and aims to generalize the results on Norton’s theorem to queueing networks with blocking and general state-dependent service-rates. Analysis of the equilibrium distribution of queueing networks and the aggregation and decomposition method for queueing networks proposed 6.2 Electrical circuit theory 117 in this chapter is motivated by Thevenin’s theorem rather than Norton’s theorem from electrical circuit theory (cf. [6]). As will be discussed in Sec- tion 2, for electrical circuits Thevenin’s theorem is equivalent to Norton’s theorem. The analogue of Thevenin’s theorem for queueing networks, however, is different and more general than the analogue of Norton’s theorem. As the analysis of queueing networks presented in this chapter is based on aggregation results from electrical circuit theory, Section 2 presents these results, that is Thevenin’s theorem and Norton’s theorem from electrical circuit theory are reviewed. Section 3 presents the general queueing network studied in this chapter. This queueing network model is more general than the model used in Chapter 5. In general, the routing part of the transition rates is the most difficult part to analyse. Therefore, in Section 4 first the routing characteristics of the queueing network are analysed. Motivated by electrical circuit theory, the routing probabilities are decomposed into a global component and a local component. It is shown that this implies the same decomposition for the solution of the traffic equations. Second, the service-characteristics are analysed. Similar to the routing characteristics, the service functions are decomposed into a global component and a local component. From this decomposition, Section 5 presents a full decomposition of the equilibrium distribution resulting in Norton’s theorem for queueing networks with state-dependent routing and service-characteristics. Section 6 summarizes the practical applications of Norton’s theorems obtained in this chapter. Section 7 presents some examples and Section 8 discusses the literature.

2 Electrical circuit theory

The aggregation and decomposition method for queueing networks proposed in this chapter is motivated by Thevenin’s theorem from electrical circuit theory. However, as Norton’s theorem has become the standard phrasing in the literature on aggregation in queueing networks, the aggregation and decomposition results obtained in this chapter will be referred to as Norton’s theorem. To shed some light on the diﬀerence between Thevenin’s theorem and Norton’s theorem, and to give some background information, consider Thevenin’s theorem and Norton’s theorem for electrical circuits in more detail. Consider an electrical circuit consisting of generators and impedances as shown in Figure 6.1. Based on Kirchhoﬀ’s laws the circuit 118 Norton’s theorem

I a - circuit of generators t and V Z impedances ? tb Figure 6.1. Electrical circuit

I I a - a -

t t εeff Zeff Z εeff Zeff

tb tb Figure 6.2. a: Equivalent circuit b: Z shorted of generators and impedances can be replaced by an effective generator and impedance. Kirchhoff’s first law: “The sum of the currents into any node must be zero.” gives Norton’s theorem (cf. [6]) stating that the circuit of generators and impedances can be replaced by an effective generator and a parallel internal impedance without affecting the behaviour of Z (Figure 6.2a). The value of the current source is set equal to the current flowing from a to b when Z is replaced by a short (Figure 6.2b). Kirch- hoff’s second law: “The sum of the potential differences around a complete loop of a circuit is equal to zero.” gives Thevenin’s theorem (cf. [6]) stating that, without affecting the behaviour of Z, the circuit of generators and impedances can be replaced by an effective generator in series with an internal impedance (Figure 6.3a). The value of the potential difference generated by the generator is set equal to the potential difference between a and b when Z is removed (Figure 6.3b). The behaviour of Z in the equivalent network obtained via Norton’s theorem and Thevenin’s theorem is identical. The internal structure and behaviour of the replacement circuit consisting of εeff and Zeff may differ. Obviously, analysing the behaviour of Z in the equivalent network requires less computational effort. Analogues to the results discussed above for electrical circuits can be 6.2 Electrical circuit theory 119

a a

t t Zeff Zeff

V Z V

εeff εeff

? ? tb tb Figure 6.3. a: Equivalent circuit b: Z removed obtained for queueing networks. The aggregation and decomposition results for queueing networks obtained in this chapter are motivated by Thevenin’s theorem rather than Norton’s theorem from electrical circuit theory. As is observed above, for electrical circuits Thevenin’s theorem is equivalent to Norton’s theorem. The analogue of Thevenin’s theorem for queueing networks, however, is diﬀerent and more general than the analogue of Norton’s theorem. The diﬀerences in the structure of the analysis are the following: Thevenin’s theorem Norton’s theorem

1. Subnetworks are analysed Subnetworks are analysed as open queueing networks. as closed queueing networks.

2. The service speed of The throughput of aggregated a closed loop version subnetworks determines of subnetworks determines the equilibrium distribution. the equilibrium distribution.

ad 1. Note that in the original non-aggregated queueing network the subnetworks appear as open queueing networks. Therefore, Thevenin’s theorem is more natural in a queueing network environment than Norton’s theorem.

ad 2. The service speed of an aggregated subnetwork may depend on the total global (aggregated) state of the queueing network when 120 Norton’s theorem

Thevenin’s theorem is applied. This is natural as Thevenin’s theorem analyses the global behaviour of the queueing network. For Norton’s theorem, however, for obtaining the throughput of a closed loop version, the rest of the queueing network is ignored. There- fore, the throughput cannot depend on the state of the rest of the queueing network.

3 Queueing network model

Consider a queueing network consisting of N stations. A customer at station i requires a negative-exponential amount of service with parameter µi, i = 1,...,N. Letn ¯ = (n1, . . . , nN ) denote the number of customers at the stations, i.e. ni is the number of customers at station i, i = 1,...,N. When the queueing network is in staten ¯ and a customer at station i completes service it will route to station j with state-dependent routing probability pij(¯n), j = 1,...,N, and it will leave the network PN with probability pi0(¯n) = 1 − j=1 pij(¯n). The service-rate in staten ¯ at station i is given by ψ(¯n − e ) µ (¯n) = µ i , i = 0,...,N, (3.1) i i φ(¯n) where ψ is a non-negative function and φ a positive function such that ψ(¯n−ei) φ(¯n) represents the total service-eﬀort allocated to station i in staten ¯, and µ0 is the parameter of the arrival process to the queueing network. With service and routing functions speciﬁed above, the queueing network N can be represented as a continuous-time Markov chain at S ⊂ IN0 with transition rates q(¯n, n¯0), n,¯ n¯0 ∈ S, given by

( µ (¯n)p (¯n), n¯0 =n ¯ − e + e , i, j = 0,...,N, q(¯n, n¯0) = i ij i j (3.2) 0, otherwise.

Note that the routing probabilities used in (3.2) diﬀer slightly from the form used in Chapter 5. In Chapter 5 emphasis is on the dual process, and the routing probability from staten ¯ to staten ¯ − ei + ej is written as pij(¯n − ei) to explicitly show that this transition virtually passes through staten ¯ − ei. In the present chapter emphasis will be on a decomposition of the routing probabilities into a local part and a global part. For this decomposition the form pij(¯n) is more convenient. Note that the routing probabilities used in (3.2) are equivalent to those used in Chapter 5. 6.3 Queueing network model 121

Assume that the Markov chain is stable, regular, irreducible, and that there exists a unique equilibrium distribution at S, i.e. a set of non- negative numbers π = (π(¯n), n¯ ∈ S), summing to unity, that satisﬁes the global balance equations at S π(¯n) X q(¯n, n¯0) = X π(¯n0)q(¯n0, n¯), n¯ ∈ S. n¯0∈S n¯0∈S As the transition rates have the special form (3.2), the global balance equations can be written for alln ¯ ∈ S

N N N X X X π(¯n) µi(¯n) = π(¯n − ei + ej)µj(¯n − ei + ej)pji(¯n − ei + ej). (3.3) i=0 i=0 j=0 The following theorem shows that a product-form equilibrium distribution can be concluded if the state-dependent traffic equations can be solved. Note that Theorem 3.1 generalizes the result of Theorem 5.2.2, since the solution to the traffic equations (3.4) may now be an arbitrary function Q ni H(¯n) of the state of the queueing network instead of a product i ci as assumed in Chapter 5. Note that Theorem 3.1 implies that the equilibrium distribution can be obtained as a solution to the local balance equations (2.3.7), and that local balance is expressed by the state-dependent traffic equations (3.4). The result of Theorem 3.1 is known in the literature. Similar results are obtained in [27], [41], [76], [87]. Theorem 3.1 If a positive solution H = (H(¯n), n¯ ∈ S) exists to the state-dependent traffic equations for all n¯ ∈ S, i = 1,...,N,

N X H(¯n) = H(¯n − ei)µ0p0i(¯n − ei) + H(¯n − ei + ej)pji(¯n − ei + ej), (3.4) j=1 then, with B a normalizing constant, the equilibrium distribution of the process with transition rates (3.2) is given by N 1 !nk π(¯n) = Bφ(¯n)H(¯n) Y , n¯ ∈ S. (3.5) k=1 µk Proof Substitution of (3.1) and (3.5) into (3.3) gives

N N !nk−δki N N !nk−δki X Y 1 X Y 1 Bψ(¯n − ei) H(¯n) = Bψ(¯n − ei) i=0 k=1 µk i=0 k=1 µk  N   X  × H(¯n − ei)µ0p0i(¯n − ei) + H(¯n − ei + ej)pji(¯n − ei + ej) ,  j=1  122 Norton’s theorem which immediately shows π satisﬁes (3.3) for each i separately, that is π satisﬁes local balance. 2

The equilibrium distribution (3.5) is of product-form. This form further generalizes the product-form distributions discussed in Section 2.4 to product-form distributions that factorize into a general state-dependent nk service part, φ(¯n) QN 1 , and a general state-dependent routing part, k=1 µk H(¯n). Furthermore, the service part consists of a term representing the total state-dependent service-effort in the queueing network, φ(¯n), and a term representing the expected amount of service required at the stations, nk QN 1 . The term representing the expected amount of service factor- k=1 µk izes into the individual stations, which is characteristic for product-form queueing networks. The decomposition into a service part and routing part is complete in the sense that the service part and the routing part can be analysed separately. This observation will be used in Section 4, where first the routing part of the equilibrium distribution is analysed and subsequently the service part of the equilibrium distribution is analysed. The parameterization models both open and closed queueing networks. If the queueing network is closed, for alln ¯ and i = 1,...,N, assume that pi0(¯n) = p0i(¯n) = 0. The traffic equations now read forn ¯ ∈ S, i = 1,...,N,

N X H(¯n) = H(¯n − ei + ej)pji(¯n − ei + ej). j=1 If the queueing network is open, summation of (3.4) over all i, i = 1,...,N, gives the traﬃc equations for the outside:

N N X X H(¯n)µ0 p0j(¯n) = H(¯n + ej)pj0(¯n + ej). j=1 j=1 Note that some freedom remains in determining the terms appearing in the service function µi(¯n). In Chapter 3 canonical forms are presented to remove this freedom from the description of the model. Here the additional factors µi appearing in the service function give an additional degree of freedom. Similar to the canonical form presented in Chapter 3, ψ and φ are determined up to a common constant. Without loss of generality, this constant can be fixed by assuming that φ(0) = 1. Second, µi and ψ are determined up to a common constant. This constant can be fixed by assigning a fixed value to the arrival parameter µ0. When 6.3 Queueing network model 123

the queueing network has been analysed for a ﬁxed parameter µ0, the 0 service-characteristics for the queueing network with arrival parameter µ0 are obtained by the following substitution

0 0 µ0 0 µ0 µi = µi , i = 0, . . . , N, ψ (¯n) = 0 ψ(¯n), (3.6a) µ0 µ0 which leaves the service function µi(¯n) unchanged. Also note that the equilibrium distribution is not affected by this substitution as H is proportional to µ0 too. If H is a solution to the traffic equations (3.4) for µ0, then H0 given by N µ0 !nk H0(¯n) = H(¯n) Y 0 (3.6b) k=1 µ0 0 is a solution to the traffic equations for µ0. This observation will be used in Example 7.1.

Notation for clusters In the sequel, the stations of the queueing network will be grouped into clusters or subnetworks. A cluster consists of a number of stations such that clusters are disjoint, non-empty and all stations are part of a cluster, i.e. the stations are grouped into clusters Cr, r ∈ R, such that

Cr ⊂ {1,...,N}, r ∈ R, Cr ∩ Cs = ∅, r 6= s, S r∈R Cr = {1,...,N}, C0 = outside.

The following notation will be used for clusters of stations.

(r) n¯ =n ¯|Cr number of customers at the stations in Cr (r) ei = ei|Cr unit-vector for station i at Cr P nr = i∈Cr ni total number of customers at Cr n¯ = (n1,..., nR) e¯r unit-vector for cluster r, r ∈ R

(r) Thus, forn ¯ ∈ S the vectorn ¯ consists of the components ofn ¯ inside Cr only, and n¯ gives the total number of customers at the clusters. The following example illustrates the notation for clusters. 124 Norton’s theorem

Example 3.2 (Service function; notation for clusters) The service function µi(¯n) as given in (3.1) is the most general form appearing in the literature and is of theoretical interest as it combines generality with theoretical elegance. An extensive illustration of the service-rates that can be modelled can, for example, be found in [76] and in batch movement setups in [9], [41]. In practical cases, for a given function µi(¯n), the diﬃcult task is to ﬁnd functions ψ and φ such that the service function can be written in the form (3.1). A natural and illustrative example of a service function that can be written in the form (3.1) is the following. Assume that the service-speed at station i is a function of the number of customers at station i, fi(ni) if ni customers are present at station i, and the number of customers at the cluster containing station i, say cluster r, Fr(nr) if nr customers are present at cluster r. This gives

µi(¯n) = µifi(ni)Fr(nr), i ∈ Cr, which can be written in the form (3.1) with ψ and φ given by

−1  −1 " N ni # nr Y Y Y Y ψ(¯n) = φ(¯n) = fi(k)  Fr(k) . i=1 k=1 r∈R k=1

4 Decomposition into clusters

This section gives a decomposition of the transition rates of the queueing network into local and global parts. As is discussed in Section 3, the routing part and the service part of the transition rates can be analysed separately. In general the routing part is more complex than the service part. Furthermore, the decomposition of the queueing network will be based on the routing structure of the network. Therefore, in Section 4.1 ﬁrst the routing part of the transition rates is discussed. Section 4.3 considers the service part of the transition rates. Examples illustrating the decomposition of the routing and service parts are given in Sections 4.2, and 4.4.

4.1 Routing: Conditions

This section decomposes the routing characteristics pij(·) and H(·) into a global part and a local part. To this end, by analogy with the structure of electrical circuits at which Thevenin’s theorem can be applied (cf. [6]), the 6.4 Decomposition into clusters: routing 125 routing probabilities are decomposed into a global component and a local component (Assumption 4.1). This natural decomposition immediately implies a full decomposition of H into a global part and a local part. Moreover, the global part of H is completely determined by the global routing probabilities and the local part of H is completely determined by the local routing probabilities. The key-observation for electrical circuits at which Thevenin’s theorem can be applied is that an electrical circuit consisting of generators and impedances can be replaced by an eﬀective generator and impedance if and only if this circuit has a single input node (e.g. b in Figure 6.1) and a single output node (e.g. a in Figure 6.1). As the behaviour of stations is in general more complex than the behaviour of impedances, which more or less resemble M|M|∞-queues, for a similar theorem to hold in a queueing network the routing from one cluster of stations to another must be independent of the actual stations within these clusters. The following assumption guarantees this behaviour for the queueing network discussed in Section 3.

Assumption 4.1 (Routing probabilities) Assume that the stations of the queueing network can be grouped into clusters of stations, say Cr, r ∈ R, such that for i, j = 0,...,N, r, s ∈ R, and n¯ ∈ S, the routing probabilities have the form

 (r) (r) (r) (r) rr ¯ (r) (r) (r)  pij (¯n ) + pi0 (¯n )p (n)p0j (¯n − ei ), i, j ∈ Cr,    (r) (r) rs (s) (s)  pi0 (¯n )p (n¯)p0j (¯n ), i ∈ Cr, j ∈ Cs, pij(¯n) =  (r) (r) r0  pi0 (¯n )p (n¯), i ∈ Cr, j ∈ C0,    0s (s) (s)  p (n¯)p0j (¯n ), i ∈ C0, j ∈ Cs, (4.1) (r) rs where pij , i, j ∈ Cr ∪ {0}, r ∈ R, and p , r, s ∈ R ∪ {0}, satisfy for all n¯

P (s) (s) P (s) (s) (s) (s) j∈Cs p0j (¯n ) = 1, j∈Cs pij (¯n ) + pi0 (¯n ) = 1, P 0s P rs r0 s∈R p (n¯) = 1, s∈R p (n¯) + p (n¯) = 1. Assumption 4.1 implies that with respect to the input and output of customers the behaviour of a cluster in the queueing network with routing probabilities pij is similar to the structure of an electrical circuit at which Thevenin’s theorem can be applied. To show this, consider a customer 126 Norton’s theorem routing from station i at cluster r to station j at cluster s. First, according (r) to pi0 the customer leaves station i and routes to an input/output-node for cluster r (e.g. b in Figure 6.1). Subsequently, the customer is routed from the input/output node of cluster r to the input/output-node of cluster s (e.g. a in Figure 6.1) according to prs, independent of the local state of the clusters. Finally, the customer is routed from the input/output-node (s) of cluster s to station j at cluster s according to p0j . (r) (r) rs pij depends onn ¯ , the local state of cluster r, only, and p depends on n¯, the global state of the network, only. Therefore, the routing proba- (r) bilities pij can be interpreted as the state-dependent routing probabilities at cluster r when cluster r would be considered in isolation, that is when cluster r is considered as a queueing network without interaction with the rest of the network. In contrast, prs can be interpreted as the state- dependent routing probabilities for the network in which each cluster is replaced by a single station. This interpretation will be formalized in Sec- tion 5, when the transition rates of the queueing network at local level and at global level are defined. (r) (r) Blocking of transitions can arise at local level due to pij (¯n ) or at global level due to prs(n¯). These types of blocking are reflected in the (r) functions H and HR below and are independent. Note that customers (r) (r) arriving at a cluster may be blocked since p0j (¯n ) = 0 is allowed. How- (r) ever, a customer arriving at a cluster cannot be rejected due to p0j (·). (r) (r) Therefore, if p0j (¯n ) = 0 for some j an arriving customer must route to another station of the cluster. As the routing probabilities are separated into a global component and a local component which can be interpreted as the global and local routing probabilities, it is natural to investigate when the traffic equations (3.4) can be decomposed into global and local traffic equations. The following natural conditions on the global and local routing probabilities will imply this decomposition.

Condition 4.2 (Global traﬃc equations) There exists a positive solution, HR, to the global traﬃc equations, that is, for each subnetwork r ∈ R and for all n¯ such that n¯ ∈ S:

X sr HR(n¯) = HR(n¯ − e¯r + e¯s)p (n¯ − e¯r + e¯s) s∈R 0r +µ0HR(n¯ − e¯r)p (n¯ − e¯r). (4.2) 6.4 Decomposition into clusters: routing 127

Condition 4.3 (Local traﬃc equations) For r ∈ R there exists a pos- (r) itive solution, H , to the local traﬃc equations, that is, for each i ∈ Cr and all n¯(r) such that n¯ ∈ S:

(r) (r) X (r) (r) (r) (r) (r) (r) (r) (r) H (¯n ) = H (¯n − ei + ej )pji (¯n − ei + ej ) j∈Cr (r) (r) (r) (r) (r) (r) +H (¯n − ei )p0i (¯n − ei ). (4.3) For H to be decomposable into a global and local part compatible with the assumption and conditions stated above, H(¯n) may be expected to (r) factorize into a part involving H and a part involving HR. To this end, observe that insertion of (4.1) into (3.4) gives, for i ∈ Cr, r ∈ R,n ¯ ∈ S

X (r) (r) (r) (r) H(¯n) = H(¯n − ei + ej)pji (¯n − ei + ej ) j∈Cr   X (r) (r) (r) (r) rr + H(¯n − ei + ej)pj0 (¯n − ei + ej )p (n¯) j∈Cr X X (s) (s) (s) sr + H(¯n − ei + ej)pj0 (¯n + ej )p (n¯ − e¯r + e¯s) s∈R, s6=r j∈Cs  0r  (r) (r) (r) + µ0H(¯n − ei)p (n¯ − e¯r) p0i (¯n − ei ). (4.4)  In this form the traffic equations show great similarities to the traffic equations at local level (4.3), when the part inside brackets {} is replaced by H(¯n − ei) and although not as obvious, the traffic equations at global level (4.2). This suggests that H will indeed factorize into a global and a local component. This is established in the following theorem.

Theorem 4.4 (Routing decomposition) Assume that HR is a solution to the global traffic equations, (4.2), and H(r) is a solution to the local traffic equations, (4.3). Then H(¯n) defined as

Y (r) (r) H(¯n) = HR(n¯) H (¯n ), n¯ ∈ S, (4.5) r∈R is a solution of the traffic equations (3.4). Proof Summation of (4.2) gives that if H(r) satisfies (4.3) then for all n¯(r) such thatn ¯ ∈ S, H(r) also satisfies

(r) (r) X (r) (r) (r) (r) (r) (r) H (¯n ) = H (¯n + ej )pj0 (¯n + ej ). (4.6) j∈Cr 128 Norton’s theorem

Consider the right-hand side of the traﬃc equations (4.4), where, for notational convenience, (4.4) is divided by H(¯n − ei). This is not essential to the proof. By insertion of (4.5) and from (4.6), (4.2), (4.3), for i ∈ Cr, n¯ ∈ S: H(¯n − e + e ) X i j p(r)(¯n(r) − e(r) + e(r)) H(¯n − e ) ji i j j∈Cr i   X H(¯n − ei + ej) (r) (r) (r) + p (¯n(r) − e + e )prr(n¯) H(¯n − e ) j0 i j j∈Cr i H(¯n − e + e ) + X X i j p(s)(¯n(s) + e(s))psr(n¯ − e¯ + e¯ ) H(¯n − e ) j0 j r s s∈R, s6=r j∈Cs i  0r  (r) (r) (r) + µ0H(¯n − ei)p (n¯ − e¯r) p0i (¯n − ei ) 

(r) (r) (r) (r) (4.5) X HR(n¯) H (¯n − ei + ej ) (r) (r) (r) (r) = pji (¯n − ei + ej ) H (n¯ − e¯ ) (r) (r) (r) j∈Cr R r H (¯n − ei )  (r) (r) (r) (r)  X HR(n¯) H (¯n − ei + ej ) (r) (r) (r) (r) + pj0 (¯n − ei + ej ) H (n¯ − e¯ ) (r) (r) (r) j∈Cr R r H (¯n − ei ) ×prr(n¯) (s) H (n¯ − e¯ + e¯ ) H(s)(¯n(s) + e ) + X X R r s j p(s)(¯n(s) + e(s)) H (n¯ − e¯ ) H(s)(¯n(s)) j0 j s∈R, s6=r j∈Cs R r sr ×p (n¯ − e¯r + e¯s)  0r  (r) (r) (r) + µ0p (n¯ − e¯r) p0i (¯n − ei ) 

(r) (r) (r) (r) (4.6) X HR(n¯) H (¯n − ei + ej ) (r) (r) (r) (r) = pji (¯n − ei + ej ) H (n¯ − e¯ ) (r) (r) (r) j∈Cr R r H (¯n − ei ) ( ) X HR(n¯ − e¯r + e¯s) + psr(n¯ − e¯ + e¯ ) + µ p0r(n¯ − e¯ ) ¯ ¯ r s 0 r s∈R HR(n − er) (r) (r) (r) ×p0i (¯n − ei ) (r) (r) (r) (r) (4.2) X HR(n¯) H (¯n − ei + ej ) (r) (r) (r) (r) = pji (¯n − ei + ej ) H (n¯ − e¯ ) (r) (r) (r) j∈Cr R r H (¯n − ei )

HR(n¯) (r) (r) (r) + p0i (¯n − ei ) HR(n¯ − e¯r) 6.4 Decomposition into clusters: routing 129

(r) (r) (4.3) H (n¯) H (¯n ) = R H (¯ − ¯ ) (r) (r) (r) R n er H (¯n − ei ) (4.5) H(¯n) = . H(¯n − ei) 2

A decomposition of the routing probabilities similar to (4.1) is introduced in [27]. This reference concentrates on the notion of blocking at global level and on the characterization of product-form global blocking structures. More precisely, it purely concentrates on the global traﬃc equations (4.2). The local routing probabilities were assumed to be state- independent. Most notably, a decomposition result of the form (4.5) into a global and local solution was not concluded. Therefore, Theorem 4.4 generalizes the results obtained in [27].

4.2 Routing: Examples The following examples illustrate the decomposition of the routing characteristics into a global and a local component. Example 4.5 illustrates the decomposition in the case of state-independent routing. This example also shows that, at least for the case of state-independent routing, a decomposition of H into global and local components satisfying the global and local traffic equations is possible if a solution exists to the standard traffic equations. This justifies Conditions 4.2, and 4.3. Example 4.6 is adopted from [27] and considers state-dependent global routing. In this example a capacity constraint is imposed on a subnetwork, say no more than U2 customers are allowed to be present at cluster 2 simultaneously. To retain a solution to the global traffic equations various blocking protocols are allowed. As an illustration the global recirculation protocol is discussed. This protocol states that if a cluster becomes saturated, then customers departing from non-saturated clusters are rerouted into the cluster from which they departed. Finally, Example 4.7 shows that the decomposition of the routing characteristics into a global and a local component must be based on the structure of the queueing network. Most notably, blocking effects play a crucial role in this decomposition. Example 4.5 (State-independent routing) Assume that the routing probabilities are state-independent, that is for alln ¯ ∈ S (r) (r) (r) pij (¯n ) = pij , i, j ∈ Cr ∪ {0}, prs(n¯) = prs, r, s ∈ R ∪ {0}. 130 Norton’s theorem

(r) Assume that {a }r∈R is a solution of the global traffic equations (r) X (s) sr 0r a = a p + µ0p , r ∈ R, (4.7) s∈R (r) and, for each r, that {ci }i∈Cr is a solution of the local traffic equations (r) X (r) (r) ci = cjpji + p0i , i ∈ Cr. (4.8) j∈Cr These assumptions imply that Conditions 4.2, and 4.3 are satisfied with

n(r) Y (r)nr (r) (r) Y (r) j HR(n¯) = a ,H (¯n ) = cj . (4.9) r∈R j∈Cr N Furthermore, {ci}i=1 defined as (r) (r) ci = ci a , i ∈ Cr, r ∈ R, (4.10) is a solution to the standard traffic equations N X ci = cjpji + µ0p0i, i = 1,...,N. (4.11) j=1 This shows that the solution H to (3.4) indeed satisfies (4.5) as (4.9) and (4.10) imply

N (r) r n Y nk Y (r)n Y (r) j Y (r) (r) H(¯n) = ck = a cj = HR(n¯) H (¯n ). k=1 r∈R j∈Cr r∈R Furthermore, this example illustrates that the total traffic equations (4.11) can be solved by solving a number of subproblems (4.8) and a global problem (4.7). The reasoning above can be reversed to show that Conditions 4.2, and 4.3 are satisfied if a solution exists to the traffic equations (4.11). To N (r) this end, note that if {ci}i=1 is a solution to (4.11) then {a }r∈R defined as (r) X (r) a = cjpj0 , r ∈ R, j∈Cr (r) is a solution to the global traffic equations (4.7), and {ci }i∈Cr, r∈R defined as c c(r) = i , i ∈ C , i P (r) r j∈Cr cjpj0 is a solution to the local traffic equations (4.8) as can be seen by substitution. 2 6.4 Decomposition into clusters: routing 131

Example 4.6 (Global recirculation protocol) Consider the following modification of Example 4.5 for closed queueing networks. A capacity constraint is imposed on cluster r, stating that at most Ur customers are allowed to be present at cluster r simultaneously. As arrivals into cluster r are blocked when this cluster is saturated, i.e. when nr = Ur, the global routing probabilities must be modified. To retain a solution to the global traffic equations (4.2), a modification of the global routing probabilities is obtained from the global recirculation protocol. This protocol states that, when nr = Ur, departures from all other clusters s 6= r, and arrivals into the queueing network are prohibited, which is achieved by letting customers departing from cluster s 6= r recirculate into cluster s, while customers arriving to the network are blocked. As a consequence, no more than one cluster can become saturated at the same time. (For an extensive discussion of global blocking phenomena, see [27].) For the global recirculation protocol, the global routing probabilities take the form ( prs if n < U for all k 6= r, prs(n¯) = k k 1 if nk = Uk for some k 6= r for r = s. As a consequence, with C the set of admissible states,

C = {n¯ : nr ≤ Ur, for all r ∈ R, and nr + ns < Ur + Us for any r 6= s}, the global traﬃc equations require for each subnetwork r ∈ R, for n¯ ∈ C:

HR(n¯) = HR(n¯)1{nk = Uk for some k 6= r}. (4.12)

X sr + HR(n¯ − e¯r + e¯s)1{n¯ − e¯r + e¯s ∈ C}1{nk < Uk for all k 6= s}p s∈R, s6=r

Observe that nk = Uk for some k 6= r, say nq = Uq, implies that 1{nk < Uk for all k 6= s} = 1 iff s = q. This gives for all s 6= r that n¯ −e¯r +e¯s 6∈ C if nk = Uk for some k 6= r, which immediately gives that HR given in Example 4.5 is a solution to (4.12). As the local routing characteristics are not affected by this modification, (4.9) remains valid. This example shows that blocking at global level is possible, while re- taining a solution of the global traffic equations, and without affecting the (r) (r) local routing characteristics, pij and H . Blocking at global level does affect the input process of the subnetworks. Therefore, global blocking does affect the local behaviour of the subnetworks. However, the input process is a global process. In Theorem 4.4 subnetworks are analysed with constant (unit) input rate. As a consequence, changing of the global 132 Norton’s theorem input process to the subnetworks does not affect the local solution H(r). Changing the input process to the subnetworks is a scaling effect for H and is incorporated into HR. 6 9 8 7 ? 6 rCluster 3 r

? 1 2 3 - 4 5 6 6 Cluster 1 Cluster 2 rrrr ? U2

Figure 6.4. Capacity constraint U2 at cluster 2

As an illustration, consider the cyclic queueing network consisting of 9 stations grouped into 3 clusters as depicted in Figure 6.4. A capacity constraint is imposed on cluster 2, stating that the total number of customers present at cluster 2 cannot exceed U2. As a consequence of the global recirculation protocol, when cluster 2 is saturated departures from stations 3 and 9 are recirculated into stations 1 and 7 respectively. As the routing probabilities are modiﬁed by the global recirculation protocol, the traﬃc equations change too as is shown in the table below.

System without blocking System with blocking Traﬃc equations (3.4) 1 H(¯n) = H(¯n − ei + ei+1), ∀i, ∀n¯ H(¯n) = H(¯n − ei + ei+1), i 6= 1, 7, ∀n¯ 2 H(¯n) = H(¯n − ei + ei+1), i = 1, 7, n2 < U2 3 H(¯n) = H(¯n − ei + ei+2), i = 1, 7, n2 = U2 Global equations (4.2) 4 H(n¯) = H(n¯ − e¯r + e¯r+1), ∀r, ∀n¯ H(n¯) = H(n¯ − e¯r + e¯r+1), ∀r, n2 < U2 5 H(n¯) = H(n¯ − e¯r + e¯r+1), r = 2, n2 = U2 6 H(n¯) = H(n¯), r = 1, 3, n2 = U2 Local equations (4.3) (r) (r) (r) (r) (r) (r) (r) (r) (r) (r) 7 H (¯n ) = H (¯n + ei ), H (¯n ) = H (¯n + ei ), i = 1, 4, 7 i = 1, 4, 7 (r) (r) (r) (r) (r) (r) (r) (r) (r) (r) (r) (r) 8 H (¯n ) = H (¯n − ei + ei+1), H (¯n ) = H (¯n − ei + ei+1), i = 2, 3, 5, 6, 8, 9 i = 2, 3, 5, 6, 8, 9 6.4 Decomposition into clusters: routing 133

Note that the traffic equations (3.4) are modified at (3) only. When cluster 2 is non-saturated a positive rate out of station 7 is balanced by a positive rate in from station 6. However, when cluster 2 is saturated this rate out is balanced by a positive rate in from station 9. This modification is due to the global recirculation protocol and is due to the modification of prs(n¯) only. This can immediately be seen from the global and local traffic equations. As the subnetworks in isolation are analysed using a state-dependent input process with unit rate, relation (7) and (8) are not affected by global blocking phenomena. The global traffic equations, P however, are modified if n2 = U2. As j∈Cr nj − (ei)j + (ei+1)j = nr = P j∈Cr nj, (6) completely characterizes the modification (3). 2

M5 ? Jackson - 4 5 6 6 network rrrr

U2 ?

Figure 6.5. Capacity constraints M5 at station 5 and U2 at cluster 2

Example 4.7 (Choice of clusters) In Assumption 4.1 and in Exam- ples 4.5, and 4.6 the decomposition of the routing probabilities is imposed on the queueing network. In practical applications, however, this decomposition must be established based on the structure of the queueing network. This example shows that blocking effects play a crucial role in this decomposition. To this end, consider the queueing network consisting of a Jackson subnetwork and a tandem subnetwork as illustrated in Fig- ure 6.5. Assume a capacity constraint U2 is imposed on the total number of customers at stations 4, 5 and 6 simultaneously, and in addition that the number of customers at station 5 is constrained not to exceed M5. If the global behaviour of the Jackson subnetwork and the tandem subnetwork is of interest only, a natural decomposition, similar to Example 4.6, would be to aggregate both the Jackson subnetwork and the tandem subnetwork into a single station, where, as global blocking is involved, the global recirculation protocol is used to modify the global routing probabilities. However, the capacity constraint at station 5 affects the global routing too. This can be seen as follows. If station 5 is saturated, customers cannot route from station 4 to station 5. Therefore, according 134 Norton’s theorem to the recirculation protocol, these customers are rerouted into station 4. Similarly, customers leaving the Jackson subnetwork are rerouted. There- fore, the global routing is affected by the state of station 5. As this is excluded by Assumption 4.1, the decomposition into the Jackson subnetwork and the tandem subnetwork cannot be modelled in the framework of this chapter. However, the queueing network can be decomposed into the Jackson cluster and stations 4, 5 and 6 seen as clusters 4, 5 and 6, i.e. the tandem cluster cannot be aggregated, but the Jackson cluster can be aggregated into a single station. 2

4.3 Service: Conditions In Section 4.1 the solution to the traﬃc equations is decomposed into a global and a local component due to a similar decomposition of the routing probabilities. This section aims to obtain similar decomposition results for the service-mechanism. The equilibrium distribution will then factorize into global and local components. From the natural decomposition of the routing probabilities (4.1), Sec- tion 4.1, shows that the solution to the traﬃc equations consists of a global part describing the global routing characteristics and a local part describing the routing characteristics at the subnetworks. The following natural assumption guarantees a similar decomposition-result for the service part of the equilibrium distribution. Assumption 4.8 (Service function φ) Assume that the service function φ has the form Y (r) (r) φ(¯n) = φR(n¯) φ (¯n ), (4.13) r∈R (r) where φR(0) = 1, and φ (0) = 1, r ∈ R. By (4.5) and (4.13) the equilibrium distribution (3.5) decomposes into global and local routing and service components:

!n(r) Y Y 1 k π(¯n) = Bφ (n¯)H (n¯) φ(r)(¯n(r))H(r)(¯n(r)) , n¯ ∈ S. R R µ r∈R k∈Cr k (4.14) Assumption 4.8 establishes a decomposition of the equilibrium distribution into a global and a local part. In this case, it is natural to assume that the total service-mechanism decomposes into a global and local part too. This is guaranteed by the following assumption. 6.4 Decomposition into clusters: service 135

Assumption 4.9 (Service function ψ) Assume that the service function ψ has the form

Y (r) (r) ψ(¯n) = ψR(n¯) ψ (¯n ). (4.15) r∈R Assumptions 4.8, and 4.9 imply that the service-rate (3.1) decomposes into a global and a local part:

ψ (n¯ − E ) ψ(r)(¯n(r) − e(r)) µ (¯n) = µ R s Y i , i ∈ C . i i ¯ (r) (r) s φR(n) r∈R φ (¯n )

4.4 Service: Examples The following examples illustrate the service-rates included in the framework of Section 3, and the decomposition into global and local service- characteristics. In Example 4.10 the practical service-rates described in Example 3.2 are decomposed into a global and a local component. Exam- ples 4.11 and 4.12 illustrate that modiﬁcation of the global state-dependent service-rates helps to avoid congestion. Example 4.10 (Factorizing form) The service function given in Ex- ample 3.2 factorizes into a global and a local component. To this end, note that φ can be written in the form (4.13) with

 −1  −1 nr nj Y Y (r) (r) Y Y φR(n¯) =  Fr(k) , φ (¯n ) =  fj(k) . (4.16) r∈R k=1 j∈Cr k=1 2

Example 4.11 (Service-speed reduction to avoid congestion) This example shows that due to the total global state dependency of the service functions ψ and φ the service-rate at the clusters can be chosen such that it helps to avoid congestion. To this end, reconsider the service- rates introduced in Example 3.2:

µi(¯n) = µifi(ni)Fr(nr), i ∈ Cr. (4.17) In addition, assume that the number of customers at cluster 2 is constrained not to exceed U2. As is shown in Example 4.6, as soon as the total number of customers at cluster 2 reaches its upper bound customers leaving the other clusters are recirculated to be processed by the same cluster once more. As a more smooth alternative, the service-speed at clusters 136 Norton’s theorem s 6= 2 can be slowed down before cluster 2 reaches its upper bound. For 1 example, the service-speed at clusters s 6= 2 may be reduced by 2 if the number of customers at cluster 2 exceeds some given value L2. As the input process usually cannot be modiﬁed, this process remains unchanged. This gives 1/2

¨¨* JN 3 H ¨ HH ¨¨ H 1/2 ¨ HHj - JN 1 JN 2 - H ¨¨* H 1/2 ¨ HH ¨ H ¨ Hj JN 4 ¨

Figure 6.6. Service speed reduction

 µ f (n )F (n ), i ∈ C , r = 2,  i i i r r r   µifi(ni)Fr(nr), i ∈ Cr, r 6= 2, n2 < L2, µˆi(¯n) = 1  µifi(ni)Fr(nr), i ∈ Cr, r 6= 2, n2 ≥ L2,  2   µ0, i = 0.

This service-rate can be written in the form (3.1). To this end, with φR given in (4.16), deﬁne   φR(n¯), if n2 < L2, ˆ  φR(n¯) = P (4.18)  k6=2 nk  2 φR(n¯), if n2 ≥ L2.

Thenµ ˆi(¯n) can be written

φˆ (n¯ − E ) φ(r)(¯n(r) − e(r)) µˆ (¯n) = µ R s Y i , i ∈ C . i i ˆ (r) (r) s φR(n¯) r∈R φ (¯n ) Note that the local service functions are not affected by this modification. As the service-speed reduction is a global effect, this is what one should expect. Further note that in the case of a closed queueing network service- speed reduction at all clusters k 6= 2 is equivalent to increasing the service- P speed at cluster 2 as can be seen from (4.18) by noting that k6=2 nk = N − n2, where N is the total number of customers present at the system. In the case of an open queueing network this equivalence is lost. 2 6.5 Norton’s theorems 137

stop

¨* JN 3 H ¨¨ H ¨ HH stop¨¨ HHj stop - JN 1 JN 2 - H ¨* H stop ¨ H ¨¨ HH ¨ Hj JN 4 ¨

Figure 6.7. Stop service and arrivals

Example 4.12 (Stop protocol) In the previous example service at clusters k 6= 2 is slowed down if n2 ≥ L2. A more drastic approach is to stop service at all clusters as well as arrivals from the outside if the number of customers at cluster 2 reaches some upper bound U2. The service-rate given in (4.17) is then modiﬁed to

 µ f (n )F (n ), i ∈ C , r = 2,  i i i r r r   µifi(ni)Fr(nr), i ∈ Cr, r 6= 2, n2 < U2, µ˜i(¯n) =  µ0, i = 0, n2 < U2,   0 otherwise.

This service-rate can be written in the form (3.1). To this end, with φR given in (4.16), deﬁne

( ˜ φR(n¯), if n2 < U2, ψR(n¯) = 0, if n2 = U2.

Thenµ ˜i(¯n) can be written

ψˆ (n¯ − E ) φ(r)(¯n(r) − e(r)) µ˜ (¯n) = µ R s Y i , i ∈ C , s ∈ R ∪ {0}. i i ¯ (r) (r) s φR(n) r∈R φ (¯n ) Note that this example cannot be obtained as a special limiting case of the previous example: in the formulation of Example 4.11 the service-speeds are required to be positive, and the arrival rate µ0 is not aﬀected, whereas here it has to be set equal to zero here. 2 138 Norton’s theorems

5 Norton’s theorems

In Section 4 the transition rates (3.2) are decomposed into a global and local part such that the equilibrium distribution decomposes similarly. In Section 4.1, motivated by electrical circuit theory, the routing probabilities are decomposed such that the routing characteristics at global level and at local level can be analysed separately. In Section 4.3, the service- rates are decomposed such that the equilibrium distribution decomposes into a global and a local part. Although the form (4.14) is build up by separate components associated with global and local characteristics, it is not yet clear whether it can indeed be seen as a full decomposition of the equilibrium distribution into a purely global process and the marginal distributions of purely local processes, or allows aggregation of subnetworks into single stations with appropriate service-rates without affecting the equilibrium distribution of the rest of the queueing network. Based on the decomposition of the transition rates given in Section 4, this section derives these results. This section first gives Norton’s theorem in weak form. This theorem allows separate calculation of the global and local characteristics of the queueing network. Second, Norton’s theorem in strong form is formulated. In addition to the applications of Norton’s theorem in weak form, this theorem gives a formal justification for determining global characteristics via monitoring of the behaviour of the queueing network at global level (e.g. monitoring of the state-dependent service-rate and state-dependent routing at global level), and shows that conditional on the global state of the queueing network, the subnetworks are independent. Furthermore, these theorems imply that aggregation of subnetworks into single stations does not affect the behaviour of the rest of the queueing network. This extends Norton’s theorem to queueing networks with state-dependent routing and service. The following theorem establishes a decomposition of the process with transition rates (3.2) into a global process and a local process such that, except for normalization, the equilibrium distribution of the process decomposes into a global equilibrium distribution determined by the global process and a local equilibrium distribution factorizing into the subnetworks determined by the local processes at the subnetworks. Theorem 5.1 (Norton’s theorem in weak form) (i) Service-rates If the process with transition rates (3.2) satisfies Assumptions 4.1, 4.8, 4.9 and Conditions 4.2, 4.3, then the service function (3.1) can be written 6.5 Norton’s theorems 139 as Y (s) (s) µi(¯n) = Mr(n¯) µi (¯n ), i ∈ Cr, r ∈ R ∪ {0}, (5.1) s∈R where Mr is the global service-rate defined by  ψ (n¯ − e¯ )  R r  , if r ∈ R,  φR(n¯) Mr(n¯) = (5.2)  ψ (¯)  R n  µ0 , if r = 0,  φR(n¯)

(s) while µi is the local service-rate at cluster s deﬁned by

 (s) (s) (s)  ψ (¯n − ei )  µ , if i ∈ C ,  i (s) (s) s (s) (s)  φ (¯n ) µi (¯n ) = (5.3)  (s) (s)  ψ (¯n )  , if i ∈ Cr, r 6= s.  φ(s)(¯n(s))

(ii) Global and local processes With BR a normalizing constant, the distribution

πR(n¯) = BRφR(n¯)HR(n¯) (5.4) is the equilibrium distribution of the global process with transition rates deﬁned as

rs qR(n¯, n¯ − e¯r + e¯s) = Mr(n¯)p (n¯), r, s ∈ R ∪ {0}. (5.5)

With B(r) a normalizing constant, the distribution

n(r) 1 ! k π(r)(¯n(r)) = B(r)φ(r)(¯n(r))H(r)(¯n(r)) Y (5.6) µ k∈Cr k is the equilibrium distribution of the local process with transition rates deﬁned as

(r) (r) (r) (r) (r) (r) (r) (r) (r) q (¯n , n¯ − ei + ej ) = µi (¯n )pij (¯n ), i, j ∈ Cr ∪ {0}. (5.7) Furthermore, with B a normalizing constant,

Y (r) (r) π(¯n) = BπR(n¯) π (¯n ), n¯ ∈ S. (5.8) r∈R 140 Norton’s theorems

Proof (5.1) follows by insertion of Assumptions 4.8, and 4.9 into (3.1). Insertion of (5.4), (5.5) and (5.6), (5.7) into the appropriate global balance equations establishes that (5.4) and (5.6) are indeed the equilibrium distributions at global and local level. Finally, (5.8) is a direct consequence of Theorem 4.4. 2

Although Theorem 5.1 establishes a full decomposition of the equilibrium (r) distribution, π, into a global component, πR, and local components, π , r ∈ R, it does not allow computation of the global part of the equilibrium distribution by simply monitoring the output processes of the subnetworks. This can be seen by observing that πR is not the equilibrium distribution of the aggregated network. Summing π over all statesn ¯ such P that j∈Cr nj = nr, r ∈ R, leads to the aggregated equilibrium distribution, ΠR:

X Y (r) ΠR(n¯) ≡ π(¯n) = BπR(n¯) π (nr), (5.9) (r) n¯ :nr, r∈R r∈R

(r) (r) P (r) wheren ¯ : nr is short forn ¯ : j∈Cr nj = nr. As Theorem 5.1 establishes a decomposition of the process with transition rates (3.2) into a global process and a local process similar to the decomposition in Norton- type results it is called Norton’s theorem in weak form. This decomposition is such that:

• the global and local transition rates can be immediately obtained from the transition rates of the original process;

• the local process factorizes into the subnetworks;

• the global process and the local processes at the subnetworks can be analysed separately;

• except for normalization, the equilibrium distribution of the original process consists only of a global part determined by the global process and a local part factorizing into the subnetworks determined by the local processes at the subnetworks.

As is discussed above, Norton’s theorem should give a full decomposition of the equilibrium distribution into a global part and a local part such that the global part can be computed via monitoring of the output processes of the subnetworks. To this end, the transition rates of the global process 6.5 Norton’s theorems 141

must be such that ΠR as defined in (5.9) is the equilibrium distribution of the global process and such that the probability flows between subnetworks in the global process equals the probability flows between subnetworks in the original non-aggregated process. The following theorem establishes this decomposition.

Theorem 5.2 (Norton’s theorem in strong form) (i) Global process Deﬁne the global process with transition rates

 Ψ (n¯ − e¯ )  R r rs ¯  p (n), if r ∈ R,  ΦR(n¯) Q(n¯, n¯ − e¯r + e¯s) = (5.10)  Ψ (¯)  R n 0s  µ0 p (n¯), if r = 0,  ΦR(n¯) and service functions ΨR, ΦR given by

(r) !nj Y X (r) (r) (r) (r) Y 1 ΨR(n¯) = ψR(n¯) ψ (¯n )H (¯n ) , (5.11) (r) µj r∈R n¯ :nr j∈Cr (r) !nj Y X (r) (r) (r) (r) Y 1 ΦR(n¯) = φR(n¯) φ (¯n )H (¯n ) . (5.12) (r) µj r∈R n¯ :nr j∈Cr If the process with transition rates (3.2) satisﬁes Assumptions 4.1, 4.8, 4.9 and Conditions 4.2, 4.3 then, with BR a normalizing constant, the global process possesses an equilibrium distribution, Πr, given by

ΠR(n¯) = BRΦR(n¯)HR(n¯). (5.13)

(ii) Aggregated process If for all r ∈ R the service function ψ(r) satisﬁes the following technical assumption for all i and all nr

(r) !nj X X (r) (r) (r) (r) (r) (r) (r) Y 1 ψ (¯n − ei )H (¯n )µipi0 (¯n ) (r) µj n¯ :nr i∈Cr j∈Cr n(r) 1 ! j = X ψ(r)(¯n(r))H(r)(¯n(r)) Y (5.14), (r) µj n¯ :nr−1 j∈Cr 142 Norton’s theorems

then the global process equals the aggregated process, that is ΠR is the aggregated equilibrium distribution: X ΠR(n¯) = π(¯n), (5.15) (r) n¯ :nr, r∈R and the probability flow between aggregated subnetworks equals the probability flow between subnetworks in the original non-aggregated queueing network: X X ΠR(n¯)Q(n¯, n¯ − e¯r + e¯s) = π(¯n)q(¯n, n¯ − ei + ej), (5.16) n¯:nt, t∈R i∈Cr, j∈Cs where for r = s the summation in the right-hand side is over all transitions in which a customer first leaves cluster r and then routes back to cluster r.

(iii) Equilibrium distribution factorizes With π(r) the equilibrium distribution of the local process at cluster r, (r) (r) (r) and π (¯n |nr) the conditional probability of n¯ given nr customers at cluster r, the equilibrium distribution, π, of the original process can be written Y (r) (r) π(¯n) = ΠR(n¯) π (¯n |nr). (5.17) r∈R

Proof From Condition 4.2, insertion of ΠR as given in (5.13) into the appropriate global balance equations gives that ΠR is the equilibrium distribution of the global process. Insertion of (5.4),(5.6),(5.8) into the right-hand side of (5.15) gives

X (5.8) X Y (r) (r) π(¯n) = BπR(n¯) π (¯n ) (r) (r) n¯ :nr, r∈R n¯ :nr, r∈R r∈R ! (5.4),(5.6) Y (r) = BBR B ΦR(n¯)HR(n¯), r∈R which gives (5.15). Insertion of the speciﬁc form (5.10) for Q, (5.13) for ΠR, (3.1), (3.2) for q, (3.5) for π, Assumptions 4.1, 4.8, 4.9, Condi- tions 4.2, 4.3, and Theorem 4.4 for H into (5.16) gives that (5.16) holds if and only if the following relation holds true.

rs rs BRHR(n¯)ΨR(n¯ − e¯r)p (n¯) = BHR(n¯)ψR(n¯ − e¯r)p (n¯)  (k)  n 1 ! j X Y (k) (k) (k) (k) Y  ×  ψ (¯n )H (¯n )  µj n¯:nt, t∈R k6=r j∈Ck 6.5 Norton’s theorems 143

n(r) 1 ! j × X µ ψ(r)(¯n(r) − e(r))H(r)(¯n(r))p(r)(¯n(r)) Y i i i0 µ i∈Cr j∈Cr j

Relations (5.11), and (5.14) imply that this equation is satisﬁed. Finally, P (r) from Theorem 5.1, if j∈Cr nj = nr

n(r) φ(r)(¯n(r))H(r)(¯n(r)) Q 1 j (r) (r) j∈Cr µj π (¯n |nr) = . n(r) P (r) (r) (r) (r) Q 1 j (r) φ (¯n )H (¯n ) n¯ :nr j∈Cr µj

(5.12), (5.13) and Theorem 4.4 now imply (5.17). 2

At first glance, the results of Norton’s theorem in strong from are similar to the results of Norton’s theorem in weak form. However, the global process with transition rates qR as defined in Theorem 5.1 is not the same as the global process Q defined in Theorem 5.2. In contrast to the global process qR which can immediately be obtained from the transition rates, the global process Q must be computed from the transition rates and cannot as easily be obtained as qR. The result of Norton’s theorem in strong form, however, is much stronger than the result of Norton’s theorem in weak form. In Theorem 5.2 the decomposition into global and local distributions, (5.17), is complete whereas in (5.8) the normalizing constant relates the global and local processes and must still be computed. Furthermore, the global process of Theorem 5.2 describes the service-rates of the subnetworks, whereas the global process of Theorem 5.1 merely describes the global part of the transition rates. Norton’s theorem in strong form establishes a decomposition of the process with transition rates (3.2) into a global process and a local process such that:

• the local transition rates can be immediately obtained from the transition rates of the original process;

• the local process factorizes into the subnetworks;

• the global process is the process aggregated over the clusters, i.e. the global equilibrium distribution equals the aggregated equilibrium distribution and the probability ﬂow between stations of the global process equals the probability ﬂow between clusters of the original process; 144 Norton’s theorems

• when the transition rates of the global process are determined, the global and local processes can be analysed separately; • the equilibrium distribution of the original process consists only of a global part determined by the global process and a local part factorizing into the subnetworks determined by the local processes at the subnetworks. In addition to the points mentioned above, the following corollary gives a major non-trivial result following from Theorem 5.2. Corollary 5.3 (Conditional independence of subnetworks) Con- ditional on the global state of the queueing network the subnetworks are independent: Y (r) (r) π(¯n|n¯) = π (¯n |nr), n¯ ∈ S. r∈R Proof (5.15) and the deﬁnition of conditional probability imply that π(¯n) = ΠR(n¯)π(¯n|n¯), and (5.17) completes the proof. 2

The main result of Norton’s theorem as appearing in the literature is that the marginal distribution of the subnetwork of interest is the same in the original queueing network and in the partly aggregated queueing network. As the equilibrium distribution π as given in (3.5) can, due to the aggregation method, be written in the form (5.17) this result is an immediate consequence of Norton’s theorems. This is stated in the following corollary, where for simplicity of formulation it is assumed that ψ = φ. Note that, if ψ = φ the service-rate µi(¯n) as given in (5.1) in Theorem 5.1 depends on the global state and the local state of cluster r only: (r) (r) µi(¯n) = Mr(n¯)µi (¯n ), i ∈ Cr. Furthermore, note that the property ψ = φ carries over to the global process of Theorem 5.2, i.e. ψ = φ implies ΨR = ΦR, as can immediately be seen from (5.11), and (5.12). Corollary 5.4 (Marginal distribution) Assume that ψ = φ. Then the marginal distribution, P (¯n(k)), of n¯(k) customers at cluster k in the queueing network in which all clusters except cluster k are aggregated into single stations with service-rates  ΦR(n¯ − e¯r)  , if r ∈ R, r 6= k,  Φ (¯) Mr(n¯) = R n   µ0, if r = 0, 6.5 Norton’s theorems 145 is the same as the marginal distribution, π(¯n(k)), of n¯(k) customers at cluster k in the original queueing network.

Proof From (5.1) and ψ = φ, the service-rate µi(¯n) for serving a customer at station i of cluster k for the partly aggregated queueing network is given by (k) (k) (k) φR(n¯ − e¯r) φ (¯n − ei ) µi(¯n) = µi (k) (k) , i ∈ Ck. φR(n¯) φ (¯n ) From (5.12) this can be written

(k) (k) (k) φ (¯n − ei ) n(k) P (k) (k) (k) (k) Q 1 j (k) φ (¯n )H (¯n ) ΦR(n¯ − e¯k) n¯ :nk−1 j∈Ck µj µi(¯n) = µi (k) (k) , ΦR(n¯) φ (¯n ) n(k) P (k) (k) (k) (k) Q 1 j (k) φ (¯n )H (¯n ) n¯ :nk j∈Ck µj where H(k) is the solution to the traﬃc equations for cluster k if cluster k would be considered in isolation. Form Norton’s theorem in weak form

(k) X P (¯n ) ∝ ΦR(n¯)HR(n¯) ¯ P (k) n: nk= n j∈Ck j n(k) (k) (k) (k) (k) Q 1 j φ (¯n )H (¯n ) j∈C × k µj . n(k) P (k) (k) (k) (k) Q 1 j (k) φ (¯n )H (¯n ) n¯ :nk j∈Ck µj From Norton’s theorem in strong form, by summation

π(¯n(k)) = X π(¯n) (k) n¯: nj =nj X (r) (k) = ΦR(n¯)π (¯n |nk) ¯ P (k) n: nk= n j∈Ck j X = BR ΦR(n¯)HR(n¯) ¯ P (k) n: nk= n j∈Ck j n(k) (k) (k) (k) (k) Q 1 j φ (¯n )H (¯n ) j∈C × k µj , n(k) P (k) (k) (k) (k) Q 1 j (k) φ (¯n )H (¯n ) n¯ :nk j∈Ck µj 146 Norton’s theorem which completes the proof. 2

Remark 5.5 (Assumptions justifying (5.14)) As is mentioned in Theorem 5.2, relation (5.14) is a technical assumption. This assumption is used only to show that (5.16) holds true. Note that this technical assumption is not a very stringent condition and is satisﬁed in many practical cases. For example, if the state space is such that the set of admissible states for subnetwork r is coordinate convex, that is

(r) (r) X (r) (r) C = {n¯ : nj ≤ U }, r ∈ R, (5.18) j∈Cr then the left-hand side of (5.14) can be written

(r) X X (r) (r) (r) (r) (r) (r) B · LHS = π (¯n )q (¯n , n¯ − ei ) (r) n¯ : nr i∈Cr X X X (r) (r) (r) (r) (r) (r) (r) = π (¯n )q (¯n , n¯ − ei + ej ) (r) n¯ : nr i∈Cr j∈Cr∪{0} X X X (r) (r) (r) (r) (r) (r) (r) − π (¯n )q (¯n , n¯ − ei + ej ) (r) n¯ : nr i∈Cr j∈Cr (4.3) X X X (r) (r) (r) (r) (r) (r) (r) (r) (r) = π (¯n − ei + ej )q (¯n − ei + ej , n¯ ) (r) n¯ : nr i∈Cr j∈Cr∪{0} X X X (r) (r) (r) (r) (r) (r) (r) − π (¯n )q (¯n , n¯ − ei + ej ) (r) n¯ : nr i∈Cr j∈Cr X X (r) (r) (r) (r) (r) (r) (r) = π (¯n − ei )q (¯n − ei , n¯ ) (r) n¯ : nr i∈Cr (5.18) X X (r) (r) (r) (r) (r) (r) = π (¯n )q (¯n , n¯ + ei ), (r) n¯ : nr−1 i∈Cr which immediately implies that (5.14) is satisﬁed. 2

6 Practical applications of Norton’s theorems

Norton’s theorems establish a decomposition of the process with transition rates (3.2) into a global process and a local process. This decomposition is more difficult to establish in Norton’s theorem in strong form than in Norton’s theorem in weak form. The result of Norton’s theorem in strong form, however, is indeed stronger than the result of Norton’s theorem in 6.6 Practical applications 147 weak form. The following points characterize practical applications of Norton’s theorems. The first two points are valid for both theorems. The last three points are valid for Norton’s theorem in strong form only. 1. For large queueing networks, determination of a solution to the (state-dependent) traffic equations is (computationally) very hard. Norton’s theorem implies that the global and local traffic equations can be analysed separately. As the local traffic equations factorize into the subnetworks, this gives much smaller problems. The solution to the total traffic equations is just the product of the global and the local parts. 2. Calculation of the normalization constant for large processes (i.e. processes with a large state space) is, in general, very difficult. Nor- ton’s theorem allows to first analyse the normalization constants at local level and subsequently to analyse the normalization constant at global level. As the local and global state spaces are substantially smaller than the total state space, these problems are substantially smaller than the original problem. Note, however, that the total computational effort needed to determine the normalizing constant is not reduced by this decomposition. 3. If the global characteristics of the queueing network, i.e. characteristics depending on the global state of the queueing network only, are of interest, Norton’s theorem in strong form allows to only analyse the queueing network at global level. To this end, note that the global routing probabilities can be determined by observing the routing between clusters in the original queueing network. The global service functions, ΨR,ΦR, can be obtained by monitoring the state- dependent speed at which customers leave the clusters. From the thus obtained transition rates the global equilibrium distribution can be obtained. From this global equilibrium distribution the global characteristics can be determined. 4. If characteristics depending on the local state of a single cluster are of interest only, Norton’s theorem in strong form allows to analyse the queueing network at a global level as discussed in (3). For the cluster of interest, Norton’s theorem allows to analyse this cluster with transition rates as given in Theorem 5.1 separately. The equilibrium distribution of the queueing network containing full information on the state of the cluster of interest is then given by the 148 Norton’s theorem

global equilibrium distribution multiplied with the distribution of the cluster at interest conditional on the global state of that cluster. This method can also be applied when characteristics depending on the local state of several clusters are of interest.

5. If the transition rates of the clusters in isolation (i.e. not part of the queueing network) are known, then the service functions ΨR,ΦR can simply be calculated from the functions ψ(r) and φ(r) as obtained from the transition rates of the clusters in isolation. When ΨR and ΦR are calculated, the global behaviour of the queueing network can be analysed without considering the local behaviour.

As practical implications of these results the following two methods can be proposed for analysing the equilibrium distribution for queueing networks of which the routing and service-mechanisms can be decomposed into local and global components:

Monitoring method

1. Determine the global routing probabilities from the global structure of the queueing network.

2. Determine the global service-characteristics by monitoring the state- dependent output process of the global parts of the queueing network.

3. Compute the global equilibrium distribution. This global distribution equals the equilibrium distribution of the original queueing network aggregated over subnetworks.

4. For the subnetworks for which the local distribution is of interest, ﬁrst analyse the subnetwork in isolation with constant (e.g. unit) arrival rates. The equilibrium distribution of the subnetworks in isolation then equals the real equilibrium distribution of the subnetworks when incorporated in the network conditional on the global number of customers present at the subnetwork.

Computational method

1. Compute the equilibrium distribution of the subnetworks in isolation with constant (e.g. unit) arrival rates. 6.7 Examples 149

2. Calculate the global service-characteristics from the equilibrium distribution of the subnetworks and the global part of the original service-characteristics. 3. Compute the global equilibrium distribution from the global component of the routing probabilities and the global service-characteristics. 4. Obtain the equilibrium distribution from the global and local equilibrium distributions.

7 Examples

7.1 Global throughput determines local behaviour; workload balancing This example shows that under global blocking conditions such as capacity constraints on the total number of customers at the clusters it is possible to obtain the local equilibrium distribution from the global throughput of the clusters and thus to obtain both global and local performance measures by merely analysing the global throughputs. To this end, consider a queueing network consisting of N stations grouped into R clusters such that the routing probabilities have the form given in Assumption 4.1. Assume that the workload is to be balanced over the clusters in the sense that the total number of customers at the clusters is constrained to be in the range Lr ≤ nr ≤ Ur, r ∈ R. Furthermore, except for effects due to the boundaries, assume that the service-rate at a cluster depends only on the state of that cluster, and that the global routing probabilities are state-independent except for blocking phenomena arising at the boundaries. As all blocking effects are due to global requirements on the workload at the clusters, the local transition rates are not affected by these blocking effects, i.e. all blocking effects are incorporated in the global process. Let prs be the (r) state-independent part of the global routing probabilities, and {a }r∈R the solution to the global traffic equations (4.7) as given in Example 4.5. Similar to Example 4.6, for Condition 4.2 to be satisfied, the global routing probabilities must be modified according to a blocking protocol. The following blocking protocol is used:

– Customers leaving cluster r when nr = Lr are rerouted into cluster r. This guarantees the minimal workload. 150 Norton’s theorem

– Customers leaving cluster s 6= r, when nr = Ur are rerouted into cluster s according to the global rerouting protocol. This guarantees that the upper boundaries are not exceeded.

By noting that the set of admissible states is restricted to

C = {n¯ :Lr ≤ nr ≤ Ur, for all r ∈ R, and nr +ns < Ur +Us for any r 6= s}, the global traﬃc equations (4.2) now read for r ∈ R, n¯ ∈ C:

HR(n¯) = HR(n¯)1{nk = Uk for some k 6= r or nr = Lr}

X sr + HR(n¯ − e¯r + e¯s)1{n¯ − e¯r + e¯s ∈ C}1{nk < Uk for all k 6= s}p . s∈R, s6=r

As n¯ − e¯r + e¯s 6∈ C if nk = Uk for k 6= r or if nr = Lr, with

Y (r)nr HR(n¯) = a , n¯ ∈ C, (7.1) r∈R Condition 4.2 is satisfied. This specifies the global behaviour of the queueing network. Now consider the local behaviour. Up to normalization constants, a(r) equals the global throughput of cluster r, that is a(r) equals the parameter of the arrival process to cluster r. The transition rates for the local process at cluster r with unit arrival parameter are given in (5.7). Con- dition 4.3 implies that the local equilibrium distribution at cluster r is given in (5.6), where H(r) is obtained by analysing cluster r with unit arrival parameter. The modification (3.6a), (3.6b) shows that the local equilibrium distribution of cluster r with input rate a(r) is given by

n(r) a(r) ! j π(r)(¯n(r)) = B(r)φ(r)(¯n(r))H(r)(¯n(r)) Y . (7.2) µ j∈Cr j Norton’s theorem in weak form 5.1 shows that the marginal distribution of cluster r, π(¯n(r)), is then given by

π(¯n(r)) = π(r)(¯n(r)). (7.3)

Note that the global blocking protocol used here is not essential to the derivation of (7.3). For example, an alternative blocking protocol is the stop protocol, where service is stopped at all non-saturated clusters if the 6.7 Examples 151 number of customers at a cluster reaches its upper bound. For example, in Example 4.6 this requires that the service at all stations 1, 2, 3 and 7, 8, 9 is stopped when cluster 2 is saturated, i.e. when n2 = U2. As is shown in [27], [26], the “stop protocol” and “recirculate protocol” are equivalent under product-form conditions, that is under Assumptions 4.1, 4.8, 4.9 and Conditions 4.2, 4.3. Essential to the derivation of (7.2) is that HR has the form (7.1) as this form contains the throughput, a(r), of cluster r.

7.2 Internal blocking In all examples given so far emphasis was on the global behaviour of the queueing network. In particular, global routing and blocking is treated in detail. From the decomposition of the routing probabilities, however, (r) the internal routing probabilities pij can be state-dependent too. From Assumption 4.1, all examples appearing in the literature on product-form queueing networks with state-dependent routing in which, except for a constraint on the total number of customers present at the queueing network, arrivals cannot be blocked, i.e. for alln ¯(r) customers arriving at the cluster are accepted at a station of the cluster, that is X (r) (r) p0i (¯n ) = 1, i∈Cr can be inserted as subnetwork. As an illustrative example, consider the queueing network consisting of two clusters as depicted in Figure 6.8, where cluster 1 is an arbitrary Jackson network and cluster 2 consists of three stations. Due to capacity constraints, the number of customers at station 2 is constrained not to exceed Z2 and the total number of customers at cluster 2 is not to exceed U2. Assume that, except for blocking phenomena, the routing probabilities are as indicated in Figure 6.8. Un- der the recirculation protocol, if station 2 is saturated customers leaving station 1 and 3 to route to station 2 are blocked and recirculated into station 1 and 3 respectively. If cluster 2 is saturated, customers leaving cluster 1 are recirculated into cluster 1. The local routing probabilities at cluster 2 are given by

(2) (2) (2) (2) 1 (2) p21 (¯n ) = p23 (¯n ) = 2 , for alln ¯ , (2) (2) (2) (2) 1 (2) p10 (¯n ) = p30 (¯n ) = 2 , for alln ¯ , but (2) (2) (2) (2) 1 p12 (¯n ) = p32 (¯n ) = 2 , if n2 < Z2, (2) (2) (2) (2) 1 p11 (¯n ) = p33 (¯n ) = 2 , if n2 = Z2. 152 Norton’s theorem

- 1 1/2- 1/2 61/2 1/2 ? - Jackson 2 network @ 1/2 @ 1/2 61/2 @ ? @@R - - 3 1/2

Figure 6.8. Internal blocking due to capacity constraints M5

The global routing probabilities are given by

12 p (n¯) = 1, if n2 < U2, 11 p (n¯) = 1, if n2 = U2.

The standard solution (4.9) still applies to both the global and the local level.

7.3 Nested aggregation The aggregation procedure described in this chapter can be nested. To describe this roughly, consider a queueing network consisting of N stations. Assume that the stations can be grouped into clusters, say clusters 1,...,R, such that the routing probabilities satisfy Assumption 4.1 and Conditions 4.2, 4.3, and the service-rate satisfies Assumptions 4.8, 4.9. Norton’s theorem in strong form shows that the clusters can be aggregated into single stations. This gives a new queueing network consisting of R stations. At this new queueing network the aggregation procedure can again be applied to obtain a queueing network consisting of R0 < R stations. Nesting of aggregation allows to first aggregate simple structures for which the aggregated service-rates can easily be computed, such as for example a number of M|M|∞-queues or a tandem-line and subsequently aggregate the simplified clusters into single stations. Numerous examples of nested aggregation can be given. However, since each additional nesting basically remains the same, consider one simple example only. Consider the queueing network as depicted in Figure 6.9. 6.8 Literature 153

JN 3 @ @ @ ? - JN 1 - JN 2 - - @@R @ s s s @ s @ @@R JN 4

Figure 6.9. Nested aggregation

In the ﬁrst aggregation step cluster i is aggregated into a single station, i = 1, 2, 3, 4. This gives a 4 station queueing network. In this queueing network station 1 and station 2 can be aggregated into a single station, for example with a total capacity constraint, and stations 3 and 4 can be aggregated into a single station, for example with service-delay depending on the total number of customers at stations 3 and 4. Thus a closed 2 station queueing network is obtained that can be analysed by using standard methods.

8 Literature

8.1 Literature on Norton’s theorem Norton’s theorem for queueing networks is introduced in [13] and states that the marginal distribution of a subnetwork, say σ, in the original queueing network equals the marginal distribution of σ when the rest of the queueing network is aggregated into a single station with service-rate set equal to the throughput of σ when the service-rate of all stations in σ is reduced to zero. [13] prove the aggregation method to be correct for queueing networks of the BCMP-type in which σ consists of a single station only. The proof given in this reference can easily be extended to the case where σ consists of several stations such that customers enter σ through a single station and leave σ from a single station. In [3], [57] Norton’s theorem is generalized to include BCMP-networks in which the routing between subnetworks is allowed to be arbitrary, but state- 154 Norton’s theorem independent. Norton’s theorem is further generalized in [10], [82], where queueing networks consisting of quasireversible subnetworks are discussed. In all of these references the proof is based on the fact that the equilibrium distribution factorizes into the subnetworks and in addition that the routing between subnetworks is state-independent, which implies that customers arriving at a subnetwork are always accepted. Summarizing, these references show that:

1a subnetworks are analysed as closed queueing networks;

2a the service-rate of the aggregated station cannot depend on the state of σ as the state of σ is deleted from the description when the throughput of σ with zero service-rate is determined;

3a the global routing probabilities cannot be state-dependent which excludes blocking.

In this chapter, the decomposition and aggregation of subnetworks is treated diﬀerently. This chapter shows that:

1b subnetworks are analysed as open queueing networks;

2b the service-rate of the aggregated stations is allowed to depend on the full global state;

3b global blocking is included as the global routing probabilities are state-dependent.

Note that 2b and 3b immediately generalize 2a and 3a. Comparison of 1a and 1b shows that subnetworks are analysed differently. This may cause different internal behaviour of the aggregated stations. In contrast, note that the routing probabilities between subnetworks in [3], [57] are more general than the routing probabilities for the state- independent case in this chapter. In [3], [57] arbitrary routing probabilities pij, i ∈ Cr, j ∈ Cs, r, s = 1, 2, are included, whereas this assumes (r) (s) that pij = pi0 p0j . This specific choice is due to the inclusion of blocking phenomena in the local routing probabilities. Note that arbitrary state-independent routing probabilities can easily be incorporated into the framework of this chapter. In this case the routing probabilities have the rs form pij(¯n) = pijp (n¯). If a solution to the traffic equations for pij exists, the local part of the solution to the traffic equations factorizes into the 6.8 Literature 155 stations (cf. [27]). However, in this case the local process does not factorize into the subnetworks. As emphasis is on a complete factorization into the subnetworks of the local equilibrium distribution and process, which allows independent analysing of the subnetworks, the routing probabilities are chosen to be of the form (4.1). Finally, although not mentioned in [3], [13], [57], and [82], Norton’s theorem as presented by these authors can be generalized to allow aggregation of the equilibrium distribution of several subnetworks independently. Again, state-dependent routing cannot be incorporated in the generalization of Norton’s theorem as presented in these references.

8.2 Literature on factorization into subnetworks Results similar to the factorization results obtained here have been reported in the literature. In [61] reversible subnetworks are connected via state-independent global routing and in [55], [80] quasireversible subnetworks are connected via state-dependent global routing. In these references, the transition rates depend on the local state only and a factorization of the equilibrium distribution similar to (5.8) for BπR = 1 is obtained. As this chapter allows global state-dependence too, it generalizes these results.

8.3 Weak coupling In [87, chapters 10, 11] the concept of weak coupling is described. A system is said to be weakly coupled if the equilibrium distribution consists of a part determined by the global routing probabilities and a part independent of the global routing. Thus, weak coupling is a property of the equilibrium distribution. In this chapter, both (5.8) and (5.17) state that the system is weakly coupled as the equilibrium distribution depends on the global routing characteristics through the global components, πR and ΠR, only. If the global routing is state-independent, based on a condition similar to global balance for each cluster separately, [87] proves the system to be weakly coupled. If the global routing is state-dependent, however, the condition stated in [87] for proving weak coupling is that a global balance- like condition is satisfied for the process with global routing removed from the routing characteristics, i.e. with prs(·) deleted from the routing probabilities pij(·). In the case of state-independent routing [87] established a decomposition of the equilibrium distribution into components for each 156 Norton’s theorem subnetwork. In the case of state-dependent routing, however, such a decomposition cannot be concluded. Also, the approach taken in [87] is different from the approach taken here. In [87] emphasis is on establishing a decomposition of the equilibrium distribution into a part determined by the global routing and a part independent of the global routing. A decomposition of the local part is not established. Furthermore, an essential part of Norton’s theorem is that the process decomposes into a global process and a local process. This is guaranteed by the assumptions made in this chapter and cannot be concluded from the parameterization in [87]. Finally, in [87] global balance-like conditions are used when proving weak coupling. Although global balance is more general than local balance as guaranteed by the traffic equations, in queueing networks one usually requires that the traffic equations possess a solution. Therefore, the assumptions on the routing part of the transition rates used in this chapter are less general, but are sufficiently general in practical applications.

8.4 Relation to electrical circuit theory In Section 8.1 the queueing network analogue of Norton’s theorem as described in the literature is compared to the queueing network analogue of Thevenin’s theorem as described in this chapter. Although Norton’s and Thevenin’s theorem for electrical circuits are discussed extensively in Sec- tion 2, up to this moment Norton’s and Thevenin’s theorem for queueing networks have not been related to these theorems. Moreover, the distinc- tion between Thevenin’s theorem and Norton’s theorem for queueing networks is not yet justified. To this end, reconsider Norton’s and Thevenin’s theorem for electrical circuits. Figure 6.2 shows that the replacement circuit for Norton’s theorem is determined by analysing the current I flowing in a closed loop version of the electrical circuit. Similarly, Figure 6.3 shows that the replacement circuit for Thevenin’s theorem is determined by analysing the potential difference V in an open version of the electrical circuit. Current flowing in the electrical circuit is the electrical analogue of customers flowing in the queueing network (throughput). This justifies the name Norton’s theorem for queueing networks as this method is based on analysing the throughput in a closed loop version of the subnetwork. The potential difference can be seen as the electrical analogue of the service-potential ΨR (cf. [87]). As the method presented in this chapter ΦR is based on determining the service-potential in an open network version 6.8 Literature 157 of the subnetworks, the name Thevenin’s theorem for queueing networks would be justified. Note that, as Norton’s theorem has become the standard phrasing in the literature on aggregation in queueing networks, the results obtained in this chapter are called Norton’s theorem. 158 Norton’s theorem Chapter 7

Amalgamation of Markov chains

1 Introduction

Over the last decades considerable attention has been paid to the determination of equilibrium distributions of stochastic processes arising from queueing networks. Most of this work considers product-form equilibrium distributions only. At this moment, for a wide class of stochastic processes product-form equilibrium distributions are proven to exist. However, the class of stochastic processes with a product-form equilibrium distribution is a very restricted class. This chapter aims to extend this class to a class with a more general form of equilibrium distribution. In particular, this chapter considers an amalgamation of stochastic processes each having an equilibrium distribution and provides a so-called cross-balance condition which implies that the equilibrium distribution of this amalgamation is itself an amalgamation of the equilibrium distributions of the underlying stochastic processes. A well-known result in the theory of stochastic processes is the following. Consider a process that can start in K different configurations and starts in configuration k with probability r(k). Then, without any constraints on the transition rates of the process, the equilibrium distribution π is given by K π = X r(k)π(k), (1.1) k=1 where π(k) is the equilibrium distribution for the process that starts in configuration k. This result is valid only if the process selects a configura-

159 160 Amalgamation of Markov chains tion at the start and always remains in the selected configuration. If the process can, independent of the previous or successive transitions, select upon each transition from a set of configurations via which the transition will be made, i.e. if the transition rates q are a mixture of the transition rates of the configurations K q = X r(k)q(k), (1.2) k=1 where q(k) is the transition rate for configuration k, then the equilibrium distribution will, in general, not be of the form (1.1) with π(k) the equilibrium distribution for configuration k. This chapter gives a sufficient condition on the transition rates q(k) for the amalgamated process, that is the process with transition rates (1.2), to have an equilibrium distribution (1.1). This sufficient condition is cross- balance, a generalization of global balance to a set of processes. It relates the transition rates q(k) for process k in the set to the transition rates q(k0) for process k0 in the set.

2 Model and cross-balance

Consider a set of K continuous-time queueing networks, labelled k = 1,...,K, consisting of N stations. Assume that each queueing network in the set can be represented by a stable, regular, continuous-time Markov N chain at state space IN0 . A state of the queueing network is a vector n¯ = (n1, . . . , nN ), where ni denotes the number of customers at station i, i = 1,...,N. The transition rate from staten ¯ to staten ¯0 for network k is denoted by q(k)(¯n, n¯0), k = 1,...,K. For a set of queueing networks with transition rates q(k), k = 1,...,K, define the following process. Definition 2.1 (Amalgamated process) Consider a set of queueing networks with transition rates q(k), k = 1,...,K. The amalgamated process with amalgamation coefficients r(k) ∈ IR, k = 1,...,K, such that for 0 N 0 all n,¯ n¯ ∈ IN0 , n¯ 6=n ¯ , K X r(k)q(k)(¯n, n¯0) ≥ 0, (2.1) k=1 N is the process at state space IN0 with transition rates q defined as K 0 X (k) (k) 0 0 N q(¯n, n¯ ) = r q (¯n, n¯ ), n,¯ n¯ ∈ IN0 . (2.2) k=1 7.2 Model and cross-balance 161

The amalgamation coefficients r(k) are not assumed to be non-negative. Therefore, Condition (2.1) is necessary for the transition rates q to be properly defined, that is for q to satisfy (2.1.3). If r(k) ≥ 0 for all k then (2.1) is trivially satisfied. (k) N Assume that for a set V ⊆ IN0 the process with transition rates q(k) is irreducible. Then the irreducible set V of the amalgamated process cannot be immediately obtained from the irreducible sets V (k) of the processes in the set. For example, consider a set of two queueing net- (1) (2) (1) (2) (2) works such that V ∩ V 6= ∅ and V 6= V . Letn ¯0 ∈ V and (1) define a sequence of statesn ¯0, n¯1, ··· , n¯j−1, n¯j such thatn ¯j ∈ V and (1) (2) (1) n¯i 6∈ V ∪ V , i = 1, . . . , j − 1. If q (¯ni, n¯i+1) > 0, i = 0, . . . , j − 1, (2) and q (¯nj, n¯0) > 0. Then the irreducible set of the amalgamated process contains the statesn ¯1,..., n¯j−1 which are not elements of the irreducible sets of the processes in the set. This implies that, at least in SK (k) SK (k) some cases, V ⊃ k=1 V . Also, the case where V ⊂ k=1 V is possible. For example, consider the following set of three queueing networks. Assume that q(1) = q(2), V (1) = V (2) ⊃ V (3). Then the amalgamated process with amalgamation coefficients r(1) = 1, r(2) = −1, r(3) = 1 satisfies (2.1) and is given by q(¯n, n¯0) = q(3)(¯n, n¯0) with irreducible set (3) (1) S3 (k) V = V ⊂ V = k=1 V . If q(k)(¯n, n¯0) = 0 ifn ¯ orn ¯0 ∈/ V (k), then the irreducible set of the amalgamated process is determined by the irreducible sets of the processes in the set. However, this seems to be an unnecessary assumption. In the sequel, the problem of determining V is reconsidered when the notion of cross-balance is introduced. It will be shown that the irreducible set V is a subset of the union of the V (k) if the set satisfies cross-balance (see Lemma 2.4). The remaining part of this section relates the equilibrium distribution π of the amalgamated process to the equilibrium distributions π(k) of the processes in the set. In order to avoid problems with the normalizing constant when deriving this relation invariant measures rather than equilibrium distributions are considered. The following lemma gives a sufficient condition for the amalgamated process to have an invariant measure m that is a sum of the invariant measures for the networks in the set if the processes in the set posses an invariant measure, where it is assumed that the process with transition rates q(k) has an invariant measure m(k). This lemma is a precursor to the main result and illustrates the relation between the processes in the set. Lemma 2.2 Consider a set of K queueing networks with transition rates 162 Amalgamation of Markov chains

(k) (k) N (k) q , irreducible sets V ⊆ IN0 , and invariant measures m , k = 1,...,K. Then the amalgamated process with amalgamation coeﬃcients r(k), k = 1,...,K, such that

K X (k) (k) N r m (¯n) ≥ 0, n¯ ∈ IN0 , k=1 has an invariant measure m given by

K X (k) (k) N m(¯n) = r m (¯n), n¯ ∈ IN0 , (2.3) k=1

0 0 N if for all k, k , k, k = 1,...,K, the following relation holds for all n¯ ∈ IN0

n 0 0 o X m(k)(¯n)q(k )(¯n, n¯0) + m(k )(¯n)q(k)(¯n, n¯0) n¯06=¯n n 0 0 o = X m(k)(¯n0)q(k )(¯n0, n¯) + m(k )(¯n0)q(k)(¯n0, n¯) . (2.4) n¯06=¯n

Proof It is sufficient to prove that m defined in (2.3) satisfies the global balance equations for the amalgamated process

X {m(¯n)q(¯n, n¯0) − m(¯n0)q(¯n0, n¯)} = 0. (2.5) n¯06=¯n

Substitution of (2.2) and (2.3) into the global balance equations gives

X {m(¯n)q(¯n, n¯0) − m(¯n0)q(¯n0, n¯)} n¯06=¯n K K 0 n 0 0 o = X X X r(k)r(k ) m(k)(¯n)q(k )(¯n, n¯0) − m(k)(¯n0)q(k )(¯n0, n¯) n¯06=¯n k=1 k0=1 K K  1 X X (k) (k0) X n (k) (k0) 0 (k0) (k) 0 o = r r  m (¯n)q (¯n, n¯ ) + m (¯n)q (¯n, n¯ ) 2 k=1 k0=1 n¯06=¯n  X n (k) 0 (k0) 0 (k0) 0 (k) 0 o − m (¯n )q (¯n , n¯) + m (¯n )q (¯n , n¯)  n¯06=¯n = 0, where the second equality is obtained by changing the order of summation and the last equality from (2.4). 2 7.2 Model and cross-balance 163

Condition (2.4) is a technical relation. In the obvious matrix notation (2.4) reads

m(k)Q(k0) + m(k0)Q(k) = 0, k, k0 = 1,...,K. (2.6)

Since m(l) is an invariant measure for process l: m(l)Q(l) = 0, l = 1,...,K, insertion into (2.6) gives

0 0 m(k) + m(k ) Q(k) + Q(k ) = 0, k, k0 = 1,...,K.

Thus (2.4) expresses that m(k) + m(k0) is an invariant measure for the process with transition rates Q(k) + Q(k0), k, k0 = 1,...,K. For K = 2 Lemma 2.2 gives a necessary and suﬃcient condition for the existence of an invariant measure of the form (2.3) for the amalgamated process. This can easily be seen by substitution of (2.2) and (2.3) into the global equations (2.5) mQ = r(1)m(1) + r(2)m(2) r(1)Q(1) + r(2)Q(2) = r(1)r(2) m(1)Q(2) + m(2)Q(1) .

Thus for K = 2, mQ = 0 if and only if (2.4) is satisﬁed. As is discussed above, (2.4) gives a general condition for the existence of an invariant measure m for the amalgamated process. However, (2.4) is a rather complicated condition to verify. Therefore, the following deﬁni- tion gives a more practical form of balance, so-called cross-balance, which implies (2.4), and is shown to be relatively easy to verify.

Deﬁnition 2.3 (Cross-balance) Consider a set of queueing networks with transition rates q(k), k = 1,...,K. If there exists a collection of (k) (k) N measures m = {m (¯n), n¯ ∈ IN0 }, k = 1,...,K, such that for all 0 0 N k, k , k, k = 1,...,K, and for all n¯ ∈ IN0

n 0 0 o X m(k)(¯n)q(k )(¯n, n¯0) − m(k )(¯n0)q(k)(¯n0, n¯) = 0, (2.7) n¯06=¯n then the set of queueing networks satisﬁes cross-balance with measures m(k), k = 1,...,K.

Note that in the deﬁnition above it is not assumed that the measures m(k) are invariant measures for the processes in the set. However, since (2.7) must hold for all k, k0, for k = k0 this implies that m(k) is an invariant 164 Amalgamation of Markov chains measure for process k. Therefore, cross-balance is a generalization of global balance to sets of processes, and a set of processes can satisfy cross- balance only if each process possesses an invariant measure. Based on the uniqueness of the equilibrium distributions for the processes in the set, the following lemma reduces the irreducible set of the amalgamated process to the union of the irreducible sets of the processes (k) N in the set. Therefore, in the sequel it is assumed that for a set V ⊆ IN0 the process with transition rates q(k) is irreducible and that there exists a unique equilibrium distribution π(k) at V (k), and π(k)(¯n) = 0 if n¯ 6∈ V (k), k = 1,...,K.

Lemma 2.4 Consider a set of queueing networks with transition rates q(k), irreducible sets V (k) and unique equilibrium distributions π(k) at V (k). Then, if there exists a set of constants c(k) > 0, k = 1,...,K, such that the set of processes satisﬁes cross-balance with measures c(k)π(k), k = 1,...,K, then the amalgamated process cannot have transitions out of the set V = SK (k) 0 k=1 V , i.e. for all k it must be the case that if n¯ ∈ V and n¯ 6∈ V then q(k)(¯n, n¯0) = 0.

(k0) (k0) Proof Letn ¯0 ∈ V , sayn ¯0 ∈ V , andn ¯1 6∈ V . Then π (¯n0) > 0 (k) and π (¯n1) = 0 for all k, k = 1,...,K. Assume that q(¯n0, n¯1) > 0, (k1) then for some k, say k1, it must be that q (¯n0, n¯1) > 0. Now consider cross-balance for k0 and k1 atn ¯1:

X n (k1) (k1) (k0) 0 (k0) (k0) 0 (k1) 0 o c π (¯n1)q (¯n1, n¯ ) − c π (¯n )q (¯n , n¯1) 0 n¯ 6=¯n1

X (k0) (k0) 0 (k1) 0 = − c π (¯n )q (¯n , n¯1) 0 n¯ 6=¯n1

(k0) (k0) (k1) ≤ −c π (¯n0)q (¯n0, n¯1) < 0, which is in contradiction to the assumption that the set satisﬁes cross- balance. 2

This chapter considers sets of networks satisfying cross-balance only. From Lemma 2.4, without loss of generality, it may now assume that the initial distribution of the amalgamated process is such that with probability 1 the SK (k) process starts at V = k=1 V . The following theorem states the main result. In this theorem the equilibrium distribution of the amalgamated process is related to the equilibrium distributions of the processes in the set. 7.2 Model and cross-balance 165

Theorem 2.5 (Main result) Consider a set of queueing networks with transition rates q(k), irreducible sets V (k), and unique equilibrium distributions π(k) at V (k), k = 1,...,K. If there exists a set of constants c(k) > 0, k = 1,...,K, such that the set satisfies cross-balance with measures c(k)π(k), k = 1,...,K, then the amalgamated process with amalgamation coefficients r(k) such that K X r(k)c(k) = C, C > 0, (2.8) k=1 SK (k) has an equilibrium distribution π at V = k=1 V given by 1 K π(¯n) = X r(k)c(k)π(k)(¯n), n¯ ∈ V. (2.9) C k=1 Proof By Lemma 2.4, the amalgamated process cannot leave the set V as defined in the theorem. It is sufficient to prove that π defined in (2.9) is a probability distribution at V and satisfies the global balance equations (2.5). Assume that π(¯n0) < 0 for somen ¯0 ∈ V . Then (2.1) and cross-balance imply K X (k) X n (k) (k) (k0) 0 (k0) (k0) 0 (k) 0 o 0 = r c π (¯n0)q (¯n0, n¯ ) − c π (¯n )q (¯n , n¯0) 0 k=1 n¯ 6=¯n0 ( K K ) X X (k) (k) (k) (k0) 0 (k0) (k0) 0 X (k) (k) 0 = r c π (¯n0)q (¯n0, n¯ ) − c π (¯n ) r q (¯n , n¯0) 0 n¯ 6=¯n0 k=1 k=1 X n (k0) 0 (k0) (k0) 0 0 o = Cπ(¯n0)q (¯n0, n¯ ) − c π (¯n )q(¯n , n¯0) < 0. 0 n¯ 6=¯n0 Thus, π(¯n) ≥ 0 for alln ¯ ∈ V . Summation of π yields 1 K 1 K X π(¯n) = X X r(k)c(k)π(k)(¯n) = X r(k)c(k) X π(k)(¯n) = 1, n¯∈V n¯∈V C k=1 C k=1 n¯∈V which implies 0 ≤ π(¯n) ≤ 1 for alln ¯ ∈ V and π(V ) = 1. Now consider N (k) a sequence of mutually exclusive events Ei ⊆ IN0 then, since π is a probability distribution for all k ∞ K ∞ K ∞ [ 1 X (k) (k) (k) [ 1 X (k) (k) X (k) π( Ei) = r c π ( Ei) = r c π (Ei) i=1 C k=1 i=1 C k=1 i=1 ∞ K ∞ X 1 X (k) (k) (k) X = r c π (Ei) = π(Ei). i=1 C k=1 i=1 166 Amalgamation of Markov chains

Note that (2.7) implies (2.4) and apply Lemma 2.2 with m(k) = c(k)π(k). 2

At first glance, the result of Theorem 2.5 seems to be an obvious result. To this end, consider a queueing network that can start in K different configurations. If the network starts with probability r(k) in configuration k with transition rates q(k) and equilibrium distribution π(k), then the equilibrium distribution of this network is given by

K π = X r(k)π(k). (2.10) k=1 This is exactly the form obtained by inserting c(k) ≡ 1, r(k) > 0 such that P (k) k r = 1 into Theorem 2.5. This form is obvious when once and for all a process with corresponding transition rates is selected in advance. In contrast, the amalgamated process presented here allows to select from a set of transition rates q(k) at any transition. With probability r(k) it selects transition rate q(k) for a particular transition, independent of the previous or successive transitions. In this case the above form (2.10) is no longer obvious. For the process to have this form for the equilibrium distribution there will be some restrictions on the transition rates q(k). These restrictions are given by cross-balance. The coefficients c(k) introduced in the main result are not essential for the theory, for example with c(k) ≡ 1 Theorem 2.5 remains valid. In the applications, however, these coefficients play a very important role. In many cases a set of queueing networks satisfies cross-balance for a special choice of the c(k) only (cf. Examples 3.2, 3.3). In some applications the coefficients c(k) will replace the normalizing constant and will be chosen such that at the union of the irreducible sets of the processes in the set the invariant measures for the processes are the same (cf. Examples 3.6, 3.7). In the main result above the amalgamation coefficients are not necessarily positive, for example see Example 3.6. Note that the amalgamation coefficients may be chosen such that C = 1. This can, without loss of generality, be obtained by replacing r(k) := r(k)/C. Note, however, that (2.9) does not express a mixture of the distributions π(k). This would be the case if r(k) > 0 for all k, which in the general setting is not necessary. Although the initial condition of the amalgamated process is such that SK (k) with probability 1 the process starts at k=1 V , the amalgamated process is not necessarily irreducible, and thus, the equilibrium distribution of the amalgamated process is not necessarily unique. In general, conditions on the processes in the set and the amalgamation coefficients which 7.3 Examples 167 guarantee that the equilibrium distribution of the amalgamated process is unique are hard to give. These conditions will depend on the specific form of the transition rates (cf. Examples 3.1 and 3.2). However, in some cases general conditions are possible. For example, in each of the following two cases it can easily be verified that the equilibrium distribution of the amalgamated process is unique.

1. If r(k) > 0 for all k and the irreducible sets are such that

V (i) ∩ V (i+1) 6= ∅, i = 1,...,K − 1.

2. If the irreducible sets are such that

V (i) ∩ V (i+1) 6= ∅, i = 1,...,K − 1, n o V (i) \ V (i−1) ∪ V (i+1) 6= ∅, i = 2,...,K − 1. 2

3 Examples

This section gives some examples of sets of processes that satisfy cross- balance. The aim of this section is to illustrate some applications, such as the construction method for the equilibrium distribution in Example 3.6, and to give some examples of the implications of cross-balance on the transition rates of the processes in the set. These examples show that the notion of cross-balance unifies various known special situations and leads to possible new examples. First, Example 3.1 considers a standard simple example that can directly be incorporated in the theory. This example combined with Example 3.2 shows that the uniqueness of the equilibrium distribution of the amalgamated process depends on the specific form of the transition rates of the processes in the set. In particular, it depends on the transition rates between the irreducible sets of the processes in the set. Examples 3.3 and 3.4 consider some well-known processes from the literature. Example 3.3 considers the truncated process, and Example 3.4 shows that a process and its time-reversed process satisfy cross-balance. Example 3.5 gives a novel example. In this example two processes are combined into one amalgamated process. The implications on the transition rates of the two processes are worked out in detail as to illustrate the implications of cross-balance. In Example 3.6 the approach is different. Starting with a process with given transition rates and irreducible set, a 168 Amalgamation of Markov chains set of processes is constructed such that the amalgamated process has the same transition rates and irreducible set as the original process. This set of processes is used to derive the equilibrium distribution of the original process. Finally, Example 3.7 considers product-form queueing networks and shows that a queueing network has a product-form equilibrium distribution if the irreducible set is a union of “triangles”.

3.1 Disjoint irreducible sets; reducible process Consider a set of K queueing networks consisting of N stations with transition rates q(k), irreducible set V (k), and unique equilibrium distribution π(k) at V (k), k = 1,...,K. Assume that V (k) ∩ V (k0) = ∅ for all k 6= k0, SK (k) (k) and deﬁne V = k=1 V . If the transition rates q , k = 1,...,K, are such that

q(k)(¯n, n¯0) = 0 ifn ¯0 ∈/ V (k) orn ¯0 ∈ V (k) andn ¯ ∈ V \ V (k), (3.1) then the set of queueing networks trivially satisﬁes cross-balance with measures π(k), k = 1,...,K. The amalgamated process cannot make any transitions between the irreducible sets V (k). Therefore, the amalgamated process is reducible and has an equilibrium distribution

K π(¯n) = X r(k)π(k)(¯n), n¯ ∈ V, (3.2) k=1

(k) P (k) for arbitrary coefficients r > 0, k r = 1. The following example is of interest. Consider a closed queueing network consisting of N stations, labelled j = 1,...,N. Let V (k) = {n¯ :n ¯ = PN (k) (n1, . . . , nN ), j=1 nj = k}, k = 1,...,K, q the transition rates of network k and π(k) the corresponding equilibrium distribution. Then process k is a closed queueing network that started off with k customers. This set satisfies cross-balance with measures π(k). The equilibrium distribution of the amalgamated process is given by (3.2), where r(k) represents the probability that the network starts with k customers. An example similar to this example is given in [83, p. 6].

3.2 Disjoint irreducible sets; irreducible process The essential assumption in the example above is not that the irreducible sets of the processes in the set are disjoint, but that the amalgamated 7.3 Examples 169 process cannot make any transitions between the irreducible sets of the processes in the set. For example, consider a set of 2 queueing networks with transition rates q(1), q(2), irreducible set V (1),V (2), and unique equilibrium distribution π(1) at V (1), π(2) at V (2). Assume that V (1) ∩V (2) = ∅. (1) (2) Let the transition rates be as in (3.1) but add for ﬁxedn ¯1 ∈ V , n¯2 ∈ V

(1) (2) q (¯n2, n¯1) = α1 > 0, q (¯n1, n¯2) = α2 > 0. Then the set satisﬁes cross-balance with measures

(1) α1 (1) (2) α2 (2) m = (1) π , m = (2) π . π (¯n1) π (¯n2) The amalgamated process with amalgamation coeﬃcients r(1), r(2) > 0 has a unique equilibrium distribution π at V = V (1) ∪ V (2) given by

(1) (2) ! 1 r α1 (1) r α2 (2) π(¯n) = (1) (2) (1) π (¯n) + (2) π (¯n) . r α1 r α2 π (¯n1) π (¯n2) (1) + (2) π (¯n1) π (¯n2)

3.3 State space truncation Consider a queueing network with transition rates q(1), irreducible set V (1), and unique equilibrium distribution π(1) at V (1). Assume that there exists a set V (2) ⊂ V (1) such that for each state in V (2) separately, the rate out of V (2) is balanced with the rate into V (2), i.e. for alln ¯ ∈ V (2) n o X π(1)(¯n)q(1)(¯n, n¯0) − π(1)(¯n0)q(1)(¯n0, n¯) = 0. n¯0∈V (1)\V (2) Then the queueing network can be restricted to V (2). This process is called the truncated process and has transition rates q(2) deﬁned by ( q(1)(¯n, n¯0), ifn ¯0 ∈ V (2), q(2)(¯n, n¯0) = 0, otherwise,

The truncated process has an equilibrium distribution π(2) at V (2) given by (1) (2) π (¯n) π (¯n) = P (1) . n¯∈V (2) π (¯n) The set q(1), q(2) satisﬁes cross-balance with measures m(1) = π(1), m(2) = π(2) X π(1)(¯n). n¯∈V (2) 170 Amalgamation of Markov chains

3.4 Time-reversal Consider a queueing network with transition rates q(1), irreducible set V , and unique equilibrium distribution π(1). In reversed time, the queueing network has the same equilibrium distribution π(2) = π(1) at irreducible set V . The transition rates q(2) of the time-reversed process are deﬁned as the set of numbers that satisﬁes (cf. [55])

π(1)(¯n)q(1)(¯n, n¯0) = π(1)(¯n0)q(2)(¯n0, n¯). (3.3)

Summation of (3.3) immediately implies that the set consisting of a process and its time-reversed process satisﬁes cross-balance with measures m(1) = π(1), m(2) = π(2) and the amalgamated process has equilibrium distribution π = π(1). An intuitive interpretation of the amalgamated process with amalgamation coeﬃcients r(1), r(2) is the following. For process q(1) the time passes by at rate 1, therefore for the process with transition rates r(1)q(1) time passes at rate r(1). For the time-reversed process with rates q(2) time passes at rate −1, therefore for the process with rates r(2)q(2) time passes at rate −r(2). For the amalgamated process time passes at rate r(1) − r(2), but since the process is stationary the speed at which time passes does not play a role in determining the equilibrium distribution. Therefore, the process for which time passes at rate r(1) − r(2) has the same equilibrium distribution as the process for which time passes at rate 1.

3.5 Nearly disjoint irreducible sets This example considers a set of 2 processes. The transition rates of these processes will be modified such that the set satisfies cross-balance under the restriction that the equilibrium distributions of the processes in the set remain unchanged. To this end, note that for a process with transition rates q(k), irreducible set V (k), and equilibrium distribution π(k) at V (k), the transition rates q(k)(¯n, n¯0) forn ¯ 6∈ V (k) can be arbitrarily changed without affecting the equilibrium distribution π(k). Consider a set of 2 processes derived from a queueing network, that is, the transitions allowed for the processes are those allowed in a queueing network only, i.e. a customer is allowed to enter the system at station i corresponding to a transition from staten ¯ to staten ¯ + ei, a customer is allowed to leave station i and route to station j corresponding to a transition from staten ¯ to staten ¯ − ei + ej, and a customer is allowed to 7.3 Examples 171 leave the system from station i corresponding to a transition from staten ¯ to staten ¯ − ei. The rates at which customers enter or leave the stations (k) 0 is given by φ for process k, k = 1, 2. Forn ¯ =n ¯ + ei,n ¯ − ei,n ¯ − ei + ej, the transition rates q(k) for the processes are then given by

(1) 0 (1) 0 0 (1) q (¯n, n¯ ) = φ (¯n, n¯ )1{0 ≤ ni ≤ Ji }, (3.4) (2) 0 (2) 0 (1) 0 (2) q (¯n, n¯ ) = φ (¯n, n¯ )1{Ji ≤ ni ≤ Ji }. Then the irreducible sets of the processes in the set are given by

(1) n (1) o V = n¯ : 0 ≤ ni ≤ Ji , i = 1,...,N ,

(2) n (1) (2) o V = n¯ : Ji ≤ ni ≤ Ji , i = 1,...,N . Process 1 corresponds to a queueing network in which the number of (1) customers at station i is constrained not to exceed Ji and process 2 is a (1) queueing network in which the number of customers cannot fall below Ji (1) (1) at station i. The two irreducible sets are connected atn ¯0 = (J1 ,...,JN ) only. Thus irreducibility of the amalgamated process is guaranteed. Assume that equilibrium distributions π(1) and π(2) exist, i.e. π(1), π(2) satisfy n o X π(k)(¯n)φ(k)(¯n, n¯0) − π(k)(¯n0)φ(k)(¯n0, n¯) = 0, k = 1, 2. (3.5) n¯06=¯n, n¯0∈V (k)

Then, for some arbitrary coefficients c(1), c(2), the set satisfies cross- balance with measures m(k) = c(k)π(k) if and only if the transition rates q(k)(¯n, n¯0) forn ¯ 6∈ V (k) are defined as, for i = 1,...,N,

(1) (1) (1) c π (¯n0) (2) q (¯n0 + ei, n¯0) = (2) (2) q (¯n0, n¯0 + ei), (3.6a) c π (¯n0 + ei) q(1)(¯n, n¯0) = 0, ifn ¯0 ∈ V (2) \ V (1), (3.6b) (2) (2) (2) c π (¯n0) (1) q (¯n0 − ei, n¯0) = (1) (1) q (¯n0, n¯0 − ei), (3.6c) c π (¯n0 − ei) q(2)(¯n, n¯0) = 0, ifn ¯0 ∈ V (1) \ V (2). (3.6d)

Note that (3.6a) and (3.6c) are well-deﬁned since the equilibrium distributions π(k) are assumed to be known. (3.6a), (3.6b) determine the transition rates of process 1 at V (2), and (3.6c), (3.6d) determine the transition rates of process 2 at V (1). 172 Amalgamation of Markov chains

To illustrate how the framework of cross-balance applies consider the proof of the statement above: the set of processes with transition rates defined in (3.4) satisfies cross-balance with measures m(k) = c(k)π(k), k = 1, 2, if and only if the transition rates are modified as given in (3.6a) – (3.6d). To this end, first note that cross-balance must be proven for k 6= k0 only, since for k = k0 (2.7) is already given by (3.5). If k = 1, k0 = 2 and n¯ ∈ V (2) \ V (1) then m(1)(¯n) = 0 and (2.7) reduces to

X m(2)(¯n0)q(1)(¯n0, n¯) = 0. n¯06=¯n

Since m(2)(¯n0) > 0 for alln ¯0 ∈ V (2) this relation can hold if and only if (3.6b) holds. For k = 2, k0 = 1, by a similar argument, cross-balance holds forn ¯ ∈ V (1) \ V (2) if and only if (3.6d) holds. If k = 1, k0 = 2 and n¯ ∈ V (1) \ V (2) n o X m(1)(¯n)q(2)(¯n, n¯0) − m(2)(¯n0)q(1)(¯n0, n¯) n¯06=¯n (3.6d) n o = X m(1)(¯n)q(2)(¯n, n¯0) − m(2)(¯n0)q(1)(¯n0, n¯) n¯0∈V (2) ( m(1)(¯n)q(2)(¯n, n¯ ) − m(2)(¯n )q(1)(¯n , n¯), ifn ¯ =n ¯ − e , = 0 0 0 0 i 0, otherwise, where the last equality is obtained by observing that the process can make transitions allowed in (3.4) only. This implies that cross-balance can hold if and only if (3.6c) holds. The argument for (3.6a) can be given in a similar way. It remains to check that with the transition rates deﬁned in 0 (3.6a) – (3.6d) the set satisﬁes cross-balance forn ¯ =n ¯0. For k = 1, k = 2 andn ¯ =n ¯0 cross-balance reads

X n (1) (2) 0 (2) 0 (1) 0 o m (¯n0)q (¯n0, n¯ ) − m (¯n )q (¯n , n¯0) 0 n¯ 6=¯n0 X n (1) (2) 0 (2) 0 (1) 0 o = m (¯n0)q (¯n0, n¯ ) − m (¯n )q (¯n , n¯0) n¯0∈V (2) N X n (1) (2) (2) (1) o = m (¯n0)q (¯n0, n¯0 + ei) − m (¯n0 + ei)q (¯n0 + ei, n¯0) i=1 = 0, where the last equality is obtained from (3.6a). For k = 2, k0 = 1 and n¯ =n ¯0 by a similar argument (2.7) is obtained from (3.6c). 7.3 Examples 173

For c(1) = c(2) = 1 the amalgamated process with amalgamation co- eﬃcients r(1) = r, r(2) = 1 − r, 0 ≤ r ≤ 1 has a unique equilibrium distribution at V = V (1) ∪ V (2) given by

π(¯n) = rπ(1)(¯n) + (1 − r)π(2)(¯n), n¯ ∈ V (1) ∪ V (2).

In this example the irreducible sets V (1),V (2) intersect in exactly one point. This is crucial for the simple analysis presented above. For example, there are no restrictions on the transition rates q(k) at V (k), which, in general, will be the case. However, this example can be generalized to irreducible sets that intersect in several points. The analyses will become more complex and also there will be restrictions on the transition rates of the processes in the set. This example reﬂects some of the key-features of a set that satisﬁes cross-balance.

– Forn ¯0 6∈ V (k) cross-balance implies that q(k)(¯n, n¯0) = 0 for alln ¯.

– Relation (2.7) in the deﬁnition of cross-balance may be replaced by: for alln ¯ ∈ V (k)

n 0 0 o X m(k)(¯n)q(k )(¯n, n¯0) − m(k )(¯n0)q(k)(¯n0, n¯) = 0. n¯06=¯n, n¯0∈V (k0)

To illustrate the implications of (3.6a) and (3.6c) on the transition rates of the amalgamated process, consider the following explicit example, where each process is a network of single-server queues with Poisson arrivals and (2) state-independent routing. Let Ji = ∞, i = 1, . . . , n, and

 0  λp0i, ifn ¯ =n ¯ + ei, 0  0 φ(¯n, n¯ ) = µipij, ifn ¯ =n ¯ − ei + ej,  0  µipi0, ifn ¯ =n ¯ − ei.

Furthermore, deﬁne the transition rates for process 1 and 2 as

φ(2)(¯n, n¯0) = φ(¯n, n¯0), (1) 0 0 (1) φ (¯n, n¯ ) = φ(¯n, n¯ ), if ni < Ji , i = 1,...,N, (1) (1) (1) φ (¯n, n¯ − ei + ej) = φ(¯n, n¯ − ei + ej), if ni = Ji , nj < Jj , (1) (1) (1) (1) φ (¯n, n¯ + ej) = λp0j , if ni = Ji , nj < Jj , (1) (1) (1) (1) φ (¯n + ej, n¯) = µjpj0 , if ni = Ji , nj < Jj , 174 Amalgamation of Markov chains where p(1) will be chosen such that process 1 is reversible at the boundary. N With {γi}i=1 the solution of the traﬃc equations

N X γi = λp0i + γjpji, i = 1,...,N, j=1

(1) (1) (1) (1) and pj0 , p0j such that λp0j = γjpj0 , j = 1,...,N, both process 1 and 2 have a unique product-form equilibrium distribution 1 π(k) = m, c(k) where m is given by N γ !ni m(¯n) = Y i , i=1 µi and 1/c(k) is the normalizing constant for process k. (3.6a), (3.6c) give the following relations for the transition rates.

(1) λ q (¯n0 + ei, n¯0) = µi p0i (3.7a) γi γ q(2)(¯n − e , n¯ ) = λ i p(1). (3.7b) 0 i 0 λ i0 If the transition rates satisfy these equations then the equilibrium distribution of the amalgamated process with amalgamation coeﬃcients r(1), r(2) is given by

 (1) N !ni  r Y γi (1) (2)  , ifn ¯ ∈ V \ V ,  r(1)c(1) + r(2)c(2) µ  i=1 i   !ni  r(1) + r(2) N γ π(¯n) = Y i (1) (2) (3.8) (1) (1) (2) (2) , ifn ¯ ∈ V ∩ V ,  r c + r c i=1 µi   N !ni  r(2) γ  Y i , ifn ¯ ∈ V (2) \ V (1).  (1) (1) (2) (2)  r c + r c i=1 µi Note that, although the equilibrium distributions of the processes in the set are of product-form, the equilibrium distribution of the amalgamated process is not of product-form since the “normalizing constants” are not the same for all states. 7.3 Examples 175

The transition rates given in (3.7a), (3.7b) seem to have a strange form. However, they can be rewritten as

(1) ∗ q (¯n0 + ei, n¯0) = µipi0,

(2) (1) q (¯n0 − ei, n¯0) = λp0i , where p∗ are the transition probabilities of the time-reversed process for process 2. Thus, (3.7a) represents a departure from the network and (3.7b) represents an arrival to the network. The transition rates of the amalgamated process are given by

 r(1)φ(1)(¯n, n¯0), ifn ¯0 ∈ V (1) \ V (2),    (1) ∗ (2) 0  µi r pi0 + r pi0 , ifn ¯ =n ¯0 + ei, n¯ =n ¯0, q(¯n, n¯0) = (1) (1)  λ r(1)p + r(2)p , ifn ¯ =n ¯ − e , n¯0 =n ¯ ,  0i 0i 0 i 0   r(2)φ(2)(¯n, n¯0), ifn ¯0 ∈ V (2) \ V (1).

The probability of leaving the system from staten ¯0 +ei is changed and also the probability of entering the system to staten ¯0 is changed. The form of the transition rates of the amalgamated process, however, is exactly the same as the form of the transition rates of the processes in the set.

3.6 Construction method In the previous example, the irreducible sets V (1) and V (2) intersect in exactly one point. In that case, the transition rates q(1) at V (2) and q(2) at V (1) could be constructed such that the set satisﬁes cross-balance. In the case of identical invariant measures for both processes in the set one would expect that the amalgamated process allows the same invariant measure. However, as can be seen from (3.8) for the special case of product-form invariant measures, this is not true. This example extends the previous example to state spaces that intersect in several points. Also, in the case of identical invariant measures for the processes in the set, it will be shown that it is possible to construct an amalgamated process such that the amalgamated process allows the same invariant measure. Moreover, this example shows that for a given process, a set of processes can be constructed such that the amalgamated process has the same transition rates as the original process. This implies that the equilibrium distribution for the original process is given by the equilibrium distribution of the 176 Amalgamation of Markov chains amalgamated process. Thus, this section gives a construction method for constructing the equilibrium distribution for a process via the equilibrium distributions for the processes in the set. For simplicity, the discussion is restricted to a system with two stations but can be generalized to queueing networks with N stations (N ≥ 1). Consider a two station queueing network with transition rates

 0 0  φ(¯n, n¯ ), ifn, ¯ n¯ ∈ V, 0  0 0 0 q(¯n, n¯ ) = andn ¯ =n ¯ + ei, n¯ =n ¯ − ei, n¯ =n ¯ − ei + ej,  0, otherwise, (3.9) where V is given by

0 0 V = {n¯ : 0 ≤ ni ≤ Ji} ∪ {Ji − 1 ≤ ni} . A set of queueing networks that satisfies cross-balance such that the amalgamated process has transition rates q as defined above will be constructed. From this construction, the equilibrium distribution for the amalgamated process and thus for the process with transition rates q is obtained explicitly. First, consider the following set of 2 queueing networks with transition rates q(k) of network k defined by

(1) 0 0 0 q (¯n, n¯ ) = q(¯n, n¯ )1{0 ≤ ni, ni ≤ Ji, i = 1, 2},

(2) 0 0 0 q (¯n, n¯ ) = q(¯n, n¯ )1{Ji − 1 ≤ ni, ni, i = 1, 2}. Then the irreducible sets of the networks in the set are given by

(1) (2) V = {n¯ : 0 ≤ ni ≤ Ji} ,V = {n¯ : Ji − 1 ≤ ni} .

Assume that there exist invariant measures for these processes, i.e. some sets of non-negative numbers m(1), m(2) that satisfy n o X m(k)(¯n)q(k)(¯n, n¯0) − m(k)(¯n0)q(k)(¯n0, n¯) = 0, k = 1, 2. n¯06=¯n, n¯0∈V (k)

As can be seen from Figure 7.1, the state spaces V (1) and V (2) intersect in exactly four points. In order to guarantee that the set satisﬁes cross- balance it must be assumed that q(1), q(2) satisfy the following relations at the intersection of the irreducible sets. In these relations the states are labelled as depicted in Figure 7.1, where, for examplen ¯3 = (J1 − 1,J2), 7.3 Examples 177

V (2) n¯0 n¯1

r r n¯2 n¯3 n¯4 n¯5

r r r r n¯6 n¯7 n¯8 n¯9

r r r r n¯10 n¯11 V (1) r r

Figure 7.1. State spaces and labelling of states

n¯4 = (J1,J2),n ¯7 = (J1 − 1,J2 − 1),n ¯8 = (J1,J2 − 1). The ﬁrst relation (3.10a) represents cross-balance for staten ¯4, the second relation (3.10b) represents cross-balance for staten ¯7 and the third relation (3.10c) expresses that the total ﬂow in the box consisting ofn ¯3, n¯4, n¯7, n¯8 is balanced. Additional relations for statesn ¯3 andn ¯8 should be imposed, but these relations follow by combination of (3.10a), (3.10b), (3.10c).

(2) n (1) (1) o m (¯n4) q (¯n4, n¯3) + q (¯n4, n¯8) (1) (2) (1) (2) = m (¯n3)q (¯n3, n¯4) + m (¯n8)q (¯n8, n¯4) (3.10a)

(1) n (2) (2) o m (¯n7) q (¯n7, n¯3) + q (¯n7, n¯8) (2) (1) (2) (1) = m (¯n3)q (¯n3, n¯7) + m (¯n8)q (¯n8, n¯7) (3.10b)

(1) n (2) (2) (2) o m (¯n3) q (¯n3, n¯4) + q (¯n3, n¯7) + q (¯n3, n¯8) (1) n (2) (2) o +m (¯n4) q (¯n4, n¯3) + q (¯n4, n¯8) (1) n (2) (2) o +m (¯n7) q (¯n7, n¯3) + q (¯n7, n¯8) (1) n (2) (2) (2) o +m (¯n8) q (¯n8, n¯3) + q (¯n8, n¯4) + q (¯n8, n¯7) (2) n (1) (1) (1) o = m (¯n3) q (¯n3, n¯4) + q (¯n3, n¯7) + q (¯n3, n¯8) (3.10c) (2) n (1) (1) o +m (¯n4) q (¯n4, n¯3) + q (¯n4, n¯8) (2) n (1) (1) o +m (¯n7) q (¯n7, n¯3) + q (¯n7, n¯8) 178 Amalgamation of Markov chains

(2) n (1) (1) (1) o +m (¯n8) q (¯n8, n¯3) + q (¯n8, n¯4) + q (¯n8, n¯7)

Furthermore, as in Example 3.5, the transition rates q(1) at V (2) and q(2) at V (1) are defined such that the set satisfies cross-balance. Note that this does not affect the invariant measures m(k). To this end, by analogy with (3.6a) and (3.6c), define the following transition rates.

m(1)(¯n ) q(2)(¯n , n¯ ) + q(2)(¯n , n¯ ) + q(2)(¯n , n¯ ) + q(2)(¯n , n¯ ) q(1)(¯n , n¯ ) = 3 3 0 3 4 3 7 3 8 0 3 (2) m (¯n0) m(2)(¯n )q(1)(¯n , n¯ ) + m(2)(¯n )q(1)(¯n , n¯ ) + m(2)(¯n )q(1)(¯n , n¯ ) − 4 4 3 7 7 3 8 8 3 (2) m (¯n0) m(1)(¯n )q(2)(¯n , n¯ ) + m(1)(¯n )q(2)(¯n , n¯ ) q(1)(¯n , n¯ ) = 3 3 0 4 4 0 − q(1)(¯n , n¯ ) 0 4 (2) 0 3 m (¯n0) m(1)(¯n )q(2)(¯n , n¯ ) q(1)(¯n , n¯ ) = 4 4 1 1 4 (2) m (¯n1) m(1)(¯n )q(2)(¯n , n¯ ) q(1)(¯n , n¯ ) = 4 4 5 5 4 (2) m (¯n5) m(1)(¯n ) q(2)(¯n , n¯ ) + q(2)(¯n , n¯ ) + q(2)(¯n , n¯ ) + q(2)(¯n , n¯ ) q(1)(¯n , n¯ ) = 8 8 3 8 4 8 7 8 9 9 8 (2) m (¯n9) m(2)(¯n )q(1)(¯n , n¯ ) + m(2)(¯n )q(1)(¯n , n¯ ) + m(2)(¯n )q(1)(¯n , n¯ ) − 3 3 8 4 4 8 7 7 8 (2) m (¯n9) m(1)(¯n )q(2)(¯n , n¯ ) + m(1)(¯n )q(2)(¯n , n¯ ) q(1)(¯n , n¯ ) = 4 4 9 8 8 9 − q(1)(¯n , n¯ ) 9 4 (2) 9 8 m (¯n9) m(2)(¯n ) q(1)(¯n , n¯ ) + q(1)(¯n , n¯ ) + q(1)(¯n , n¯ ) + q(1)(¯n , n¯ ) q(2)(¯n , n¯ ) = 3 3 2 3 4 3 7 3 8 2 3 (1) m (¯n2) m(1)(¯n )q(2)(¯n , n¯ ) + m(1)(¯n )q(2)(¯n , n¯ ) + m(1)(¯n )q(2)(¯n , n¯ ) − 4 4 3 7 7 3 8 8 3 (1) m (¯n2) m(2)(¯n )q(1)(¯n , n¯ ) + m(2)(¯n )q(1)(¯n , n¯ ) q(2)(¯n , n¯ ) = 3 3 2 7 7 2 − q(2)(¯n , n¯ ) 2 7 (1) 2 3 m (¯n2) m(2)(¯n )q(1)(¯n , n¯ ) q(2)(¯n , n¯ ) = 7 7 6 6 7 (1) m (¯n6) m(2)(¯n )q(1)(¯n , n¯ ) q(2)(¯n , n¯ ) = 7 7 10 10 7 (1) m (¯n10) m(2)(¯n ) q(1)(¯n , n¯ ) + q(1)(¯n , n¯ ) + q(1)(¯n , n¯ ) + q(1)(¯n , n¯ ) q(2)(¯n , n¯ ) = 8 8 3 8 4 8 7 8 11 11 8 (1) m (¯n11) m(1)(¯n )q(2)(¯n , n¯ ) + m(1)(¯n )q(2)(¯n , n¯ ) + m(1)(¯n )q(2)(¯n , n¯ ) − 3 3 8 4 4 8 7 7 8 (1) m (¯n11) m(2)(¯n )q(1)(¯n , n¯ ) + m(2)(¯n )q(1)(¯n , n¯ ) q(2)(¯n , n¯ ) = 7 7 11 8 8 11 − q(2)(¯n , n¯ ) 11 7 (1) 11 8 m (¯n11) 7.3 Examples 179

The set of processes with transition rates q(1), q(2) as specified above satisfies cross-balance, and the amalgamated process with amalgamation coefficients r(1), r(2) has a unique equilibrium distribution π at V given by π = B(r(1)m(1) + r(2)m(2)), where B is a normalizing constant. The transition rates of the amalgamated process, however, are not identical to the transition rates of the original process (3.9). Therefore, another network is added to the set to correct for this difference. To this end, define the process with transition rates q(3) given by  (1) 0 (2) (1)  q (¯n, n¯ ), ifn ¯ ∈ V \ V ,   (2) 0 (1) (2) (3) 0  q (¯n, n¯ ), ifn ¯ ∈ V \ V , q (¯n, n¯ ) = 0 0 (1) (2)  q(¯n, n¯ ), ifn, ¯ n¯ ∈ V ∩ V ,   0, otherwise. The amalgamated process with amalgamation coefficients r(1) = 1, r(2) = 1, r(3) = −1, has transition rates q as given in (3.9). Thus a set of queueing networks such that the transition rates of the amalgamated process equal the transition rates of the original process is now constructed. The equilibrium distribution for the amalgamated process can be obtained from the equilibrium distributions of these three processes. To this end, assume that process 3 allows an invariant measure m(3) at V (3) = V (1) ∩ V (2). As before, general conditions on q(1), q(2), q(3) can be given such that the set satisfies cross-balance with measures m(1), m(2), m(3). Under these conditions the equilibrium distribution of the amalgamated process and thus the equilibrium distribution of the original process can be obtained. To illustrate the implications of these assumptions on the transition rates of the original process consider the following special case. Assume that forn ¯ ∈ V (3) the invariant measures satisfy m(1)(¯n) = m(2)(¯n) = m(3)(¯n) = m(¯n). Then it is obvious that under the assumptions previously made for q(1) and q(2), without any further assumptions on q(3), the set satisfies cross- balance. The transition rates for the amalgamated process with amalgamation coefficients r(1) = 1, r(2) = 1, r(3) = −1 are given in (3.9). Furthermore, the amalgamated process has a unique equilibrium distribution π at V given by π = B(m(1) + m(2) − m(3)) 180 Amalgamation of Markov chains where B is a normalizing constant. As can be seen from this example, in order to derive the equilibrium distribution of the amalgamated process the following assumption on the processes in the set and thus on the original process is sufficient. • There exists a measure m for the original process that satisfies the global balance equations at V (k), k = 1, 2, 3, i.e. for alln ¯ ∈ V (k) and for k = 1, 2, 3, m is a solution to X {m(¯n)q(¯n, n¯0) − m(¯n0)q(¯n0, n¯)} = 0. n¯06=¯n, n¯0∈V (k)

The other assumptions made on the transition rates of the processes in the set, i.e. (3.10a) – (3.10c) are implied by this assumption. (3.10a) is implied by global balance atn ¯4 for process 1, (3.10b) by global balance (1) (2) atn ¯7 for process 2, and (3.10c) is trivially satisﬁed since m = m and q(1) = q(2) at V (3).

3.7 Product-form queueing networks The previous examples may seem artiﬁcial as the shape of the irreducible sets does not correspond to the shape of the irreducible sets for product- form queueing networks. These examples were chosen to illustrate the consequences of cross-balance for a set of queueing networks. For product- form queueing networks at “standard” irreducible sets cross-balance does not impose major restrictions on the transition rates of the processes in the set. This example considers product-form queueing networks and shows that a product-form equilibrium distribution can be concluded at irreducible sets of arbitrary shape provided that the process satisﬁes some conditions at the boundary of the irreducible set. For simplicity, a two station queueing network is discussed only, because the irreducible set can in this case be depicted. The results can immediately be extended to queueing networks consisting of an arbitrary number of stations, in particular, Lemma 3.1 holds true in arbitrary dimension. Consider an open two station queueing network at V˜ with transition rates ˜ q(¯n, n¯ − ei + ej) = µi(ni)pij1{n,¯ n¯ − ei + ej ∈ V }, i, j = 0, 1, 2, (3.11) 7.3 Examples 181

@ @ @ @ @ @ @ @ @ @ @ V (2)@ (1,@2) @ V @ @ V (1) @ @

Figure 7.2. Intersection of irreducible sets

˜ 2 where V ⊂ IN0 is the irreducible set. It is well-known that for a triangular set V˜ , that is for V˜ of the form ˜ ˜ ˜ ˜ V = {n¯ : n1 ≥ N1, n2 ≥ N2, n1 + n2 ≤ N3}, (3.12) the queueing network possesses a unique equilibrium distributionπ ˜ at V˜ given by 2 ni c π˜(¯n) = Bm˜ (¯n), m(¯n) = Y Y i , (3.13) i=1 k=1 µi(k) where B˜−1 = X m(¯n), n¯∈V˜ and {ci} the unique solution to the traﬃc equations

2 X ci = λp0i + cjpji, i = 1, 2. j=1

Let q(1) be the transition rates of a queueing network at triangular set V (1), that is q(1) has the form (3.11) for V˜ = V (1). Consider a second queueing network with transition rates q(2) at triangular set V (2). Then either V (1) ∩V (2) = ∅, or V (1) ∩V (2) 6= ∅. If V (1) ∩V (2) = ∅, then for the set of processes with transition rates q(1), q(2), the amalgamated process with amalgamation coefficients r(1) = 1, r(2) = 1 has an invariant measure m as given in (3.13) at V = V (1) ∪ V (2). If V (1) ∩ V (2) 6= ∅, then the intersection of V (1) and V (2) is a triangular set, say V (1,2), as depicted in Figure 7.2, where the emptyset and the set consisting of a single state are considered to be triangular sets too (this is in agreement with the definition (3.12)). The queueing networks with transition rates q(1), q(2), q(1,2) at irreducible sets V (1),V (2),V (1,2) have invariant measures m as given in (3.13). For 182 Amalgamation of Markov chains this set of queueing networks to satisfy cross-balance, by analogy with the construction method presented in Example 3.6, the transition rates q(1) at V (2), q(2) at V (1), and q(1,2) at V (1) ∪ V (2) must be defined as (1) 0 (2) 0 (2) 0 (1) q (¯n, n¯ ) = q (¯n, n¯ )1{n¯ ∈ V , n¯ =n ¯ − ei + ej ∈ V }, (2) 0 (1) 0 (1) 0 (2) q (¯n, n¯ ) = q (¯n, n¯ )1{n¯ ∈ V , n¯ =n ¯ − ei + ej ∈ V }, (1,2) 0 (2) 0 (2) 0 (1) q (¯n, n¯ ) = q (¯n, n¯ )1{n¯ ∈ V , n¯ =n ¯ − ei + ej ∈ V } (1) 0 (1) 0 (2) + q (¯n, n¯ )1{n¯ ∈ V , n¯ =n ¯ − ei + ej ∈ V } Then, since m(1) = m(2) = m(1,2) = m, this set of processes satisfies cross-balance, and the amalgamated process with amalgamation coefficients r(1) = 1, r(2) = 1, r(1,2) = −1 has a unique invariant measure m given in (3.13) at V = V (1)∪V (2). The transition rates of the amalgamated process are (1) q(¯n, n¯ − ei + ej) = µi(ni)pij1{(¯n, n¯ − ei + ej ∈ V ) (2) or (¯n, n¯ − ei + ej ∈ V )}. (3.15) Transitions for the amalgamated process occur “inside” the irreducible sets of the processes in the set only. This is explicitly stated in (3.15), where, for example, transitionsn ¯ ∈ V (1) → V (2) \ V (1) are excluded. This observation states the conditions imposed on the transition rates of the amalgamated process at the boundary of the irreducible set for the amalgamated process. Now consider a third queueing network with transition rates q(3) as given in (3.11) at triangular set V (3). Then, as depicted in Figure 7.3, the intersection of V (3) with V (1),V (2),V (1,2) is a triangular set. Denote V (1,3) = V (1) ∩ V (3), V (2,3) = V (2) ∩ V (3), and V (1,2,3) = V (1,2) ∩ V (3). Then a queueing network can be defined at each of these intersections, similar to the transition rates at V (1,2) defined above, and the transition rates for queueing networks 1, 2 and 3 can be modified similar to the modification of the transition rates for queueing networks 1 and 2 given above, such that the amalgamated process with amalgamation coefficients r(1) = r(2) = r(3) = 1, r(1,2) = r(1,3) = r(2,3) = −1, r(1,2,3) = 1, has transition rates 0 0 (k) q(¯n, n¯ ) = µi(ni)pij1{n,¯ n¯ ∈ V , for some k, k = 1, 2, 3, 0 n¯ =n ¯ − ei + ej}. 7.3 Examples 183

@@ @ @ @ @ @ @ @ @ @ @ @ @ V (3) @ @ @ @ @ @ @ V (2)@ (1,@2) @ V @ @ V (1) @ @

Figure 7.3. Intersection of irreducible sets

S3 (k) at irreducible set V = k=1 V , and invariant measure m given in (3.13). The construction method for product-form queueing networks at triangular irreducible sets can be further generalized to sets consisting of K queueing networks (K ≥ 1), at irreducible sets V (k), k = 1, . . . .K. To this end, observe that the intersection of a number of triangular sets is a triangular set, that is, for {k1, . . . , kj} ⊂ {1,...,K}, k1 < ··· < kj, j ≥ 1, the set V (k1,...,kj ), deﬁned as

j (k1,...,kj ) \ (ki) V = V , k1 < k2 < ··· kj, i=1 is a triangular set, where the ordering of the ki is imposed to obtain a unique definition of the triangular sets. This can immediately be seen by complete induction. For j = 1 the statement is trivially fulfilled. Now assume that V (k1,...,kr) is a triangular set for j = r, then, as depicted in Figure 7.2, V (k1,...,kr) ∩ V (kr+1) is a triangular set too. Now consider the set of queueing networks with transition rates given in (3.11) at triangular (k1,...,kj ) sets V , for all {k1, . . . , kj} ⊂ {1,...,K}, j = 1,...,K. Then the transition rates of the queueing networks in this set can be modified such that the set satisfies cross-balance. Now consider the amalgamated process with amalgamation coefficients

r(k1,...,kj ) = (−1)j. Then the amalgamated process has transition rates

0 0 (k) q(¯n, n¯ ) = µi(ni)pij1{n,¯ n¯ ∈ V , for some k, k = 1,...,K, 0 n¯ =n ¯ − ei + ej}, 184 Amalgamation of Markov chains and an invariant measure m as given in (3.13) at

K V = [ V (k). k=1 This construction method can in principle be generalized to show that a product-form invariant measure can be concluded at arbitrary irreducible sets. However, the proof of this statement via the method discussed above would be very tedious. Therefore, below a different method is used to prove this statement. The following lemma generalizes the construction method yielding a product-form invariant measure. To this end, define V˜ to be a proper triangular set if V˜ is a triangular set consisting of at least 3 states. 2 Lemma 3.1 shows that for all irreducible sets V ⊂ IN0 which are the union of proper triangular sets, a set of queueing networks with triangular irreducible sets and transition rates (3.11) can be defined such that the amalgamated process possesses a product-form equilibrium distribution at V . The construction of the amalgamated process provides conditions on the transition rates of the amalgamated process at the boundary of V . These conditions are similar to the conditions imposed by the stop-protocol on the transition rates of product-form queueing networks. Lemma 3.1 gives a general characterization of product-form queueing networks.

2 Lemma 3.1 Let V ⊂ IN0. If V is the union of proper triangular sets, then there exists a set of queueing networks with transition rates (3.11) at triangular irreducible sets such that the amalgamated process possesses a product-form equilibrium distribution at V .

b Proof A basic triangular set with base-state n¯ is the triangular set Vn¯ = ˜ ˜ {n,¯ n¯ + e1, n¯ + e2}. If V is a proper triangular set, say V = {n¯ : n1 ≥ ˜ ˜ ˜ ˜ N1, n2 ≥ N2, n1 + n2 ≤ N3}, then V is the union of basic triangular sets

˜ [ b V = Vn¯ . {n¯∈V˜ :n1+n2

@ @ @ @ @ @ @ @ @ @ n¯ @ r @ @ @ @

Figure 7.4. Intersection of basic irreducible sets end note that the intersection of two basic irreducible sets either consists of a single state or is the emptyset. If the intersection is empty then the transition rates of the queueing networks at these basic irreducible sets need not be modified due to the presence of both queueing networks in the set. In contrast, if the intersection is not empty then the transition rates must be modified at the intersection. As depicted in Figure 7.4, each staten ¯ ∈ V can be an element of at most 3 basic irreducible sets. The following three cases must therefore be considered. 1. Ifn ¯ is an element of one basic irreducible set then the transition rates of the queueing network at this basic irreducible set need not be modified atn ¯.

2. Ifn ¯ is an element of exactly two basic irreducible sets then atn ¯ two basic irreducible sets intersect. In this case the transition rates into staten ¯ for the queueing networks at these two irreducible sets, say V (1) and V (2), must be modified. Furthermore, a third queueing network at V (1,2) = V (1) ∩ V (2) = {n¯} with transition rates inton ¯, only, must be defined. To this end note that {n¯} is a triangular set too. Therefore, define the additional transition rates

(1) (2) q (¯n − ej + ei, n¯) = µi(ni + 1)pij1{n¯ − ej + ei ∈ V }, (2) (1) q (¯n − ej + ei, n¯) = µi(ni + 1)pij1{n¯ − ej + ei ∈ V }, (1,2) (1) (2) q (¯n − ej + ei, n¯) = µi(ni + 1)pij1{n¯ − ej + ei ∈ V ∪ V }.

Each of these processes has an invariant measure given in (3.13) at the respective irreducible sets, and the set consisting of these three processes satisﬁes cross-balance. Set r(1) = r(2) = 1, r(1,2) = −1, then the amalgamated process has an invariant measure given in (3.13) atn ¯. 186 Amalgamation of Markov chains

3. Ifn ¯ is an element of three basic irreducible sets, say V (1), V (2) and V (3), then four additional process must be deﬁned with transitions into {n¯} only. Deﬁne the additional transition rates

(1) (2) (3) q (¯n − ej + ei, n¯) = µi(ni + 1)pij1{n¯ − ej + ei ∈ V ∪ V }, (2) (1) (3) q (¯n − ej + ei, n¯) = µi(ni + 1)pij1{n¯ − ej + ei ∈ V ∪ V }, (3) (1) (2) q (¯n − ej + ei, n¯) = µi(ni + 1)pij1{n¯ − ej + ei ∈ V ∪ V }, (?) S3 (i) q (¯n − ej + ei, n¯) = µi(ni + 1)pij1{n¯ − ej + ei ∈ i=1 V },

and

q(1,2)(¯n0, n¯) = q(1,3)(¯n0, n¯) = q(2,3)(¯n0, n¯) = q(1,2,3)(¯n0, n¯).

Each of these processes has an invariant measure given in (3.13) at the respective irreducible sets, and the set consisting of these three processes satisﬁes cross-balance. Set r(1) = r(2) = r(3) = 1, r(1,2) = r(1,3) = r(2,3) = −1, r(1,2,3) = 1, then the amalgamated process has an invariant measure given in (3.13) atn ¯.

The amalgamated process is now deﬁned for all statesn ¯, and the amalgamated process has transition rates

0 0 q(¯n, n¯ ) = µi(ni)pij1{n,¯ n¯ ∈ a basic irreducible set} (3.16) and invariant measure given in (3.13). 2

Note that the proof given above can be given directly, without using the notion of cross-balance, by noting that at each staten ¯ the basic irreducible sets represent the local balance equations. This is discussed in Example 5.6.2, and illustrated in Figure 5.4. Therefore, the result of Lemma 3.1 is not surprising. Note that product-form equilibrium distributions are usually obtained for triangular irreducible sets only. Lemma 3.1 generalizes the irreducible sets at which a product-form invariant measure can be obtained to irreducible sets of an arbitrary form. The condition imposed on the transition rates of the amalgamated process in (3.16) basically comes down to the stop-protocol (cf. Example 5.6.2) for irreducible sets that arise when capacity constraints are imposed on the stations of the queueing network. However, Lemma 3.1 generalizes this stop-protocol to irreducible sets of an arbitrary form. Chapter 8

Strong reversibility

1 Introduction

The previous chapters consider standard queueing networks characterized by the assumption that in each transition one customer can route between the stations only. This chapter includes transitions in which several customers can enter and leave the stations simultaneously. To this end, a birth-and-death process is introduced in which multiple components can change at the same time. With this process a wide variety of stochastic processes such as queueing networks with simultaneous changes due to batch servicing or a discrete-time structure, and clustering processes arising in polymer chemistry can be modelled. Analysis of the birth-and-death process is based on the concept of strong detailed balance as a multi-dimensional extension of the standard one-dimensional birth-and-death equation. This seems to be rather a strong assumption. However, based on the laws of statistical mechanics, various physical and chemical applications assume that the equilibrium distribution satisﬁes the strong detailed balance equations and, moreover, that the equilibrium distribution is of product-form (cf. [50]). Also, physical systems such as chemical processes generally assume that the equilibrium concentrations (the number of particles per unit of volume) satisfy deterministic detailed balance equations. These equations are related to the traﬃc equations for reversible queueing networks. In queueing networks, on the other hand, reversible routing has practical applications such as a centralized computer [7], a material handling manufacturing system [89], and protocols in telecommunications (e.g. ALOHA, BTMA and CSMA [25]).

187 188 Strong reversibility

This chapter presents a Markov chain describing both clustering processes and queueing networks and shows that any two of the following set of three properties implies the third:

– a partial symmetry-property of the transition rates;

– a solution of the deterministic traﬃc or concentration equations;

– a product-form distribution which satisﬁes the strong detailed balance equations.

The symmetry-property is shown to be a key-notion to characterize strongly reversible processes. The product-form equilibrium distribution is independent of the symmetrical part of the transition rates. From the symmetrical part of the transition rates, blocking phenomena and state- dependent activation of simultaneous transitions, such as state-dependent delay of simultaneous transitions in queueing networks, and congestion- dependent servicing and routing, can thus be modelled without destroying the product-form expression.

2 Model

Consider a continuous-time Markov chain {N(t), t ≥ 0} with state space N S ⊂ IN0 for ﬁxed n ∈ IN∪{∞}. A staten ¯ ∈ S is a vector with components N ni ∈ IN0, i = 1,...,N. Ifm ¯ ,g ¯ ∈ IN0 thenm ¯ +g ¯ denotes the vector with 0 components mi + gi, i = 1,...,N. Let q(¯n, n¯ ) denote the transition rate from staten ¯ to staten ¯0. A transition from staten ¯ to staten ¯0 may occur 0 due to gi units leaving component i, gi units entering component i, and mi units remaining at component i, i = 1,...,N. This can be explicitly incorporated in the notation by writingn ¯ =m ¯ +g ¯ andn ¯0 =m ¯ +g ¯0, wherem ¯ = (m1, . . . , mN ) is the vector of remaining units, whileg ¯ = 0 0 0 (g1, . . . , gN ) andg ¯ = (g1, . . . , gN ) are the vectors representing departing and entering units respectively. The corresponding transition rate for this particular transition is denoted by q(¯g, g¯0;m ¯ ). For speciﬁc groupsg ¯,g ¯0 the decomposition ofn, ¯ n¯0 inn ¯ =m ¯ +g ¯,n ¯0 =m ¯ +g ¯0 is unique, but a transition from staten ¯ to staten ¯0 might occur due to other groups too. The total transition rate from staten ¯ to staten ¯0 is then given by

q(¯n, n¯0) = X q(¯g, g¯0;m ¯ ). (2.1) {g,¯ g¯0,m¯ :m ¯ +¯g=¯n, m¯ +¯g0=¯n0} 8.2 Model 189

To illustrate the processes that can be modelled in the present framework, consider the following examples of processes that can be modelled in the present framework.

• Queueing networks

– Occupation-numbers Consider a queueing network of N stations. The state of the network is given byn ¯ = (n1, . . . , nN ), where ni is the number of customers at station i (occupation-number). A transition 0 0 m¯ +g ¯ → m¯ +g ¯ represents that gi customers leave and gi customers enter station i, while mi customers at station i do not move at all. – Positions Consider a closed queueing network with N customers present. A staten ¯ represents the position of the customers, i.e. ni is the number of the station at which customer i is present. In a 0 transitionm ¯ +g ¯ → m¯ +g ¯ it is assumed that either mi = 0 and gi 6= 0, or mi 6= 0 and gi = 0. In the ﬁrst case this indicates that 0 customer i moves from station gi to station gi, in the second 0 case that customer i does not move at all and that mi = mi 0 and gi = gi = 0. • Clustering processes

Consider a process in which basic units are grouped into clusters. Two clusters of size i and j can associate to form a cluster of size i + j and conversely a cluster of size i + j can dissociate into a cluster of size i and a cluster of size j. Symbolically

←− Ai + Aj −→ Ai+j,

where Ak denotes a cluster of size k. A staten ¯ = (n1, n2,...) represents for all i the number of clusters of size i. In a tran- 0 sitionm ¯ +g ¯ → m¯ +g ¯ a number gi of clusters of size i, i = 0 1, 2,..., interacts to form a number gi of new clusters of size i, i = 1, 2,.... 190 Strong reversibility

Assume that the transition rates q(¯g, g¯0;m ¯ ) have the form ψ(¯g, g¯0;m ¯ ) q(¯g, g¯0;m ¯ ) = λ(¯g, g¯0) , (2.2) φ(m ¯ +g ¯)

0 N 0 for allm, ¯ g,¯ g¯ ∈ IN0 such that bothm ¯ +g ¯ andm ¯ +g ¯ ∈ S, where λ(·), ψ(·), φ(·) are arbitrary non-negative functions. The specific form (2.2) for the transition rates does not impose a restriction on these rates since ψ(·) may contain full information ong ¯,g ¯0,m ¯ ,m ¯ +g ¯ andm ¯ +g ¯0. For any given λ(·) such that λ(¯g, g¯0) 6= 0, ψ(·) may be chosen as q(¯g, g¯0;m ¯ )φ(m ¯ +g ¯) ψ(¯g, g¯0;m ¯ ) = . λ(¯g, g¯0) If λ(¯g, g¯0) = 0 then q(¯g, g¯0;m ¯ ) = 0 for allm ¯ and an arbitrary function ψ(·) may be inserted. The form (2.2) for the transition rates differs from the form used in the previous chapters and is chosen for mathematical elegance. Here φ(·) is a service-function playing the same role as φ(·) in the previous chapters, λ(·) represents the state-independent routing probabilities and ψ(·) contains both a service and a routing part. In the sequel a generalization of the traffic equations is formulated. This generalization involves λ(·) as an extension of pij appearing in the previous chapters. For the equilibrium distribution to be of product-form, ψ(·) will be assumed a symmetrical function ofg ¯ andg ¯0. The Markov chain {N(t), t ≥ 0} with transition rates (2.2) is assumed to a stable, regular, and ergodic at V ⊂ S. Theorem 2.1.2 implies that the equilibrium distribution can be obtained as the unique invariant measure 0 N summing to unity. In addition, assume that for allg ¯,g ¯ ∈ IN0 for which N 0 there exists anm ¯ ∈ IN0 such thatm ¯ +g ¯,m ¯ +g ¯ ∈ V λ(¯g, g¯0) > 0 ⇐⇒ λ(¯g0, g¯) > 0, (2.3)

0 N for allg, ¯ g¯ ∈ IN0 . Note that (2.3) does not express a restriction on the transition rates since for λ(¯g, g¯0) > 0 the reversed transition can be blocked 0 N by setting ψ(¯g , g¯;m ¯ ) = 0 for allm ¯ ∈ IN0 . Irreducibility of {N(t), t ≥ 0} is determined by ψ(·). In contrast to φ(·), which must be a positive function, i.e. φ(¯n) > 0 for alln ¯ ∈ S, the function ψ(·) is allowed to be zero. Furthermore, ψ(·) may depend on both statesm ¯ +g ¯ andm ¯ +g ¯0 explicitly. Blocking of transitions can thus be included as will be illustrated in Section 3. The following property for ψ(·) plays a key-role. 8.2 Model 191

Definition 2.1 (Symmetry-property) The function ψ(·) satisfies the symmetry-property at V when ψ(·) is a symmetrical function in the first 0 N two arguments at V whenever λ(·) is positive, i.e. for all m,¯ g,¯ g¯ ∈ IN0 such that m¯ +g ¯, m¯ +g ¯0 ∈ V and both λ(¯g, g¯0) > 0 and λ(¯g0, g¯) > 0 :

ψ(¯g, g¯0;m ¯ ) = ψ(¯g0, g¯;m ¯ ). (2.4)

The function λ(·) is an essential part of the transition rates (2.2). Without λ(·) all transition rates are symmetric ing ¯,g ¯0 whenever the symmetry-property is valid. The explicit appearance of λ(·) in the transition rates is also very natural. In queueing networks λ(·) may represent the routing probability and in chemical reactions λ(·) represents the reaction speed, but this restriction is not imposed here. The following deﬁnition states a relation for λ(·) often imposed in chemical and physical processes [50], but also imposed in queueing network applications such as centralized computer models [7], manufacturing systems [89], and telecommunications [25]. For λ(¯g, g¯0), the reaction speed for a reactiong ¯ → g¯0, and ci the concentration of component i, i = 1, 2,..., the concentration equations or deterministic detailed balance equations (2.5) state that the 0 0 Q gk total reaction speed for transitionsg ¯ → g¯ in the system, λ(¯g, g¯ ) k ck , equals the total reaction speed for the reversed transition. This expresses the deterministic equilibrium concept. Deﬁnition 2.2 (Deterministic detailed balance) The Markov chain N is deterministically reversible if a positive solution {ci}i=1 exists to the 0 N deterministic detailed balance equations for all g¯, g¯ ∈ IN0 for which there N 0 exists at least one m¯ ∈ IN0 such that m¯ +g ¯, m¯ +g ¯ ∈ V :

0 0 Y gk 0 Y gk λ(¯g, g¯ ) ck = λ(¯g , g¯) ck . (2.5) k k This chapter considers birth-and-death processes. For standard birth- N and-death processes with state space S ⊂ IN0 , the equilibrium distribution is obtained by noting that the process jumps to the right (n → n + 1) or jumps to the left (n+1 → n). This process can be in equilibrium only if the probability flow between states n and n+1 are equal. This implies that for each “reaction” n ↔ n + 1 the probability flows for the forward and backward reaction balance. An immediate generalization of this concept is strong reversibility defined below. A Markov chain is strongly reversible if for each possible reactionm ¯ +g ¯ ↔ m¯ +g ¯0 due tog ¯ ↔ g¯0 the probability flows for the forward and backward reaction balance. 192 Strong reversibility

Deﬁnition 2.3 (Strong reversibility) A Markov chain {N(t), t ≥ 0} at state space S is strongly reversible if a measure m at S exists such that N 0 0 for all g¯, g¯, m¯ ∈ IN0 , n¯ =m ¯ +g ¯, n¯ =m ¯ +g ¯ ∈ S

m(¯n)q(¯g, g¯0;m ¯ ) = m(¯n0)q(¯g0, g¯;m ¯ ). (2.6)

Summation of the strong detailed balance equations (2.6) over {g,¯ g¯0, m¯ : n¯ =m ¯ +g, ¯ n¯0 =m ¯ +g ¯0} (cf. (2.1)) immediately implies that the Markov chain is reversible if it is strongly reversible. Thus, strong detailed balance is a proper name for the equations (2.6). If the transitionn ¯ ↔ n¯0 can occur due to one groupg ¯ ↔ g¯0 only, then q(¯g, g¯0;m ¯ ) = q(¯n, n¯0) and strong reversibility reduces to reversibility. The assumption thatg ¯,g ¯0 are unique for each transitionn ¯ ↔ n¯0 is not uncommon in the literature. For example, [87, Chapter 7] discusses chemical reactions in equilibrium and assumes that the transitionn ¯ ↔ n¯0 can occur due to a single type of transition (determined byg ¯,g ¯0) only. Furthermore, if single changes are allowed only, that isg ¯,g ¯0 are unit vectors, then the transitionn ¯ ↔ n¯0 uniquely determinesg ¯,g ¯0. Thus strong detailed balance reduces to detailed balance for the processes discussed in the previous chapters. The following theorem relates the properties deﬁned above and gives suﬃcient conditions for a product-form equilibrium distribution.

Theorem 2.4 For the Markov chain {N(t), t ≥ 0} at state space S with transition rates (2.2) any two of the following three properties implies the third.

(p1) The Markov chain is deterministically reversible.

(p2) ψ(·) satisﬁes the symmetry-property at V .

(p3) The Markov chain is strongly reversible with product-form equilibrium distribution π at V given by

Y nk π(¯n) = Bφ(¯n) ck , n¯ ∈ V, (2.7) k where B is a normalizing constant.

Proof The Markov chain is ergodic at V ⊂ S. Thus q(¯n, n¯0) = 0 for n¯ ∈ V ,n ¯0 ∈/ V , and when checking global balance it may be assumed that π(¯n0) = 0 forn ¯0 ∈/ V . 8.2 Model 193

0 N 0 0 Forg, ¯ g¯ , m¯ ∈ IN0 such thatn ¯ =m ¯ +g, ¯ n¯ =m ¯ +g ¯ ∈ V , insertion of (2.2) and (2.7) into (2.6) gives for the left-hand side

0 0 Y nk 0 ψ(¯g, g¯ ;m ¯ ) π(¯n)q(¯g, g¯ ;m ¯ ) = Bφ(¯n) ck λ(¯g, g¯ ) k φ(m ¯ +g ¯) ( )" # 0 Y mk 0 Y gk = Bψ(¯g, g¯ ;m ¯ ) ck λ(¯g, g¯ ) ck , (2.8a) k k and for the right-hand side

0 0 0 0 0 Y nk 0 ψ(¯g , g¯;m ¯ ) π(¯n )q(¯g , g¯;m ¯ ) = Bφ(¯n ) ck λ(¯g , g¯) 0 k φ(m ¯ +g ¯ )

( )" 0 # 0 Y mk 0 Y gk = Bψ(¯g , g¯;m ¯ ) ck λ(¯g , g¯) ck . (2.8b) k k Comparison of (2.8a) and (2.8b) shows that the theorem holds true.

– If (p1), (p2) are satisfied, then the terms in brackets {·} and in square brackets [·] are identical. Thus, π as specified in (2.7) indeed satisfies (2.6), implying (p3).

– If (p2), (p3) are valid, then for π as given in (2.7) the left-hand sides of (2.8a), (2.8b) are equal. Furthermore, (p2) implies that the terms in brackets {·} are identical, thus forcing (p1).

– If (p1), (p3) are valid, then the left-hand sides are equal and the terms in square brackets [.] are equal. 2

Although the concentration equations are natural in various applications, they are not necessarily fulfilled in general. For queueing networks (2.6) extends the standard reversible routing or traffic equations to queueing networks in which multiple customers move among the stations at the same time. The reversibility assumption is maintained in this generalization. As a consequence, non-reversible queueing networks, such as feed-forward networks, cannot be included. Non-reversible queueing networks are discussed in Chapter 9. In contrast to the strong routing assumptions, the function ψ(·) is allowed to be very general. Since ψ(·) does not appear in the equilibrium distribution for different functions ψ(·) the equilibrium distribution remains unchanged. This allows complex systems (complex ψ(·)) to be 194 Strong reversibility modelled as simple systems (simple ψ(·)). Furthermore, ψ(·) is allowed to be zero implying that blocking of transitions is included in Theorem 2.4. Theorem 2.4 characterizes strongly reversible processes, but it does not give a necessary condition for strong reversibility. The following theorem directly relates strong reversibility to deterministic reversibility and implies a necessary and sufficient condition for a product-form equilibrium distribution. Theorem 2.5 Suppose N ∈ IN and λ(¯g, g¯0) > 0 for a finite number of combinations g¯, g¯0 only. Then under the symmetry-property the process with transition rates (2.2) is strongly reversible if and only if (2.5) possesses a positive solution. Proof If the deterministic detailed balance equations possess a positive solution then Theorem 2.4, (p1), (p2) ⇒ (p3), implies that the process is strongly reversible. Conversely, assume that ψ(·) satisfies the symmetry-property and the process is strongly reversible. Let π(¯n) = φ(¯n)p(¯n). Then, by virtue of (2.4), the strong detailed balance equations become for g¯,g ¯0,m ¯ such thatm ¯ +g, ¯ m¯ +g ¯0 ∈ V λ(¯g, g¯0)p(m ¯ +g ¯) = λ(¯g0, g¯)p(m ¯ +g ¯0). (2.9) The number of the combinations (¯g, g¯0) for which λ(¯g, g¯0) > 0 is denumer- 0 0 0 able. Let (¯g, g¯ )i denote the i-th combination and define (¯gi, g¯i) = (¯g, g¯ )i 0 0 0 and λi = λ(¯gi, g¯i) and λi = λ(¯gi, g¯i). The strong detailed balance equations (2.9) can now be written

0 0 λip(m ¯ +g ¯i) = λip(m ¯ +g ¯i). (2.10) From here the proof is almost identical to the one given in [87, pp. 162- 163]. To highlight the assumptions on ﬁniteness of N and the number of combinations for which λ(¯g, g¯0) > 0 the remainder of the proof is given explicitly. 0 Letn ¯0 be a ﬁxed value ofn ¯ and set φi = log(λi/λi). Let D the matrix 0 with (ri)-th element dri = (¯gi)r − (¯gi)r, the net number of units entering component r in reaction i. Since the process is irreducible, for eachn ¯ ∈ V there exists a vector ξ(¯n) such that

n¯ =n ¯0 + Dξ(¯n), (2.11) 8.2 Model 195

where ξi(¯n) denotes the number of times the i-th transition occurs in passage fromn ¯0 ton ¯. For eachn ¯ the vector ξ(¯n) can be obtained from (2.11)

+ ξ(¯n) = D (¯n − n¯0) + ζ. (2.12)

The vector ζ lies in the null space of D, i.e. Dζ = 0. The operator D+ is the pseudoinverse of D determined by requiring that for passages fromn ¯0 ton ¯ + the vector D (¯n − n¯0) lies in the orthogonal complement of the null space of D (cf. Remark 2.6). At this point finiteness of the number of rows and columns is required since ξ(¯n) is decomposed into a vector in the null space of a linear operator and a vector in the orthogonal complement of the null space. Each finite dimensional Hilbert space can thus be decomposed. An infinite dimensional Hilbert space, however, may contain vectors contained in neither the null space nor its orthogonal complement. Furthermore, if the number of rows and columns of D is finite then the pseudoinverse + D of D is independent of the vectorn ¯ − n¯0. If the number of rows or columns of D is infinite then this independence cannot be guaranteed. The remainder of the proof is based on the fact that D+ is independent ofn ¯ − n¯0. Observe that (2.10) can be written

φ·ξ(¯n) p(¯n0 + Dξ(¯n)) = p(¯n0)e , (2.13) where · denotes the inner-product of vectors. Insertion of (2.12) into (2.13) gives forn ¯ =n ¯0 that φ · ζ = 0. Forn ¯ ∈ V this implies

h −1 i p(¯n) = p(¯n0) exp φD (¯n − n¯0) . (2.14)

−1 −1 Observe that D is a linear operator with elements dir , and deﬁne " # X −1 cr = exp φidir . i then (2.14) can be written

Y nr−n0r p(¯n) = p(¯n0) cr . r Substitution of this relation into (2.9) completes the proof. 2

Theorem 2.5 is a direct generalization of Lemma 7.3.1 in [87] to processes with multiple transitions between states and more general transition 196 Strong reversibility rates. The proof of Theorem 2.5 is adopted from [87]. The result of The- orem 2.5 cannot be obtained from Theorem 2.4 since Theorem 2.5 makes no assumptions on the equilibrium distribution. The assumptions N ∈ IN and λ(¯g, g¯0) > 0 for a finite number of pairs (¯g, g¯0) are essential to the proof of Theorem 2.5. These assumptions are natural in many practical applications. For example, in a chemical process only a finite number of different reactions is possible, and in a queueing network only finite batches can be served simultaneously.

Remark 2.6 (Pseudoinverse) Consider the system of linear equations

Dx¯ = ¯b, (2.15) where D is an m by n matrix, and ¯b an m-vector. If D has full column- rank, that is the columns of D are linearly independent, then D has an n by m left inverse B = (DT D)−1DT , where DT denotes the transpose of D, andx ¯ = B¯b is the unique solution of the linear equations (2.15). If D does not have full column-rank then the null-space of D contains elements y¯ 6= 0.¯ Ifx ¯ is a solution of (2.15) thenx ¯+y ¯ is a solution of (2.15) too. The left inverse of D cannot be deﬁned in this case. However, the generalized inverse or pseudoinverse of D can now be deﬁned similar to the case with full column-rank, that is a unique solutionx ¯+ of (2.15) can be obtained. To this end, note that any m by n matrix can be factored into

T D = Q1ΣQ2 , where Q1 is an m by m orthogonal matrix whose columns are the eigen- T vectors of DD , Q2 is an n by n orthogonal matrix whose columns are the eigenvectors of DT D, and Σ is an m by n diagonal matrix whose diagonal elements are the square roots of the non-zero eigenvalues of both DDT and DT D. The pseudoinverse D+ of D is [78]

+ + T D = Q2Σ Q1 ,

+ where Σ is the pseudoinverse of Σ: if σ1, . . . , σr are on the diagonal of Σ + then the n by m diagonal matrix Σ has on the diagonal 1/σ1,..., 1/σr. If ¯b is in the column space of D (by the assumption on irreducibility this is guaranteed in (2.11)), then a solutionx ¯ of (2.15) exists. The vector

x¯+ = D+¯b is the unique shortest solution to the system of linear equations (2.15). 2 8.2 Model 197

Theorems 2.4 and 2.5 show that the symmetry-property plays a key- role in strongly reversible processes. Not only does it relate reversibility for deterministic processes, but it also implies that a strongly reversible process possesses a product-form equilibrium distribution. The results stated above show that the Markov chain with transition rates (2.2) such that ψ(·) satisﬁes the symmetry-property is strongly reversible with product-form equilibrium distribution if a deterministic model with reaction speeds λ(·) is reversible. These results show that the Markov chain is a stochasticized version of the deterministic model. How- ever, these results do not show that the Markov chain is a proper stochastic model for the deterministic process. For this to be true it must be the case that the deterministic process represents the average behaviour of the stochastic process, that is the average probability ﬂow in the stochastic process must satisfy the deterministic detailed balance equations. The following theorem shows that the Markov chain with transition rates (2.2) and product-form equilibrium distribution (2.7) is a proper stochasticized version of the deterministic model.

Theorem 2.7 Consider the Markov chain with transition rates (2.2), N where ψ(·) satisﬁes the symmetry-property at V and {ci}i=1 is a positive solution to the deterministic detailed balance equations. Then under 0 N the equilibrium distribution for any ﬁxed g¯, g¯ ∈ IN0

( 0 ) ψ(¯g, g¯ ;m ¯ ) 0 Y gk E = f(¯g, g¯ , c) ck , φ(m ¯ +g ¯) k with 0 X 0 Y mk f(¯g, g¯ , c) = B ψ(¯g, g¯ ;m ¯ ) ck , N k {m¯ ∈IN0 :m ¯ +¯g∈V } where f(·) is a symmetric function of g¯, g¯0, i.e.

f(¯g, g¯0, c) = f(¯g0, g,¯ c), whenever λ(¯g, g¯0) > 0. The expected transition rate for a transition g¯ → g¯0 is thus given by 0 0 Y gk λ(¯g, g¯ )f(¯g, g¯ , c) ck . (2.16) k Proof Under the conditions of the theorem the equilibrium distribution is given in (2.7). Direct computation of the expectation value gives for 198 Strong reversibility any ﬁxedg, ¯ g¯0

(ψ(¯g, g¯0;m ¯ )) ψ(¯g, g¯0;m ¯ ) E ≡ X π(¯n) φ(m ¯ +g ¯) {n¯∈V :∃ m¯ such thatn ¯=m ¯ +¯g} φ(m ¯ +g ¯)

X 0 Y mk+gk = Bψ(¯g, g¯ ;m ¯ ) ck N k {m¯ ∈IN0 :m ¯ +¯g∈V }    X 0 Y mk  Y gk = B ψ(¯g, g¯ ;m ¯ ) ck  ck . N k k {m¯ ∈IN0 :m ¯ +¯g∈V }

It remains to prove that f(·) is a symmetric function ofg, ¯ g¯0 whenever λ(¯g, g¯0) > 0. First note that terms with ψ(·) = 0 do not add to the sum, and that ψ(·) is a symmetric function ofg, ¯ g¯0, so it is suﬃcient to show that

N 0 {m¯ ∈ IN0 :m ¯ +g ¯ ∈ V, ψ(¯g, g¯ ;m ¯ ) > 0} N 0 0 = {m¯ ∈ IN0 :m ¯ +g ¯ ∈ V, ψ(¯g, g¯ ;m ¯ ) > 0}, whenever λ(¯g, g¯0) > 0 so that from (2.3) it follows that also λ(¯g0, g¯) > 0. To this end, note that q(¯g, g¯0;m ¯ ) > 0 whenever both λ(¯g, g¯0) > 0 and ψ(¯g, g¯0;m ¯ ) > 0 so that the two sets are the same. 2

For a deterministic model with concentrations ci and reaction speeds λ(·), the standard method in stochasticizing the model is to replace ci by ni/R, where R is the volume of the system. This is an obvious method since the concentration is deﬁned as the number of units per volume. Theorem 2.7 shows that a proper stochastic model may have much more structure than the linear model obtained by replacing ci by ni/R. For example, blocking eﬀects can be incorporated into ψ(·), and more general state-dependencies in the stochastic reaction rates into φ(·). Due to the symmetry of f(·), for these generalizations the expected rates still satisfy the deterministic detailed balance equations:

(ψ(¯g, g¯0;m ¯ )) (ψ(¯g0, g¯;m ¯ )) λ(¯g, g¯0)E = λ(¯g0, g¯)E . φ(m ¯ +g ¯) φ(m ¯ +g ¯0)

If f(·) ≡ 1, the expected equilibrium rate reduces to the basic deterministic reaction rate. The explicit appearance of f(·) in (2.16) is a consequence of blocking or service-delay in the stochastic model. In the basic deterministic model blocking of transitions cannot occur. The symmetry of 8.3 Applications 199 f(·) implies that the concentration equations (2.5) remain valid since f(·) drops out. So although the left-hand and right-hand side in (2.5) do not reﬂect the deterministic reaction rates, the solution ci remains valid.

3 Applications

This section contains two application categories of the preceding results: queueing networks and clustering processes. In each category both known results for the purpose of uniﬁcation and illustration, and novel results showing some of the possible extensions are presented. Simple examples illustrating the key-features are given. In each example the symmetry- property is proven and therefore the equilibrium distribution is of product- form. First, Section 3.1 considers standard queueing networks with single changes as discussed in the previous chapters. Example 3.1.1 illustrates that blocking phenomena are covered. Example 3.1.2 introduces service- rates at one station to be inﬂuenced by the state at other stations more general than is reported in the literature. Example 3.1.3 extends the standard convexity condition for a product-form to hold under blocking of transitions to the inclusion of holes in the state space. The examples given for standard queueing networks can be derived from previous results appearing in the literature too (cf. [46], [56]). These examples are given to illustrate how the framework of Section 2 applies, in particular, to illustrate the role of ψ(·). Second, in Section 3.2 the examples given in Section 3.1 are extended to queueing networks with multiple changes. These are new as both multiple changes and blocking are included. Third, clustering processes from polymer chemistry are discussed in Section 3.3 in a more general way than illustrated in Section 2. Blocking of reactions is included into the framework of clustering processes in Example 3.3.4 and state-dependencies in the transition rates leading to duality relations similar to those discussed in Chapter 5 are introduced in Example 3.3.5.

3.1 Queueing networks with single changes Consider a closed or open queueing network with stations 1, 2,...,N, in which all customers are of the same type. The state of the network isn ¯ = (n1, . . . , nN ), where ni denotes the number of customers at station i. Single changes are allowed only, thus a transitionn ¯ → n¯0 uniquely determinesg ¯ andg ¯0, and strong reversibility reduces to reversibility. 200 Strong reversibility

Suppose that the transition rates of the network are

ψ(j, k;m ¯ ) q(m ¯ + ej, m¯ + ek) = λjk , j, k = 0, 1,...,N. (3.1) φ(m ¯ + ej) Let V be the set at which the queueing network is irreducible, then for all j, k such thatm ¯ + ej, m¯ + ek ∈ V , the function ψ(·) must be symmetric, i.e. ψ(j, k;m ¯ ) = ψ(k, j;m ¯ ). (3.2) In this special case the concentration equations (2.5) simplify to

cjλjk = ckλkj, which is the standard reversible traffic equation with cj the throughput at station j and (λjk) the routing matrix. The equilibrium distribution at V is given by Y nk π(¯n) = Bφ(¯n) ck . (3.3) k Reversible queueing networks have been studied extensively. However, the unifying form (3.1) does not appear in the literature. In contrast, in the literature one finds ψ(j, k;m ¯ ) = φ(m ¯ ), where φ(·) is strictly positive ([15], [55], [83], [87]). In [56] fairly general reversible processes including blocking of transitions are discussed. Here the transition rates are q(¯n, n¯− ei + ej) = λijφi(ni)ψj(nj), not as general as the form (3.1), but since ψ 6= φ and ψj(nj) = 0 is possible, in some respect more general than the transition rates obtained with ψ(j, k;m ¯ ) = φ(m ¯ ). In [53] the model of [56] is generalized to a reversible model with transition rates q(¯n, n¯ −ei +ej) = Φ(¯n−ei) Ψ(¯n) λij . This model can be incorporated into the framework Φ(¯n) Ψ(¯n+ej ) presented here by setting φ(¯n) = Ψ(¯n) , and ψ(i, j;m ¯ ) = Φ(m ¯ ) . The Φ(¯n) Ψ(m ¯ +ei+ej ) fact that different functions ψ(·) and φ(·) are allowed in (3.1) with explicit dependence on j, k in ψ(·) includes blocking and state-dependent servicing into the model. This is illustrated in the following examples. In principle these examples fit into the framework of [46], [56] and [76] and are given here to illustrate the function ψ(·).

3.1.1 1-0 blocking Consider a closed queueing network of two stations in which M customers are present. Let the service-capacity of station i be fi(xi), i = 1, 2, when 8.3 Applications: queueing networks with single changes 201

xi customers are present at station i. When at most N customers are allowed at station 2 the transition rates are ( f1(m1), if m2 < N, q(m ¯ + e1, m¯ + e2) = 0, if m2 ≥ N,

q(m ¯ + e2, m¯ + e1) = f2(m2).

These transition rates have the form (3.1) which is easily seen by setting

2 nk 1 2 mk 1 φ(¯n) = Y Y , ψ(i, j;m ¯ ) = b(i, j;m ¯ ) Y Y , k=1 l=1 fk(l) k=1 l=1 fk(l)

 1, if i = 2, j = 1,  b(i, j;m ¯ ) = 1, if i = 1, j = 2, m2 < N,  0, elsewhere,

λjk = 1, j, k = 1, 2. It is easy to see that b(·), and therefore ψ(·), is symmetric at V = {(n1, n2): n1 ≥ 0, 0 ≤ n2 ≤ N, n1 + n2 = M}. This leads to the equilibrium distribution at V :

2 n Y Yk 1 π(n1, n2) = B . k=1 l=1 fk(l)

The product-form above can be found for instance in [46], [56], and [60]. This example illustrates that the present framework allows strict blocking, which is non-reversible or non-symmetric by nature, despite the underlying reversibility structure on the network and the symmetry-property (3.2). More speciﬁc, observe that this example uses that (3.2) is required only restricted to V but not at the boundary of V .

3.1.2 State-dependent delay; congestion-dependent routing Consider a closed queueing network consisting of three stations. Let the service-capacity at station i be fi(xi) when xi customers are present. At station 3 customers are accepted with probability b(m3), while the service and routing between stations 1 and 2 is influenced by the number of customers at station 3. This influence is reflected by the appearance of 202 Strong reversibility

˜ b(m3) in the transition rates for transitions between stations 1 and 2. The transition rates for this network are ˜ q(m ¯ + e1, m¯ + e2) = f1(m1 + 1)b(m3), ˜ q(m ¯ + e2, m¯ + e1) = f2(m2 + 1)b(m3)/2, q(m ¯ + e2, m¯ + e3) = f2(m2 + 1)b(m3)/2, q(m ¯ + e3, m¯ + e2) = f3(m3 + 1).

These transition rates can be written in the form (3.1) by setting

3 n 3 m Y Yk 1 Y Yk 1 φ(¯n) = γ(n3 − 1) , ψ(i, j;m ¯ ) = α(i, j;m ¯ ) , k=1 l=1 fk(l) k=1 l=1 fk(l) ˜ α(1, 2;m ¯ ) = α(2, 1;m ¯ ) = γ(m3 − 1)b(m3), α(2, 3;m ¯ ) = α(3, 2;m ¯ ) = γ(m3), 1 λ = 1, λ = λ = , λ = 1, 12 21 23 2 32 with

x b γ(x) = Y b(k) and the convention Y b(k) = 1 if a > b. k=0 k=a ψ(·) as deﬁned above satisﬁes the symmetry-property (3.2). When M customers are present in the network, the equilibrium distribution at V = {(n1, n2, n3): n1 + n2 + n3 = M, n1 ≥ 0, n2 ≥ 0, n3 ≥ 0} is given by

n −1 ! 3 n ! Y3 Y Yk 1 π(¯n) = B b(k) 2n2 . k=0 k=1 l=1 fk(l) Two direct examples of this result are the following.

Example 1 The service-speed at stations 1 and 2 is decreased when the number of customers at station 3 exceeds a certain level: ( 1, if k ≤ N , b(k) = ˜b(k) = 3 1/2, elsewhere.

Example 2 As φ(¯n) > 0 for alln ¯ it must be that b(k) 6= 0 for all k, but ˜b(k) may have an arbitrary value, including 0. Let station 2 produce customers that can 8.3 Applications: queueing networks with single changes 203 be further served at station 1 and 3. After completing service at station 1 or 3 the customers return to station 2. Assume a customer prefers service at station 3 and will never route to station 1 when at station 3 less than N1 customers are present. When at station 3 more than N1 but less than N2 customers are present a customer will not distinguish between stations 1 and 3. When more than N2 customers are present at station 3 a customer will prefer service at station 1, but in this case still some customers are being served at station 3. Now we may choose the following values for b(·) and ˜b(·) for arbitrary α > 0 ˜ b(k) = 1, b(k) = 0, if k ≤ N1, 1 ˜ 1 b(k) = 2 , b(k) = 2 , if N1 < k ≤ N2, 1 ˜ α b(k) = α , b(k) = (k − N2) , if k > N2. 2(k − N2)

3.1.3 Transition gaps This example investigates the possibility of blocking also in the interior of V . In this case the symmetry-property (3.2) will play a restrictive role. To this end, consider an open queueing network of two parallel stations. The transition rates of the network depend on the state of both stations. Suppose the number of customers in the stations is restricted by ˆ V = {(n1, n2) : 0 ≤ n1 ≤ 9; 0 ≤ n2 ≤ 10; n1 + n2 ≤ 15}.

Let all transitions due to the arrival or departure of customers be possible except for some transitions to and from states with m1 = m2 = even. Note that for the process to be reversible q(¯n, n¯0) = 0 forn, ¯ n¯0 ∈ V implies that also q(¯n0, n¯) = 0, an observation already made in [56]. For example, suppose that state (2,2) can only be reached from states (2,3) and (3,2), so that

q((1, 2), (2, 2)) = q((2, 2), (1, 2)) = 0, q((2, 1), (2, 2)) = q((2, 2), (2, 1)) = 0, and similarly (4,4) only from (5,4), so that

q((·, ·), (4, 4)) = q((4, 4), (·, ·)) = 0, for (·, ·) = (4, 5), (3, 4), (4, 3), while (6,6) cannot be reached at all, that is q((·, ·), (6, 6)) = ((6, 6), (·, ·)) = 0, for (·, ·) = (7, 6), (6, 7), (5, 6), (6, 5). 204 Strong reversibility

Assuming that all other transitions within Vˆ are positive gives

V = Vˆ \{(6, 6)}.

−1 When both stations are single-server-queues with mean service-time µi , i = 1, 2, and the arrival stream is Poisson λi, i = 1, 2, the transition rates of this network have the form (3.1) with

λi0 = µi, λ0i = λi, φ(m ¯ ) = 1, ψ(i, j;m ¯ ) = 1, except for ψ(1, 0; (1, 2)) = ψ(0, 1; (1, 2)) = 0, ψ(0, 2; (2, 1)) = ψ(2, 0; (2, 1)) = 0, ψ(0, 2; (4, 4)) = ψ(2, 0; (4, 4)) = 0, ψ(1, 0; (3, 4)) = ψ(0, 1; (3, 4)) = 0, ψ(0, 2; (4, 3)) = ψ(2, 0; (4, 3)) = 0, ψ(1, 0; (6, 6)) = ψ(0, 1; (6, 6)) = 0, ψ(0, 2; (6, 6)) = ψ(2, 0; (6, 6)) = 0, ψ(1, 0; (5, 6)) = ψ(0, 1; (5, 6)) = 0, ψ(0, 2; (6, 5)) = ψ(2, 0; (6, 5)) = 0. Despite these exclusions, however, the following standard type product- form distribution can now be concluded from (3.3):

λ !m1 λ !m2 π(m ¯ ) = B 1 2 , m¯ ∈ V. µ1 µ2

Note that V is not convex. This example thus relaxes the “standard” condition of coordinate or graphical convex blocking-protocols for a product- form to hold (cf. [33], [46], [51], [58], [60]).

3.2 Queueing networks with multiple changes Now consider a closed queueing network in which groups of customers can change stations simultaneously. Suppose that the transition rate for 0 a transition in which gi customers leave station i, gi customers arrive at station i and mi customers remain at station i, i = 1,...,N, is given by ψ(¯g, g¯0;m ¯ ) q(¯g, g¯0;m ¯ ) = λ(¯g, g¯0) . φ(m ¯ +g ¯) 8.3 Applications: queueing networks with multiple changes 205

The deterministic detailed balance equations (2.5) are now called batch routing equations and read

0 0 Y gk 0 Y gk λ(¯g, g¯ ) ck = λ(¯g , g¯) ck . (3.4) k k

Analogous to the case of single changes cj can be interpreted as the throughput at station j with λ(¯g, g¯0) representing a routing probability. If ψ(·) satisﬁes the symmetry-property at V then the equilibrium distribution at V is given by Y nk π(¯n) = Bφ(¯n) ck . k A similar framework of queueing networks with batch services is investi- gated and shown to have a product-form in [40], [41]. On the one hand, these references are more general as they allow arbitrary rather than reversible routing as imposed here, on the other hand these references are more restrictive as the explicit dependence ong ¯ andg ¯0 in ψ(·) is not included. In particular, blocking is thereby excluded, while in the framework presented here blocking of particular type transitions is possible as will be illustrated hereafter. In Chapter 9, for queueing networks the framework of this chapter is extended to non-reversible queueing networks with blocking.

3.2.1 1-0 blocking Reconsider the cyclic two station queueing network of Example 3.1.1 with a ﬁnite capacity constraint of no more than N customers allowed at station 2, but now with the transition rates for multiple departures. Assume that customers decide to change stations independently. Then the probability that a groupg ¯ decides to change stations in statem ¯ +g ¯ is given by 2 ! Y m + g g p(¯g, m¯ +g ¯) = k k p k (1 − p )mk , g k k k=1 k where pk is the probability that a customer at station k decides to leave. After leaving their stations a groupg ¯ routes tog ¯0 according to the routing probability λ(¯g, g¯0). Depending on the number of customers at station 2 customers entering station 2 can be blocked. If one customer is blocked then the whole group is blocked. The transition rates for this process are 206 Strong reversibility given by   0 1 0 0  λ(¯g, g¯ )p(¯g, m¯ +g ¯) 0 0 , if m2 + g2 ≤ N, q(¯g, g¯ ;m ¯ ) = g1!g2!  0, otherwise.

By setting

2 1 1 2 1 1 − p !mk φ(¯n) = Y , ψ(¯g, g¯0;m ¯ ) = b(¯g, g¯0;m ¯ ) Y k , nk 0 k=1 nk! pk k=1 mk!gk!gk! pk

( 0 0 1, if m2 + g2 ≤ N, b(¯g, g¯ ;m ¯ ) = 0 0, if m2 + g2 > N, λ(¯g, g¯0) = 1, it is easily seen that ψ(·) satisﬁes the symmetry-property at V = {(n1, n2): n1 ≥ 0, 0 ≤ n2 ≤ N, n1 + n2 = M}. The results of Section 2 then apply N with {c}i=1 a solution of (3.4). The equilibrium distribution at V is given by 2 1 c !nk π(¯n) = B Y k . k=1 nk! pk

The system discussed here is applicable as a central-server computer- system with blocking and multiple departures and arrivals. As such it is closely related to a discrete-time model analysed in [23], [73]. As most essential contrast, however, in these references the restrictive underlying assumption is made that ”no more than one service-request is granted” at a time so that no more than one departure is allowed at one of the stations. No such condition is imposed here.

3.2.2 State-dependent delay; congestion-dependent routing Consider a closed queueing network with three stations as in Example 3.1.2. Assume that customers are served independently but released at negative-exponential times such as naturally arising from discrete-time formulations. After leaving their stations a groupg ¯ is routed tog ¯0 according to the routing probability λ(¯g, g¯0). Assume that customers can only route between station 1 and station 2 and between station 2 and station 3. 0 0 0 0 Then the vectorsg, ¯ g¯ must satisfy g1 + g3 = g2 and g1 + g3 = g2. De- pending on the number of customers at station 3 customers are delayed 8.3 Applications: clustering processes 207 while routing between the stations. Customers routing between station 1 and 2 are inﬂuenced by only the number m3 of customers that remain ˜ at station 3 with delay factor b(m3). Customers routing from station 2 to station 3 have full information on the total number of customers at station 3. These customers are assumed to route one by one with delay factor b(m3 + h), where m3 + h is the number of customers that still re- sides at station 3. The routing intensity from station 3 to station 2 is not inﬂuenced. The transition rates of this process are given by

3 gk ! 0 0 Y 1 Y q(¯g, g¯ ;m ¯ ) = λ(¯g, g¯ ) 0 fk(mk + l) k=1 gk!gk! l=1  0  m3+g3+g3−1 0 Y ˜ g1+g1 × b(m3) b(k) . k=m3+g3 By setting n3−1 3 nk 1 φ(¯n) = Y b(k) Y Y , k=0 k=1 l=1 fk(l)

3 mk 0 0 Y 1 Y 1 ψ(¯g, g¯ ;m ¯ ) = α(¯g, g¯ ;m ¯ ) 0 , k=1 gk!gk! l=1 fk(l)  0     0  m3+g3+g3−1 g1−1 g1−1 0 Y Y ˜ Y ˜ α(¯g, g¯ ;m ¯ ) =  b(k)  b(m3)  b(m3) , k=0 k=0 k=0 the equilibrium distribution at V = {(n1, n2, n3): n1 + n2 + n3 = M, n1 ≥ 0, n2 ≥ 0, n3 ≥ 0} is given by

n3−1 ! 3 nk c ! π(¯n) = B Y b(k) Y Y k . k=0 k=1 l=1 fk(l) N where {ci}i=1 is a solution of (3.4). This example is closely related to the discrete-time queueing networks discussed in [65], [81]. In these references the service-rate at a station depends only on the number of customers at that station while blocking ˜ g0 is excluded. Therefore, the delay-factors b(m3) 1 cannot be modelled in these references.

3.3 Clustering processes Suppose there exists a countable number of possible cluster-types, labelled r = 1, 2,.... The type of a clusters may represent the size of the cluster 208 Strong reversibility as in Section 2, but it may also represent the total structure, or some particular characteristic of a cluster, for example diﬀerent isomers. For simplicity, the following three types of transitions are allowed only:

– two clusters may associate to form a single cluster;

– one cluster may dissociate to form two clusters;

– a cluster may change type.

Each of these reactions introduces a diﬀerent function ψ(·). Example 3.3.1 reformulates the general model of Section 2 in terms of clustering processes. Example 3.3.2 discusses a special type of clustering processes, viz. polymerization processes. These processes clarify the assumptions on the concentration equations (2.5). Example 3.3.3 generalizes polymerization processes and Examples 3.3.4 and 3.3.5 investigate blocking-phenomena and general state-dependent reaction rates.

3.3.1 Basic model

The state of the process is denoted byn ¯ = (n1, n2,...), where ni is the number of clusters of type i, i = 1, 2,..., and if not stated otherwise, V = {n¯ : nk ≥ 0, k = 1, 2,...}. Since three types of transition are possible only, a transitionn ¯ → n¯0 uniquely determines the clusters involved in this transition. Therefore, for clustering processes strong reversibility reduces to reversibility. If two clusters associate, a transitionm ¯ +er +es → m¯ +eu occurs, and if a cluster dissociates the reversed transition occurs. If a clusters changes its type a transitionm ¯ + er → m¯ + es occurs. The transition rates for a clustering process are

ψ1(r, s, u;m ¯ ) q(m ¯ + er + es, m¯ + eu) = λrsu , φ(m ¯ + er + es)

ψ2(r, s, u;m ¯ ) q(m ¯ + eu, m¯ + er + es) = µrsu , φ(m ¯ + eu)

ψ3(r, s;m ¯ ) q(m ¯ + er, m¯ + es) = κrs . φ(m ¯ + er) The concentration equations (2.5) for clustering processes have the form

crcsλrsu = cuµrsu, crκrs = csκsr (3.6) 8.3 Applications: clustering processes 209

The symmetry-property at V takes the form

ψ1(r, s, u;m ¯ ) = ψ2(r, s, u;m ¯ ), ψ3(r, s;m ¯ ) = ψ3(s, r;m ¯ ). (3.7)

If a positive solution exists to (3.6), and (3.7) is satisﬁed then the equilibrium distribution at V is

Y nk π(¯n) = Bφ(¯n) ck . (3.8) k

3.3.2 Polymerization processes The best developed applications of clustering processes are those of polymer chemistry. In these processes one considers clusters which are build up by the formation of bonds between basic chemical building-blocks. The process is conﬁned to a region of volume R and supposed to be aggregated over space, which deletes the notion of distance between clusters. First consider the deterministic description of the model. The concentration cr of clusters of type r is deﬁned as n c = r . (3.9) r R

The reaction rate Λα of reaction α, deﬁned as the expected number of occurrences of reaction α per unit-time and unit-volume, is a function of the concentrations. In polymerization processes it is assumed that the reaction-rates are proportional to the concentrations [28], [29], [31], [87]. For association of an r-type and an s-type to a u-type, Λrsu = λrsucrcs. This is justiﬁed by noting that reactions occur due to collisions of clusters. The number of collisions is proportional to the number of clusters in the region. Therefore, with λrsu the “probability” that a reaction occurs if an r-type and an s-type cluster collide, Λrsu gives the total reaction-rate for the association of an r-type and an s-type cluster. For dissociation of a 0 u-type in an r-type and an s-type, Λrsu = µrsucu. The total reaction-rate for the dissociation of a u-type is proportional to the number of u-type clusters in the system. With µrsu the rate at which a single u-type cluster dissociates, the total rate at which u-type clusters dissociate into r-type 0 and s-type clusters is given by Λrsu. For a type-changing reaction, the reaction-rate is proportional to the number of clusters. With κuv the rate at which a single cluster of type u changes its type into a cluster of type v, the reaction-rate for type-changing reaction of a u-type in a v-type is Λuv = κuvcu. In the deterministic description the reaction rates represent 210 Strong reversibility the actual number of transitions, so detailed balance in the deterministic description is stated by

0 Λrsu = Λrsu, Λrs = Λsr, i.e. by (3.6). In contrast with queueing networks, for processes originating from physics or chemistry, the detailed balance or concentration equations are assumed to have a solution. This assumption represents deterministic equilibrium. Therefore, the parameters λ, µ, κ, representing the reaction speeds, are chosen such that a solution to the concentration equations (3.6) is possible. As an illustration, consider clustering processes in which type- changing is not allowed, i.e. κrs = 0. For these processes, based on the structure of the clusters in consideration, first an assumption about the aggregation coefficients λrsu is imposed. Then, from the concentration equations the corresponding fragmentation coefficients µrsu and the concentrations are obtained. For polymerization processes where the type represents the number of basic units in the cluster this method is used in [28], [31], [77]. There are many ways to stochasticize the deterministic model with reaction speeds λ, µ, κ. The most natural way is to assume that the transition rates are proportional to the concentrations so that (3.9) is substituted in the reaction rates Λ. This gives (m + 1 + δ )(m + 1) q(m ¯ + e + e , m¯ + e ) = λ r rs s , r s u rsu R

q(m ¯ + eu, m¯ + er + es) = µrsu(mu + 1),

q(m ¯ + er, m¯ + es) = κrs(mr + 1). These rates can be written in the form (3.3) by setting

nk R φ(¯n) = Y Y , k l=1 l

ψ1(r, s, u;m ¯ ) = ψ2(r, s, u;m ¯ ) = Rφ(m ¯ ), ψ3(r, s;m ¯ ) = Rφ(m ¯ ). Since ψ(·) satisﬁes the symmetry-property (3.7), and by construction a positive solution exists to the concentration equations (3.6), the equilibrium distribution at V is

(Rc )nk π(¯n) = B Y k . k nk! 8.3 Applications: clustering processes 211

The results above for transition rates and equilibrium distribution are well-known and can for example be found in [55], [87]. The results of Examples 3.3.3–3.3.5 below appear to be new in the literature on clustering processes.

3.3.3 Generalized polymerization processes In applications from polymer chemistry it seems natural to assume that the concentrations can be measured. Then from the equilibrium value for the concentrations in very large systems the coeﬃcients λ, µ, κ can be determined such that the deterministic detailed balance condition is ful- ﬁlled. From this observation it is natural to assume that the deterministic model always has a solution. In stochasticizing the deterministic model there is much freedom. For chemical processes the stochastic process should obey the laws of statistical mechanics. Therefore, the equilibrium distribution should have the form ([50], [87])

(Rc )nk π(¯n) = BΦ(¯n) Y k , n¯ ∈ V. (3.10) k nk!

n The product Q (Rck) k represents the zeroth-order model, where all clus- k nk! ters are assumed to be independent, that is the linear model discussed in Example 3.3.2 above. For the generalized polymerization process, Φ(¯n) represents the configurational energy, for example the potential energy of the configuration. Thus Φ(·) gives the influence of interaction between clusters on the equilibrium distribution, and (3.10) represents a first-order model. The sequel investigates underlying clustering processes indeed ex- hibiting this form. Different configuration factors and blocking functions can be included in the transition rates without affecting (3.10). This observation appears to be new in the literature. Comparison of (3.8) and (3.10) immediately implies that φ(·) must be chosen as nk R φ(¯n) = Φ(¯n) Y Y , (3.11) k l=1 l and from Theorem 2.4 it follows that for the process to be reversible the transition rates for this process must be of the form (3.3). The transition 212 Strong reversibility rates can be chosen proportional to the number of clusters. This gives

m Y Yk R ψi(r, s, u;m ¯ ) = bi(r, s, u;m ¯ )R , i = 1, 2, k l=1 l m (3.12) Y Yk R ψ3(r, s;m ¯ ) = b(r, s;m ¯ )R , k l=1 l for arbitrary functions b(·) such that b1(r, s, u;m ¯ ) = b2(r, s, u;m ¯ ) at V , and b(r, s;m ¯ ) a symmetrical function of r, s at V . These functions b(·) can represent diﬀerent types of blocking. Two examples are 1-0 blocking and conﬁguration dependence as is illustrated below. The explicit form for ψi(·), i = 1, 2, 3, is stated merely for elegance. With this form the transition rates remain proportional to the concentrations, so it can immediately be seen that the process is a stochasticized version of a polymerization process. The explicit form is by no means necessary as is discussed in Theorem 2.7.

3.3.4 1-0 blocking This section gives three blocking examples for closed clustering processes based on the transition rates specified in Example 3.3.3. A clustering process is called closed when clusters cannot enter or leave the system. This is the same definition as in the theory of queueing networks. Note, however, that the total number of clusters in a closed clustering process is not constant since cluster may associate and dissociate. P First assume that only N clusters are allowed, i.e. k mk ≤ N. This might occur in clustering processes confined to a volume R when all clusters occupy a fixed volume R/N. This may, for example, occur in a gas. In type-changing transitions the total number of clusters is unchanged, therefore set b(r, s;m ¯ ) ≡ 1. In association transitions the total number of clusters is reduced, so also in this case set b1(r, s, u;m ¯ ) ≡ 1. In dissociation transitions the total number of clusters is increased by 1 in each transition, therefore set

( 1, if P v + 1 ≤ N, b (r, s, u;m ¯ ) = k k 2 0, elsewhere,

P prohibiting the number of clusters to exceed N. At V = {n¯ : k nk ≤ N} the symmetry-property is satisﬁed since b1(·) = b2(·). Furthermore, the 8.3 Applications: clustering processes 213 process cannot leave V , therefore the equilibrium distribution at V is given by (3.10). Now assume that the number of clusters of type k is bounded, say nk ≤ Nk. This might appear when the total energy of the system is bounded and the formation of clusters of type k absorbs extreme amounts of energy. In this case type-changing transitions are blocked too. Deﬁne

 1, if m + δ ≤ N ,  k rk k b(r, s;m ¯ ) = 1, if mk + δsk ≤ Nk,  0, elsewhere,

( 1, if m + δ ≤ N , b (r, s, u;m ¯ ) = k uk k 1 0, elsewhere, ( 1, if m + δ + δ ≤ N , b (r, s, u;m ¯ ) = k kr ks k 2 0, elsewhere, then b1(·) = b2(·) and b(r, s, m¯ ) = b(s, r;m ¯ ) at V = {n¯ : nk ≤ Nk}, so the symmetry-property is valid. The equilibrium distribution at V is given by (3.10). In the examples above the total number of basic chemical building blocks is unbounded. In the case often considered in the physical literature on clustering processes the number of building blocks is ﬁxed (cf. [29], [59]). A total number, M, of identical building blocks is present and a cluster type represents the size of a cluster. In this case M diﬀerent types of cluster are possible since a cluster of type M contains all building blocks. As the total number of building blocks contained in clusters of size k is knk, the following relation forn ¯ must be valid

M X knk = M. k=1 Deﬁne b(r, s;m ¯ ) = 0, ( 1, if u = r + s, PM m + u = M, b (r, s, u;m ¯ ) = k=1 k i = 1, 2, i 0, otherwise, PM then the equilibrium distribution at V = {n¯ = (n1, . . . , nM ): k=1 knk = M} is given by (3.10). 214 Strong reversibility

3.3.5 Conﬁguration factors; potentials; dual process Besides blocking phenomena the functions b(·) appearing in (3.12) can be used to model transition rates depending on various conﬁgurations. This shows that the duality results of Chapter 5 can be extended to more general processes. To this end, consider the following four cases, where b1(·) ≡ b2(·): b (r, s, u;m ¯ ) = 1, (T1) 1 b(r, s;m ¯ ) = 1;

b (r, s, u;m ¯ ) = Ψ(m ¯ ), (T2) 1 b(r, s;m ¯ ) = Ψ(m ¯ );

b (r, s, u;m ¯ ) = Φ(m ¯ + e + e )Φ(m ¯ + e ), (T3) 1 r s u b(r, s;m ¯ ) = Φ(m ¯ + er)Φ(m ¯ + es); Φ(m ¯ + e + e )Φ(m ¯ + e ) b (r, s, u;m ¯ ) = r s u , 1 Ψ(m ¯ ) (T4) Φ(m ¯ + e )Φ(m ¯ + e ) b(r, s;m ¯ ) = r s . Ψ(m ¯ ) From (3.11) and (3.12), deleting the linear terms from the transition rates, 0 that is simply ignore the zeroth-order contribution, for λ(¯g, g¯ ) = λrsu, if 0 0 0 0 g¯ = er+es,g ¯ = eu, λ(¯g, g¯ ) = µrsu, ifg ¯ = eu,g ¯ = er+es, and λ(¯g, g¯ ) = κrs 0 ifg ¯ = er,g ¯ = es, the transition rates are 1 (T1) q(m ¯ +g, ¯ m¯ +g ¯0) ∝ λ(¯g, g¯0) , Φ(m ¯ +g ¯) Ψ(m ¯ ) (T2) q(m ¯ +g, ¯ m¯ +g ¯0) ∝ λ(¯g, g¯0) , Φ(m ¯ +g ¯) (T3) q(m ¯ +g, ¯ m¯ +g ¯0) ∝ λ(¯g, g¯0)Φ(m ¯ +g ¯0), Φ(m ¯ +g ¯0) (T4) q(m ¯ +g, ¯ m¯ +g ¯0) ∝ λ(¯g, g¯0) , Ψ(m ¯ ) where Φ(¯n) is the configuration-potential of staten ¯, and Ψ(m ¯ ) is the configuration-potential of statem ¯ , the state of non-moving clusters in a transition, i.e.m ¯ is a dual state as is described in Chapter 5. As is 8.3 Applications: clustering processes 215 discussed in Chapter 5, the equilibrium distribution (3.10) can be written as " # X π(¯n) = B exp −U(¯n) − ζknk , k where ζk = − log ck are site-potentials (cf. Potential-interpretation 5.2.4). This observation is valid since the process is reversible and routing reversible [87]. The transition rates can be written in terms of the configuration potential. This gives

(T1) q(m ¯ +g, ¯ m¯ +g ¯0) ∝ exp [U(m ¯ +g ¯)],

(T2) q(m ¯ +g, ¯ m¯ +g ¯0) ∝ exp [U(m ¯ +g ¯) − V (m ¯ )],

(T3) q(m ¯ +g, ¯ m¯ +g ¯0) ∝ exp [−U(m ¯ +g ¯0)],

(T4) q(m ¯ +g, ¯ m¯ +g ¯0) ∝ exp [V (m ¯ ) − U(m ¯ +g ¯0)] .

From this observation it is justified to state that the (T1)-type process is dual to the (T3)-type process and that the (T2)-type process is dual to the (T4)-type process in the sense that the transitions occur due to an inverted potential law. This immediately suggests that the results of Chapter 5 can be extended to processes with multiple changes. In addition it shows that for totally different forms of the transition rates with totally different physical interpretation the process has the same equilibrium distribution (3.10). This observation has not been made in the literature. In contrast, one usually takes proportional transition rates as in Example 3.3.2 or (T2)-type transition rates (cf. [29], [55], [59], [87]). 216 Strong reversibility Chapter 9

Group-local-balance

1 Introduction

Queueing networks with single changes have been extensively studied and are shown to have a product-form equilibrium distribution (cf. Sec- tion 2.4). Typical present-day applications of queueing networks, however, feature simultaneous transitions of multiple customers. For example, in manufacturing, parts are often processed and transported in batches, in parallel programming a number of modules can be initiated and run at the same time, and in digitized or time-slotted communication networks mes- sages or packets are simultaneously transmitted and released at discrete times. The class of product-form queueing networks has been extended to include discrete-time structures, simultaneous service-completions, and batch routing. In [23] product-form results are derived for open discrete- time queueing networks and in [73] for a closed queueing model of a computer system. These references use the so-called doubly stochastic service-discipline and allow generally distributed service requirements at the stations. However, it is assumed in these references that at any station no more than one customer can either arrive or leave at the same time. Discrete-time queueing networks with simultaneous service-completions at the stations and batch routing are studied in [64], [65], [66], and [81]. In these references the service-processes at diﬀerent stations of the queueing network are assumed to be independent and, in addition, customers route among the stations of the queueing network independent of the other routing customers and independent of the state of the queueing network. In a recent paper [40] these results were generalized to closed queueing

217 218 Group-local-balance networks with a general system-dependent batch service-function, while customers route independently. Herein, the concept of balance per group is introduced as the responsible factor for product-form results. In [41] an extension of these results is given to mixed open and closed networks with batch customer routing. These references do not include blocking. In Chapter 8 a related framework is discussed in which, under the restriction of a reversibility assumption, not only state-dependent batch service-rates, but also state-dependent batch routing probabilities and blocking of transitions are included. The present chapter extends the results of Chapter 8 to queueing networks with general state-dependent simultaneous service- completions and batch routing such as most notably modelling blocking phenomena also when a non-reversible batch routing structure is involved. The outline of this chapter is as follows. Section 2 introduces group- local-balance and local state spaces and gives the first main result stating that the stationary distribution of the queueing network can be computed by local solutions if group-local-balance holds. Section 3 presents the second main result of this chapter: a decomposition theorem by which the service-characteristics and the routing-characteristics of the queueing network can be analysed separately. In Sections 4 and 5 examples are given for the service-characteristics and the routing characteristics such that from the decomposition theorem the equilibrium distribution of the queueing network can be concluded by merely combining a service-form example from Section 4 with a routing form example from Section 5. Section 6 briefly discusses dual processes as introduced in Chapter 5 in the setting of this chapter, that is for batch routing queueing networks. Finally, Sec- tion 7 presents some specific examples. In particular, discrete-time queueing networks are discussed and compared to batch routing continuous-time queueing networks, and examples of continuous-time batch routing queueing networks with blocking are discussed.

2 Model

Consider a continuous-time queueing network consisting of N stations, labelled 1, 2,...,N, in which a single type of customers routes among the stations. Assume that the queueing network can be represented by a N stable, regular, continuous-time Markov chain with state space S ⊂ IN0 . A staten ¯ = (n1, . . . , nN ) is a vector with components ni, i = 1,...,N, where ni denotes the number of customers at station i, i = 1,...,N. The transition rate from staten ¯ to staten ¯0 is denoted by q(¯n, n¯0). A transition 9.2 Model 219

0 n¯ → n¯ may occur due to a group of customersg ¯ = (g1, . . . , gN ) that 0 0 0 leaves the stations, a group of customersg ¯ = (g1, . . . , gN ) that enters the stations, while the customersm ¯ = (m1, . . . , mN ) remain at the stations. The transition rate for this particular transition is denoted by q(¯g, g¯0;m ¯ ). As is discussed in Section 8.2, the transitionn ¯ → n¯0 may occur due to other groups entering and departing the stations too. Therefore, the total transition rate from staten ¯ to staten ¯0 is given in (8.2.1). Note that when the vectorm ¯ is fixed, then in a transition from staten ¯ to staten ¯0 the routing groupsg ¯ andg ¯0 are completely determined. Both open and closed queueing networks are included in the framework PN PN 0 presented in this chapter. When i=1 gi > i=1 gi customers depart from PN PN 0 the system and when i=1 gi < i=1 gi customers arrive at the system. PN PN 0 Note that for an open queueing network also in the case i=1 gi = i=1 gi it may be that customers enter and leave the system. In this particular transition the total number of customers in the queueing network is unchanged. Transitions in which a number of customers depart and enter the same 0 station are not excluded, that is transitions with gi > 0 and gi > 0 are allowed. Also, forg ¯ 6= 0¯ it may be that q(¯g, g¯;m ¯ ) 6= 0, a transition referred to as a dummy transition. For single changes queueing networks 0 transitions in which both gi > 0 and gi > 0 are usually excluded. This simplifies the Markov chain. However, if batch routing is allowed, such transitions cannot be excluded from the model as is illustrated below in 0 Example 2.1. Note that transitions in which both gi > 0 and gi > 0 may occur in the model of Chapter 8 too. However, due to the reversible structure of the process discussed is Chapter 8, these transitions do not affect the behaviour to the Markov chain. In the model of the present chapter, these transitions do influence both the routing structure of the Markov chain and the behaviour of the Markov chain when transitions are blocked.

Example 2.1 (Routing structure) Consider the closed three station queueing network depicted in Figure 9.1. Assume that in the initial state there are 4 customers at station 1, 3 customers at station 2 and 3 customers at station 3, that isn ¯ = (4, 3, 3). If 2 customers leave station 3 then they must route to station 1. For this particular transition the states and routing vectors are 220 Group-local-balance

1 --2 3

Figure 9.1. Cyclic three-station queueing network

n¯ = (4, 3, 3)m ¯ = (4, 3, 1)n ¯0 = (6, 3, 1) g¯ = (0, 0, 2)g ¯0 = (2, 0, 0)

Now consider a transition in which 2 customers leave station 1 and 2 customers leave station 2. Then

n¯ = (4, 3, 3)m ¯ = (2, 1, 3)n ¯0 = (2, 3, 5) g¯ = (2, 2, 0)g ¯0 = (0, 2, 2)

0 In this transition n2 = n2 = 3 although there are customers leaving and entering station 2. In fact it seems to be that the 2 routing customers route from station 1 to station 3 without visiting station 2. This turns out to be an important feature of batch routing queueing networks as it will inﬂuence blocking protocols (see Section 7). A transition in which, for example, 2 customers leave station 1, 2 customers leave station 2 and 2 customers leave station 3 is a dummy transition. In this case

n¯ = (4, 3, 3)m ¯ = (2, 1, 1)n ¯0 = (4, 3, 3) g¯ = (2, 2, 2)g ¯0 = (2, 2, 2)

Although there are customers routing among the stations, the state of the queueing network does not change. This is a feature of batch routing models only. Note that since the state of the network does not change in a dummy transition, these transitions need not be included in the balance equations. However, dummy transitions may inﬂuence the behaviour of the queueing network. 2

Assume that the Markov chain is irreducible for a set V ⊂ S and that there exists a unique equilibrium distribution π at V . Then the equilibrium distribution can be obtained as the unique solution to the global balance equations, that is π satisﬁes for alln ¯ ∈ V

π(¯n) X q(¯n, n¯0) = X π(¯n0)q(¯n0, n¯). (2.1) n¯06=¯n n¯06=¯n 9.2 Model 221

Inserting (8.2.1) into (2.1) and rearranging summations yields that a distribution π at V is the unique equilibrium distribution if for alln ¯ ∈ V

π(¯n) X X q(¯g, g¯0;m ¯ ) = X X π(m ¯ +g ¯0)q(¯g0, g¯;m ¯ ). {g,¯ m¯ :m ¯ +¯g=¯n} g¯06=¯g {g,¯ m¯ :m ¯ +¯g=¯n} g¯06=¯g A distribution satisfying these equations for the inner summations only, that is a distribution satisfying (2.2), is the unique equilibrium distribution. In general (2.2) is more restrictive than (2.1) and does not need to have a solution. The equations (2.2) are the basis for the analysis of batch routing queueing networks presented in this chapter and are shown to be necessary and sufficient for the queueing network to have a product-form equilibrium distribution. Definition 2.2 (Group-local-balance property) A distribution p at an irreducible set V satisfies group-local-balance (GLB) at V if for all m¯ and for all m¯ +g ¯ ∈ V

p(m ¯ +g ¯) X q(¯g, g¯0;m ¯ ) = X p(m ¯ +g ¯0)q(¯g0, g¯;m ¯ ). (2.2) g¯06=¯g g¯06=¯g A Markov chain has the GLB-property if the equilibrium distribution satisfies (2.2). The definition of group-local-balance is a generalization of notions as local balance or job-local-balance for the case of single changes (i.e. only one customer is allowed to leave a station simultaneously) to batch routing queueing networks. The approach followed from here on is a generalization to multiple changes of the approach followed in [46]. However, note that the results obtained in this chapter for batch routing queueing networks cannot be conclude from the results of this reference, since multiple changes are excluded from the formulation given in [46]. In this reference the assumption that a single “jobmark” can change in a transition is not satisfied in this chapter. As a consequence also the insensitivity results obtained in [46] cannot be concluded from the results of this chapter. As group-local-balance is the basis of the analysis presented in this chapter, first the feasibility of (2.2) is discussed. For fixedm ¯ let V (m ¯ ) be the local state space of the Markov chain with transition rates q(¯g, g¯0;m ¯ ) restricted to V , and let Vi(m ¯ ), i = 1, . . . , k(m ¯ ), denote the local irreducible sets in V (m ¯ ) with respect to the Markov chain with transition rates q(¯g, g¯0;m ¯ ) for fixedm ¯ . Note that a staten ¯ may be an element of different local state spaces V (m ¯ ). This allows transitions from one local 222 Group-local-balance state space to another since in general V (m ¯ )∩V (m ¯ 0) 6= ∅ form ¯ 6=m ¯ 0. The following example illustrates the local state spaces and local irreducible sets, and shows that it is not uncommon that V (m ¯ ) consists of multiple local irreducible sets.

Example 2.3 (GLB and local irreducible sets) Consider the three station queueing network of Figure 9.1 in which 10 customers are present. Ifm ¯ = (4, 3, 1) there are 2 customers routing among the stations. If all transitions are possible the local state space is

V (4, 3, 1) = {(5, 4, 1), (4, 4, 2), (5, 3, 2), (6, 3, 1), (4, 5, 1), (4, 3, 3)}.

In the first three states there is a customer departing from two different stations. In the remaining three states two customers depart from the same station. The queueing network is cyclic. As a consequence the local state space consists of two irreducible sets, one local irreducible set for two customers departing from different stations and one local irreducible set for two customers departing from the same station

V1(4, 3, 1) = {(5, 4, 1), (4, 4, 2), (5, 3, 2)}

V2(4, 3, 1) = {(6, 3, 1), (4, 5, 1), (4, 3, 3)}.

As will be seen in Section 7 the notion of diﬀerent irreducible sets Vi is important when blocking occurs. 2

The previous example shows that V (m ¯ ) may consist of multiple irreducible sets. The following lemma shows that V (m ¯ ) consists only of irreducible sets if the Markov chain satisﬁes GLB.

Lemma 2.4 If the equilibrium distribution π satisfies GLB, then for any m¯ it must be that k(m ¯ ) [ V (m ¯ ) = Vi(m ¯ ). (2.3) i=1 Proof If V (m ¯ ) = ∅ then (2.3) is trivially fulfilled. If V (m ¯ ) 6= ∅ it is sufficient to prove that the Markov chain at V (m ¯ ), for fixedm ¯ , has no transient states. By virtue of the GLB-property this local Markov chain has an equilibrium distribution cπ where

c−1 = X π(m ¯ +g ¯). {g¯:m ¯ +¯g∈V } 9.2 Model 223

As π(·) > 0 each state is positive-recurrent. 2

From Lemma 2.4 it follows that, if GLB holds, then for any ﬁxedm ¯ for which V (m ¯ ) 6= ∅ and i ∈ {1, . . . , k(m ¯ )} the following set of equations has a unique positive solution up to a constant factor:

X 0 X 0 0 x(¯g;m ¯ ) q(¯g, g¯ ;m ¯ ) = x(¯g ;m ¯ )q(¯g , g¯;m ¯ ), m¯ +g ¯ ∈ Vi(m ¯ ). (2.4) g¯06=¯g g¯06=¯g

If V (m ¯ ) 6= ∅, but q(¯g, g¯0;m ¯ ) = 0 for allg, ¯ g¯0, then V (m ¯ ) consists of singletons only, that is for i = 1, . . . , k(m ¯ ) the local irreducible sets consist of a single state only. In this case relation (2.4) is trivially fulfilled. In order to satisfy GLB for processes with general transition rates, assume that for any fixed m¯ for which V (m ¯ ) 6= ∅ relation (2.3) holds and for i ∈ {1, . . . , k(m ¯ )} the system (2.4) has a unique positive solution {x(¯g;m ¯ )|m¯ +g ¯ ∈ Vi(m ¯ )} up to a constant factor. This assumption is justified by Lemma 2.4. The equations (2.4) can often be solved explicitly. Therefore a characterization and computation of the equilibrium distribution which satisfies GLB purely in terms of these local solutions of the global balance equations will be given below. To this end, an additional process with transition ratesq ¯ is defined.

Deﬁnition 2.5 (q¯-process) A Markov chain at V with transition rates q¯ which satisfy the following relations is called a q¯-process.

 0  For any m¯ , i = 1, . . . , k(m ¯ ), and m¯ +g, ¯ m¯ +g ¯ ∈ Vi(m ¯ )   0 0  q¯(¯g, g¯ ;m ¯ ) x(¯g ;m ¯ )  = , q¯(¯g0, g¯;m ¯ ) x(¯g;m ¯ ) (2.5)    and otherwise   q¯(¯g, g¯0;m ¯ ) = 0.

Note that the transition ratesq ¯ are uniquely defined by (2.5) up to a common factor at each of the local irreducible sets Vi(m ¯ ). The quotients of theq ¯ are unique. As will be apparent below those quotients are used in the theory only. Further note that also for theq ¯-process a pathn ¯0 → n¯1 → · · · → n¯i−1 → n¯i → n¯, wheren ¯0, n¯ are elements of different local state spaces is possible. This is a direct consequence of the obvious fact that a staten ¯ may be an element of different local state spaces. 224 Group-local-balance

In Definition 2.5 theq ¯-process is defined in terms of the local solutions x(¯g;m ¯ ). These local solutions may be a function ofg ¯ andm ¯ separately. As a consequence, theq ¯-process is locally reversible, that is theq ¯-process is reversible at the local state spaces V (m ¯ ). The following definition of strong reversibility relates the local solutions to a global solutionπ ¯(m ¯ +g ¯) of theq ¯-process. Definition 2.6 (Strong reversibility) The q¯-process is strongly reversible at V if for all m¯ for which V (m ¯ ) 6= ∅ and i ∈ {1, . . . , k(m ¯ )} the equilibrium distribution π¯ satisfies

0 0 0 0 π¯(m ¯ +g ¯)¯q(¯g, g¯ ;m ¯ ) =π ¯(m ¯ +g ¯ )¯q(¯g , g¯;m ¯ ), m¯ +g, ¯ m¯ +g ¯ ∈ Vi(m ¯ ). (2.6) Since the transition rates of theq ¯-process are non-null at the local irreducible sets only, strong reversibility is defined at the local irreducible sets. Note, however, that Definition 2.6 for strong reversibility is the same as Definition 8.2.3. Therefore, the results of Chapter 8 may now be used to analyse theq ¯-process and Chapter 8 gives necessary and sufficient conditions forπ ¯ to be of product-form. The following theorem states the main result of this section and relates the equilibrium distribution of the original Markov chain to the equilibrium distribution of theq ¯-process. This theorem allows using the results for strongly reversible processes to conclude a product-form equilibrium distribution for non-reversible processes. Theorem 2.7 The equilibrium distribution π of a Markov chain at V satisfies GLB if and only if the q¯-process is strongly reversible at V. Moreover, with π¯ its equilibrium distribution, for all n¯ ∈ V

π(¯n) =π ¯(¯n). (2.7)

Proof If theq ¯-process is strongly reversible then (2.6) and (2.5) imply that π¯(m ¯ +g ¯) q¯(¯g0, g¯;m ¯ ) x(¯g;m ¯ ) = = . π¯(m ¯ +g ¯0) q¯(¯g, g¯0;m ¯ ) x(¯g0;m ¯ ) Hence, the distribution π(·) =π ¯(·) satisfies (2.4) which shows that π satisfies GLB. Conversely, if π satisfies GLB then π is a solution of (2.4). Then, recalling the assumption that (2.4) has a unique solution gives x(¯g;m ¯ ) π(m ¯ +g ¯) = , x(¯g0;m ¯ ) π(m ¯ +g ¯0) 9.2 Model 225

0 so that by (2.5) for any i ∈ {1, . . . , k(m ¯ )}, andm ¯ +g, ¯ m¯ +g ¯ ∈ Vi(m ¯ ) q¯(¯g, g¯0;m ¯ ) π(m ¯ +g ¯) = . (2.8) q¯(¯g0, g¯;m ¯ ) π(m ¯ +g ¯0) Asq ¯(¯g, g¯0;m ¯ ) =q ¯(¯g0, g¯;m ¯ ) = 0 ifm ¯ +g, ¯ m¯ +g ¯0 are not contained in the same local irreducible set Vi(m ¯ ), strong reversibility (2.6) implies that the q¯-process is strongly reversible at V withπ ¯ = π. 2

The following Corollary 2.8 shows the importance of the result of The- orem 2.7. From Corollary 2.8 GLB can be checked, and the equilibrium distribution can be concluded based upon merely the local solution x(¯g;m ¯ ) to (2.6). This solution can often be obtained explicitly. Corollary 2.8 The equilibrium distribution π satisﬁes GLB if and only if for arbitrary reference state n¯0, and all n¯ ∈ V

p 0 Y q¯(¯gk, g¯k;m ¯ k) 0 = cπ(¯n), (2.9) k=0 q¯(¯gk, g¯k;m ¯ k) for all paths of the form

0 0 n¯0 =m ¯ 0 +g ¯0 → m¯ 0 +g ¯0 =m ¯ 1 +g ¯1 → m¯ 1 +g ¯1 = ··· 0 (2.10) ··· =m ¯ p +g ¯p → m¯ p +g ¯p =m ¯ p+1 +g ¯p+1 =n, ¯ for which the denominator in (2.9) is positive, and with arbitrary p ∈ IN, −1 and c = π(¯n0) denoting a normalizing constant. Proof The proof below is very similar to that of Kolmogorov’s criterion 2.3.2 for reversibility, but since Kolmogorov’s criterion is proven for reversible processes only, for completeness, the proof is given here for strongly reversible processes. By virtue of Theorem 2.7 it is suﬃcient to prove strong reversibility for theq ¯-process. To this end, consider some arbitrary i, g,¯ g¯0, m¯ such that 0 0 n¯ =m ¯ +g, ¯ n¯ =m ¯ +g ¯ ∈ Vi(m ¯ ), and an arbitrary reference staten ¯0. By the irreducibility of V there exists a pathn ¯0 → n¯ which is of the form 0 stated above in (2.10). Then, by virtue of (2.5) there is a pathn ¯0 → n¯ of 0 0 the formn ¯0 → n¯ =m ¯ +g ¯ → n¯ =m ¯ +g ¯ such that the denominator in the next expressions, as according to (2.9), is positive

0 p 0 ! 0 π(¯n ) Y q¯(¯gk, g¯k;m ¯ k) q¯(¯g, g¯ ;m ¯ ) = 0 0 , π(¯n0) k=0 q¯(¯gk, g¯k;m ¯ k) q¯(¯g , g¯;m ¯ ) 226 Group-local-balance

p 0 ! π(¯n) Y q¯(¯gk, g¯k;m ¯ k) = 0 . π(¯n0) k=0 q¯(¯gk, g¯k;m ¯ k) As a consequence,

π(m ¯ +g ¯)¯q(¯g, g¯0;m ¯ ) = π(m ¯ +g ¯0)¯q(¯g0, g¯;m ¯ ), implying that theq ¯-process is strongly reversible withπ ¯ = π. If π satisﬁes GLB then by Theorem 2.7 theq ¯-process is strongly reversible withπ ¯ = π. Now consider some path of the given form, then from (2.6)

p 0 p 0 p Y q¯(¯gk, g¯k;m ¯ k) Y π¯(m ¯ k +g ¯k) Y π(m ¯ k+1 +g ¯k+1) π(¯n) 0 = = = . k=0 q¯(¯gk, g¯k;m ¯ k) k=0 π¯(m ¯ k +g ¯k) k=0 π(m ¯ k +g ¯k) π(¯n0) 2

Corollary 2.8 is the generalization of Kolmogorov’s criterion 2.3.2 to strongly reversible processes. Note that Corollary 2.8 can be concluded for strongly reversible processes in Chapter 8 too. However, in Chapter 8 emphasis is on conditions that are necessary and sufficient for a product- form equilibrium distribution, which can be obtained directly from the transition rates of the Markov chain. Kolmogorov’s criterion does not give conditions for product-form. It merely gives a characterization of (strong) reversibility. Note that strong reversibility can be characterized by a result similar to Kolmogorov’s criterion 2.3.1 too. Thus strong reversibility can be characterized by the transition structure of theq ¯-process. To this end, observe that the transitions in each quotient of (2.9) occur inside a local irreducible set, but that from one quotient to another the process may move from one local irreducible set to another. Further note that, similar to Kolmogorov’s criterion 2.3.2, the left-hand side of (2.9) necessarily has to be independent of the path. As the local solutions can often be obtained explicitly, the invariance condition (2.9) can in principle be checked by enumerating all possible trajectories. In various practical situations, similar to the experience with Kolmogorov’s criterion 2.3.2, it turns out that only a small number of “basic” trajectories or cycles need to be checked as based upon regular structures of the underlying Markov chain. Even more efficient, the local solutions often suggest a form for π(·) as based upon the required ra- tios (2.8). By simply checking (2.2) or (2.4) for this suggested form, GLB and an explicit form of π(·) can so be verified directly. The examples in Section 7 are verified directly in this way. 9.3 Decomposition theorem 227

If x(¯g;m ¯ ) is a solution of (2.4) then it is a solution of a subset of (2.4) too. Therefore, if blocking of transitions comes down to the exclusion of one or more equations of (2.4), the obvious procedure to obtain a solution to (2.4) is to first try to find a π that satisfies (2.2) as based upon the equations (2.4) for the system without blocking, and then conclude that the same solution x(¯g;m ¯ ) remains valid also for the system with blocking. Conversely, any solution π that satisfies the GLB equations (2.2) remains valid if an arbitrary subsets of (2.4) is excluded, i.e. if these transitions are prohibited to occur. The following three examples make this more precise. If a solution x(¯g;m ¯ ) of (2.4) where all transitions are possible is obtained, then in each of the following modifications this solution remains valid. Explicit examples will be given later on in Sections 5.4 and 7.

• For a givenm ¯ 0, set 0 q(¯g, g¯ ;m ¯ 0) = 0, 0 for allg, ¯ g¯ . Roughly speaking, the vectorm ¯ 0 is not allowed to remain unchanged if no other customers remain unchanged.

• For a giveng ¯0, set

0 q(¯g0, g¯ ;m ¯ ) = q(¯g, g¯0;m ¯ ) = 0,

0 for allg, ¯ g¯ , m¯ . Roughly speaking, a groupg ¯0 is not allowed to take part of movements.

• For a given number, say G, set

q(¯g, g¯0;m ¯ ) = 0,

PN PN 0 whenever i=1 gi = G or i=1 gi = G so that all equations in which PN PN 0 i=1(ni −mi) = G or i=1(ni −mi) = G can be removed from (2.4). In words that is, batch movements of a total of exactly G departures and/or G arrivals are not allowed.

3 Decomposition theorem

In Theorem 2.7 the information on the equilibrium distribution π is en- closed in the equilibrium distributionπ ¯ of theq ¯-process. With a more explicit form for the transition rates q(¯g, g¯0;m ¯ ) it is possible to extract explicit product-form results for π. This section considers a special form 228 Group-local-balance for the transition rates which has a natural interpretation in queueing networks. With this special form the complexity of the local equations (2.4) is reduced, which leads to factorizing results (see (3.8)) for the equilibrium distribution π with as special application product-form results. To this end, assume that the transition rates can be decomposed:

q(¯g, g¯0;m ¯ ) = f(¯g, m¯ +g ¯)p(¯g, g¯0;m ¯ ). (3.1)

Here f(·) represents the service-characteristics of the queueing network, i.e. protocols and speeds at the stations, and p(·) represents the routing characteristics, i.e. routing and blocking probabilities of customers, both up to a possible numerical factor as shown below. Note that this decomposition is not required in the general setting of Section 2. Assume that for arbitrary but given functions ψ(·) and φ(·) the service- characteristics are ψ(¯n − g¯) f(¯g, n¯) = , (3.2) φ(¯n) where the function φ(·) is assumed to be strictly positive at V , i.e. for all n¯ ∈ V , φ(¯n) > 0, and ψ(·) is an arbitrary non-negative function. The decomposition (3.1) of the transition rates is valid in many queueing networks. However, the special form (3.2) may not completely describe the service-characteristics. In some queueing networks the actual service- characteristics may have the form

ψ(¯n − g¯) fˆ(¯g, n¯) = , (3.3) φ(¯n)β(¯g) where β(¯g) is to be included to take into account diﬀerent orderings of the customers in the batchg ¯. Assume that with this service-function fˆ(·) and some routing functionp ˆ(·) the transition rates can be represented in the form q(¯g, g¯0;m ¯ ) = fˆ(¯g, m¯ +g ¯)ˆp(¯g, g¯0;m ¯ ). (3.4) By deﬁning f(¯g, n¯) = fˆ(¯g, n¯)β(¯g), pˆ(¯g, g¯0;m ¯ ) (3.5) p(¯g, g¯0;m ¯ ) = , β(¯g) (3.4) can be written as

q(¯g, g¯0;m ¯ ) = f(¯g, m¯ +g ¯)p(¯g, g¯0;m ¯ ). 9.3 Decomposition theorem 229

The functions f(·) and p(·) are thus to be seen as a service and routing function possibly up to a numerical factor. That is, a decomposition of the form (3.4) with fˆ(·) of the form (3.3) can be rewritten in the form (3.1) with f(·) of the form (3.2). If, in the examples, the service-characteristics have the form fˆ(·) then f(·) is used with the additional remark that the numerical factors β(¯g) are included in the routing-part. Ifp ˆ(·) represents the routing characteristics then p(·) is used with the additional remark that the numerical factors β(¯g) are included in the service-part. The following assumption reduces equation (2.4) and the corresponding assumption on the existence of a unique solution to (2.4) to merely the state-dependent routing equations. Assume that for any ﬁxed m¯ for which V (m ¯ ) 6= ∅ relation (2.3) holds and for i ∈ {1, . . . , k(m ¯ )} the following system has a unique positive solution {y(¯g;m ¯ )|m¯ +g ¯ ∈ Vi(m ¯ )} up to a constant factor:

X 0 X 0 0 y(¯g;m ¯ ) p(¯g, g¯ ;m ¯ ) = y(¯g ;m ¯ )p(¯g , g¯;m ¯ ), m¯ +g ¯ ∈ Vi(m ¯ ). (3.6) g¯06=¯g g¯06=¯g

By analogy with the deﬁnition of theq ¯-process, a qy-process at V is deﬁned to characterize the routing characteristics of the Markov chain at V .

Deﬁnition 3.1 (qy-process) A Markov chain at V with transition rates qy which satisfy the following relations is called a qy-process.

 0  For any m¯ , i = 1, . . . , k(m ¯ ), and m¯ +g, ¯ m¯ +g ¯ ∈ Vi(m ¯ )   0 0  qy(¯g, g¯ ;m ¯ ) y(¯g ;m ¯ )   0 = , qy(¯g , g¯;m ¯ ) y(¯g;m ¯ ) (3.7)    and otherwise   0  qy(¯g, g¯ ;m ¯ ) = 0.

The following product-form result can now be obtained from Theorem 2.7.

Theorem 3.2 (Decomposition Theorem) π satisﬁes GLB if and only if the qy-process is strongly reversible at V . Moreover with πy its equilibrium distribution, for all n¯ ∈ V and B a normalizing constant,

π(¯n) = Bφ(¯n)πy(¯n). (3.8)

Proof By virtue of Theorem 2.7, the first statement of the theorem stating that π satisfies GLB if and only if the qy-process is strongly reversible, 230 Group-local-balance can be proven by showing that theq ¯-process is strongly reversible if and only if the qy-process is strongly reversible. To this end, first note that q(¯g, g¯0;m ¯ ) = 0 if f(¯g, m¯ +g ¯) = 0, i.e. if ψ(m ¯ ) = 0. Therefore, if ψ(m ¯ ) = 0, within V (m ¯ ) there are no transitions possible, that is V (m ¯ ) consists of singletons only and (2.4) is trivially fulfilled. Now assume that ψ(m ¯ ) > 0. Then the balance equations (2.4) and (3.6) are equivalent if x(¯g;m ¯ ) has the form φ(m ¯ +g ¯) x(¯g;m ¯ ) = y(¯g;m ¯ ), (3.9) ψ(m ¯ ) for allm ¯ and i ∈ {1, . . . , k(m ¯ )} such thatm ¯ +g ¯ ∈ Vi(m ¯ ). From (3.7), (3.9), (2.5), (3.2) and substitution of the following relation for πy andπ ¯

π¯(¯n) = Bφ(¯n)πy(¯n), (3.10) it follows that 0 0 qy(¯g, g¯ ;m ¯ )πy(m ¯ +g ¯) y(¯g ;m ¯ )πy(m ¯ +g ¯) 0 0 = 0 qy(¯g , g¯;m ¯ )πy(m ¯ +g ¯ ) y(¯g;m ¯ )πy(m ¯ +g ¯ ) ψ(m ¯ ) x(¯g0;m ¯ ) π (m ¯ +g ¯) φ(m ¯ +g ¯0) y = ψ(m ¯ ) x(¯g;m ¯ ) π (m ¯ +g ¯0) φ(m ¯ +g ¯) y ψ(m ¯ ) q¯(¯g, g¯0;m ¯ ) π (m ¯ +g ¯) φ(m ¯ +g ¯0) y = ψ(m ¯ ) q¯(¯g0, g¯;m ¯ ) π (m ¯ +g ¯0) φ(m ¯ +g ¯) y 0 q¯(¯g, g¯ ;m ¯ )φ(m ¯ +g ¯)πy(m ¯ +g ¯) = 0 0 0 q¯(¯g , g¯;m ¯ )φ(m ¯ +g ¯ )πy(m ¯ +g ¯ ) q¯(¯g, g¯0;m ¯ )¯π(m ¯ +g ¯) = . q¯(¯g0, g¯;m ¯ )¯π(m ¯ +g ¯0) Thus, theq ¯-process is strongly reversible with equilibrium distributionπ ¯ if and only if the qy-process is strongly reversible with equilibrium distribution πy, whereπ ¯ and πy are related by (3.10). Theorem 2.7 or rather equation (2.7) concludes the proof. 2

For q(¯g, g¯0;m ¯ ) of the form (3.1) it is always possible to replace (2.4) by (3.6) by using a relation similar to (3.9) as a deﬁnition for y(¯g;m ¯ ): 1 x(¯g;m ¯ ) = y(¯g;m ¯ ) . f(¯g, m¯ +g ¯) 9.4 Network applications: service-form examples 231

In general, however, when f(·) does not satisfy (3.2) the equilibrium distribution will not be a product of a service and a routing factor. Note that this is up to interpretation, as the service-characteristics can be scaled to satisfy (3.2), and the remaining part can then be included in the routing characteristics. In other words, the notion of balance per group (GLB), seems to be responsible for a possible factorization of the equilibrium distribution in a part that mainly covers the service-characteristics and a part that mainly covers the routing characteristics. Similar service and routing decompositions for the single transition case (cf. [46], [76]) are hereby extended to multiple transitions. In particular, as will be illustrated in Section 4.1, a factorization to individual stations is obtained when f(·) has a factorizing form. The function ψ(m ¯ ) drops out in the proof above. This is a direct consequence of the notion of GLB since the local irreducible sets Vi(m ¯ ) are considered for each ﬁxedm ¯ so that ψ(m ¯ ) is a constant at these local irreducible sets. A similar observation is reported for the single transition case (cf. [27], [76]), and for the multiple transition case (cf. [40], [41]) as an extension of the more “standard” assumption that ψ = φ (cf. [55], [83], [87]). In both [40] and [41] a decomposition similar to (3.1) is used. The service-form (3.2) is introduced in [40], whereas the service-form (3.3) is introduced in [41]. The routing characteristics p(¯g, g¯0;m ¯ ) used here are more general than those in both [40] and [41]. In [40] the customers route independently as in Example 5.2 and in [41] customers route according to a batch routing rule p(¯g, g¯0;m ¯ ) = λ(¯g, g¯0) for allm ¯ as in Example 5.1. Therefore, in both [40] and [41] blocking-phenomena cannot be modelled. The form (3.8) for the equilibrium distribution is a direct consequence of the notion of GLB.

4 Network applications: service-form examples

This section gives some examples illustrating choices for the service-function f(·). In the ﬁrst set of examples (Section 4.1) the service-speed at a station depends on the number of customers at that station only. Various forms related to discrete-time settings are presented. In the second set of examples (Section 4.2) the stations are grouped into clusters of stations. The service-speed at a station is allowed to depend on the total number of customers within the clusters. This hints to possible generalizations of the results of Chapter 5 to batch routing queueing networks. 232 Group-local-balance

4.1 Station-dependent service-rates In the examples below f(·) has the station factorizing form

N N Y Y ψi(ni − gi) f(¯g, n¯) = fi(gi, ni) = . (4.1) i=1 i=1 φi(ni)

Here fi(·) represents the service-characteristics at station i and is allowed to depend on the number of customers present at station i only. Through- out this section the numerical factor involved is

N Y β(¯g) = gi! . i=1 In order to show the mathematical uniﬁcation of the service-form (4.1), four examples of more or less the same form but with diﬀerent substitu- tions for ψ are presented. The Examples 4.1.1 and 4.1.4 are well-known for discrete-time queueing networks (cf. [65], [81]). The Examples 4.1.2 and 4.1.3 are believed to be new. To illustrate the possible physical behaviour of the service- protocols which lead to these forms an interpretation of the service-rates is included. This interpretation may not completely cover the implications, and is far from unique. It merely illustrates the complexities of the service-function which do not exist in the standard single changes case.

4.1.1 Service upon customers selected in advance Assume that the service-characteristics at station i have the form

ˆ αi(ni)αi(ni − 1) ··· αi(ni − gi + 1) fi(gi, ni) = ci(ni) . gi! This form is used in [65] and [81] for discrete-time queueing networks. By including the numerical factors β(¯g) in the routing-part (cf. (3.5)), the service-function f(·) satisﬁes (3.2) with

N mi 1 N 1 ni 1 ψ(m ¯ ) = Y Y , φ(¯n) = Y Y . i=1 k=1 αi(k) i=1 ci(ni) k=1 αi(k) The service-characteristics given above may be interpreted in the following way. Suppose that customers move around in the station randomly 9.4 Network applications: service-form examples 233

at very high speed. At negative-exponential times with rate ci(ni) depending on the number of customers present, a server (e.g. a carrousel in manufacturing-protocols or a time-frame in token-ring-protocols) passes by to serve a selected number of customers. If k customers have not been considered for selection yet then with probability αi(k) a customer arriving at the server is served. When the ﬁrst gi customers that have arrived at the server are considered for selection the server leaves the station immediately. Then the service-characteristics have the form stated above.

4.1.2 Service on a number of customers selected during service Assume that the service-characteristics at station i have the form

ˆ αi(ni) ··· αi(ni − gi + 1) [1 − αi(ni − gi)] ··· [1 − αi(1)] fi(gi, ni) = ci(ni) . gi! (ni − gi)! By including the numerical factors β(¯g) in the routing-part (cf. (3.5)), the service-function f(·) satisﬁes (3.2) with

N 1 mi 1 − α (k) N 1 ni 1 ψ(m ¯ ) = Y Y i , φ(¯n) = Y Y . i=1 mi! k=1 αi(k) i=1 ci(ni) k=1 αi(k) Reconsider the interpretation given in 4.1.1 above. The service-protocol is changed in the following way. Again, the ﬁrst gi customers arriving at the server are all served according to the probabilities αi(k), but only if all remaining ni − gi customers are also considered for service, and not selected for service according to these probabilities, the customers are served. Otherwise more customers are served. The number gi is thus determined by the probabilities αi(k).

4.1.3 Service-selection until failure Assume that the service-characteristics at station i have the form

ˆ αi(ni) ··· αi(ni − gi + 1) fi(gi, ni) = ci(ni) [1 − αi(ni − gi)] . gi! By including the numerical factors β(¯g) in the routing-part (cf. (3.5)), the service-function f(·) satisﬁes (3.2) with

N m N n Y Yi 1 Y 1 Yi 1 ψ(m ¯ ) = (1 − αi(m ¯ )) , φ(¯n) = . i=1 k=1 αi(k) i=1 ci(ni) k=1 αi(k) 234 Group-local-balance

Reconsider the interpretation given in 4.1.1, but now the server will serve customers until failure, i.e. customers are served until a customer is not accepted by the server. If the server leaves the station immediately after a failure the service-characteristics have the form stated above.

4.1.4 Service for the whole group A special case of the service-characteristics described in 4.1.1 and 4.1.2 above is service for a whole group. Assume that the service-characteristics at station i have the form n ! ˆ i gi ni−gi fi(gi, ni) = pi (1 − pi) . (4.2) gi

By including the numerical factors β(¯g) in the routing-part (cf. (3.5)), the service-function f(·) satisﬁes (3.2) with

N 1 1 − p !mi N 1 1 !ni ψ(m ¯ ) = Y i , φ(¯n) = Y . i=1 mi! pi i=1 ni! pi The service-characteristics above correspond to a discrete-time inﬁnite- server queue with geometrical service and probability of success pi. The service-characteristics (4.2) correspond to the following choices for αi(k) and ci(ni) in 4.1.1 and 4.1.2 above

pi ni 4.1.1 αi(k) = k , ci(ni) = (1 − pi) , i = 1,...,N, 1 − pi 4.1.2 αi(k) = pi, ci(ni) = ni!, i = 1,...,N.

4.2 Cluster-dependent service-rates In this section the stations of the queueing network are grouped into disjoint clusters Ci, i = 1,...,R, and the service-speed at a station within a cluster is a function of the total number of customers in that cluster. More precisely, with the following notation for the number of customers in a cluster

X ni = nj, i = 1,...,R, j∈Ci X gi = gj, i = 1,...,R, j∈Ci 9.4 Network applications: service-form examples 235

n¯ = (n1,..., nR) represents the total number of customers in the clusters, andr ¯ = (g1,..., gR) represents the total number of customers departing from stations in the clusters. The examples in Section 4.1 above can be generalized to examples for clusters of stations. Also, the examples for clusters of stations given below can be seen as examples for single stations when the number of stations contained in a cluster is set equal to one. The importance of cluster- dependent service-rates becomes clear when the routing of customers is taken into account too. Then customers may be served with a cluster- dependent and thus station-interdependent service-rate, but the routing is still to be considered in a more detailed manner. Further as in Example 4.2.3, cluster-interdependent service-rates are allowed too.

4.2.1 Service upon customers selected in advance Assume a service-characteristic similar to the service-characteristics described in 4.1.1. To this end, assume that customers within a cluster are picked out one after the other but at negative-exponential picking-times. This gives the following service-characteristic

R Y f(¯g, n¯) = f(¯r, n¯) = di(ni)αi(ni)αi(ni − 1) ··· αi(ni − gi + 1), i=1 which satisﬁes (3.2) with

P P mj nj R j∈Ci 1 R j∈Ci 1 ψ(m ¯ ) = Y Y , φ(¯n) = Y d ( X n ) Y . α (k) i j α (k) i=1 k=1 i i=1 j∈Ci k=1 i

4.2.2 Service for the whole group As a special application, by analogy with 4.1.4, consider the case in which customers are independently selected to be served with probability pi, i = 1,...,R. Selected customers are simultaneously served and released immediately after completing service. The service-characteristics have the form

R ! Y ni gi ni−gi f(¯r, n¯) = pi (1 − pi) , i=1 (ni − gi)! which satisﬁes (3.2) with 236 Group-local-balance

P m R ! j∈C j Y 1 1 − pi i ψ(m ¯ ) = , P pi i=1 j∈Ci mj ! P R ! nj Y 1 1 j∈Ci φ(¯n) = . P pi i=1 j∈Ci nj !

4.2.3 Service-delay Suppose that the number of customers in cluster 1 should not exceed some ﬁxed number T1 too much. This can be achieved by slowing down service at the other clusters with a factor ωi at cluster i, i = 2,...,N, when n1 > T1, where it is assumed that ωi 6= 0 for all i. As a consequence, the service-speed will be cluster-interdependent. More precisely, R Y f(¯r, n¯) = fi(¯r, n¯), i=1 where fi(·) has the form   ψi(ni − gi)  , if i = 1 or i = 2,...,R, and n1 ≤ T1,  φi(ni) fi(¯r, n¯) =  ψ (n − g )  i i i  , if i = 2,...,R, and n1 > T1. ωiφi(ni) Then the service-characteristics have the form (3.2) with R Y X ψ(m ¯ ) = ψi( mj), i=1 j∈Ci

 R R !  R Y Y X Y X φ(¯n) =  ωi − ωi − 1 1{ nj ≤ T1} φi( nj). i=2 i=2 j∈C1 i=1 j∈Ci

5 Network applications: routing-form examples

Throughout this section, assume that the routing characteristics have the form p(¯g, g¯0;m ¯ ) = λ(¯g, g¯0)b(¯g, g¯0;m ¯ ), (5.1) where λ(·) represents the state-independent multiple customer routing probability, and b(·) is a blocking function. The ﬁrst and second example assume that b(·) = 1, i.e. that no blocking occurs. The third and fourth example illustrate two possible types of blocking that can be modelled. 9.5 Network applications: routing-form examples 237

5.1 No blocking; batch routing Assume that there exists a positive solution y(¯g) of the batch traﬃc equations y(¯g) X λ(¯g, g¯0) = X y(¯g0)λ(¯g0, g¯), (5.2) g¯06=¯g g¯06=¯g and that no blocking occurs, i.e. that for allg, ¯ g¯0, m¯

b(¯g, g¯0;m ¯ ) = 1.

Then, for allm ¯ such thatm ¯ +g ¯ ∈ V , a solution of (3.6) is given by

y(¯g;m ¯ ) = y(¯g).

If there exists a function Γ(·) such that for allm ¯ , all i, and allm ¯ +¯g, m¯ +¯g0 ∈ Vi(m ¯ ) Γ(m ¯ +g ¯) y(¯g;m ¯ ) = , (5.3) Γ(m ¯ +g ¯0) y(¯g0;m ¯ ) then the πy(·)-part of (3.8) is given by

πy(¯n) = Γ(¯n), for alln ¯ ∈ V .

5.2 No blocking; independent routing In various papers on queueing networks with batch routing it is assumed that customers route independent of the other routing customers and independent of the state of the network (cf. [40], [65], [81]). However, the explicit form for the state-independent batch routing probabilities λ(·) has not been reported in the literature. This example gives this form and discusses arrival processes for discrete-time open queueing networks used in [65] and [81]. Assume that customers route among the stations independent of the state of the queueing network and independent of the other routing customers. Consider a transition from staten ¯ to staten ¯0 due tog ¯ customers leaving andg ¯0 customers entering the stations. The transition rate for this transition has the form (5.1) with b(·) = 1. 238 Group-local-balance

If the queueing network is closed, the state-independent batch routing probabilities have the form

N g ! N λ(¯g, g¯0) = X Y i Y pgij , g , . . . , g ij   i=1 i1 iN j=1  gij, i = 1, . . . , N, j = 1,...,N     gij ≥ 0, gij = 0 if pij = 0,  PN g = g , i = 1,...,N,  j=1 ij i   PN 0  i=1 gij = gj, j = 1,...,N (5.4) where pij represents the routing probability for a single customer to route from station i to station j and gij represents the number of customers that route from station i to station j in the transitiong ¯ → g¯0. If there exists a N positive solution {ci}i=1 of the traﬃc equations

N X γj = γipij, j = 1,...,N, i=1 then a solution of the batch traﬃc equations (5.2) is given by

N cgi y(¯g) = Y i . (5.5) i=1 gi!

QN If the numerical factors β(¯g) = i=1 gi! appearing in (5.4) are included in the service-part, then these factors do not appear in y(·) above, and the function N Y mi+gi Γ(m ¯ +g ¯) = ci i=1 satisﬁes (5.3). The πy(·)-part of (3.8) is then given by

N Y ni πy(¯n) = ci (5.6) i=1 for alln ¯ ∈ V . Open queueing networks can be modelled too. To this end, assume that with probability P (g0) a batch consisting of g0 customers enters the network. After arrival in the network customers are independently routed to the stations. With probability p0i an arriving customer routes to station i. Then the open queueing network form of (5.4) is given by 9.5 Network applications: routing-form examples 239

∞ ∞ 0 X X λ(¯g, g¯ ) = P (g0) g =0 0 0 g0=0 N g ! N × X Y i Y pgij . g , . . . , g ij   i=0 i0 iN j=0  gij, i = 0, . . . , N, j = 0,...,N :     gij ≥ 0, gij = 0 if pij = 0, g00 = 0,  PN g = g , i = 0,...,N,  j=0 ij i   PN 0  i=0 gij = gj, j = 0,...,N (5.7) The routing probabilities defined in (5.4) and (5.7) are proper routing probabilities, that is these routing probabilities are properly normalized: X λ(¯g, g¯0) = 1. g¯0 To verify the normalization, consider the open network version (5.7). The verification for the closed network version (5.4) is almost identical. Note that some of the terms appearing in λ(¯g, g¯0) may be zero. This is a consequence of the restrictions in the sum over {gij}. Also, note that for fixed 0 PN 0 0 PN g,¯ g¯ all terms for which g0 > j=1 gj or g0 > i=1 gi do not contribute to λ(¯g, g¯0). Therefore, arbitrary batch arrivals and departures as reflected by the first two infinite summations can be included. This is found to be more convenient. X λ(¯g, g¯0) g¯0 ∞ ∞ N N gij X X X X Y Y pij = P (g0) gi! 0 g =0 0 i=0 j=0 gij! g¯ 0 g0=0    gij, i = 0, . . . , N, j = 0,...,N :     gij ≥ 0, gij = 0 if pij = 0, g00 = 0,  PN g = g , i = 0,...,N,  j=0 ij i   PN 0  i=0 gij = gj, j = 0,...,N ∞ N N gij X X Y Y pij = P (g0) gi! gij! g0=0  g , i = 0, . . . , N, j = 0,...,N :  i=0 j=0  ij  gij ≥ 0, gij = 0 if pij = 0, g00 = 0,  PN   j=0 gij = gi, i = 0,...,N  g ∞ N  N  i X Y X = P (g0)  pij g0=0 i=0 j=0 240 Group-local-balance

∞ X = P (g0) = 1. g0=0 Here the Fubini theorem is used for interchanging summations which is allowed since all terms in the summations are non-negative. As an example of the arrival probabilities to the network, assume that batch arrivals into the network have a Poisson distribution with parameter γ0, that is γg0 0 −γ0 P (g0) = e . g0! N If there exists a positive solution {cj}j=1 of the traﬃc equations

N X γj = γ0p0j + γipij, j = 1,...,N, i=1 then (5.5) is a solution of the batch traﬃc equations

X y(¯g)λ(¯g, g¯0) = y(¯g0). (5.8) g¯

This can be seen as follows. Substitution of (5.5) into the left-hand side of (5.8) gives

X y(¯g)λ(¯g, g¯0) g¯

∞ ∞ g0 N gi X X X γ Y γ = 0 e−γ0 i g¯ g =0 0 g0! i=1 gi! 0 g0=0 N g ! N × X Y i Y pgij g , . . . , g ij   i=0 i0 iN j=0  gij, i = 0, . . . , N, j = 0,...,N :     gij ≥ 0, gij = 0 if pij = 0, g00 = 0,  PN g = g , i = 0,...,N,  j=0 ij i   PN 0  i=0 gij = gj, j = 0,...,N

∞ ∞ N N gij X X X X Y Y (γipij) = e−γ0 g¯ g =0 0 i=0 j=0 gij! 0 g0=0    gij, i = 0, . . . , N, j = 0,...,N :     gij ≥ 0, gij = 0 if pij = 0, g00 = 0,  PN g = g , i = 0,...,N,  j=0 ij i   PN 0  i=0 gij = gj, j = 0,...,N 9.5 Network applications: routing-form examples 241

0 N !gj X ∞ N γipij X −γ0 Y i=0 = e 0 0 j=0 gj! g0=0 0 g0 g ∞ γ 0 N γ j X 0 −γ0 Y j = 0 e 0 0 g0! j=1 gj! g0=0 = y(¯g0).

Here again Fubini’s theorem justiﬁes interchanging summations. QN If the numerical factors β(¯g) = i=1 gi! are included in the service-part, then the πy-part of (3.8) is given by (5.6). A second example of an arrival process is that of Bernoulli arrivals. To this end, assume that P (g0) has a geometric distribution with parameter p, that is p !g0 1 − 2p! P (g ) = (p < 1 ). 0 1 − p 1 − p 2

Note that in contrast with the Poisson arrival case, g0! is not included in P (g0). To obtain a solution to the batch traﬃc equations, in this case λ(¯g, g¯0) takes the form

∞ ∞ N N gij 0 X X X Y Y pij λ(¯g, g¯ ) = P (g0) , g =0 0 i=0 j=0 gij! 0 g0=0    gij, i = 0, . . . , N, j = 0,...,N :     gij ≥ 0, gij = 0 if pij = 0, g00 = 0,  PN g = g , i = 0,...,N,  j=0 ij i   PN 0  i=0 gij = gj, j = 0,...,N where the factorials gi!, i = 0,...,N, are removed. Note that in contrast with the case of Poisson arrivals discussed above, in this case λ(¯g, g¯0) is not normalized and cannot be interpreted as a routing probability. However, since the queueing network modelled here is a continuous-time queueing network, the routing part of the transition rates is not required to be a probability distribution. Therefore, although less convenient from an interpretational point of view, the routing function λ(¯g, g¯0) as given above is a proper routing function. N If there exists a positive solution {cj}j=1 of the traﬃc equations

N X γj = γ0p0j + γipij, j = 1, . . . , N, γ0 = 1, i=1 242 Group-local-balance then N !gi Y p y(¯g) = γi (5.9) i=1 1 − p is a solution of the batch routing equations. This can be seen by substitution. Since λ(¯g, g¯0) is not normalized, the following batch routing equation must be satisﬁed by y(¯g)

X y(¯g)λ(¯g, g¯0) = X y(¯g0)λ(¯g0, g¯). (5.10) g¯ g¯

Substitution of (5.9) into the left-hand side of (5.10) gives

X y(¯g)λ(¯g, g¯0) g¯ ∞ ∞ ! N !gi X X X 1 − 2p Y p = γi g¯ g =0 0 1 − p i=0 1 − p 0 g0=0 N N pgij × X Y Y ij   i=0 j=0 gij!  gij, i = 0, . . . , N, j = 0,...,N :     gij ≥ 0, gij = 0 if pij = 0, g00 = 0,  PN g = g , i = 0,...,N,  j=0 ij i   PN 0  i=0 gij = gj, j = 0,...,N ∞ ∞ 1 − 2p! = X X X g¯ g =0 0 1 − p 0 g0=0 p !gij γ p N N 1 − p i ij × X Y Y   i=0 j=0 gij!  gij, i = 0, . . . , N, j = 0,...,N :     gij ≥ 0, gij = 0 if pij = 0, g00 = 0,  PN g = g , i = 0,...,N,  j=0 ij i   PN 0  i=0 gij = gj, j = 0,...,N 0 N !gj X p ∞ ! N γipij X 1 − 2p Y i=0 1 − p = 0 . (5.11) 0 1 − p j=0 gj! g0=0 9.5 Network applications: routing-form examples 243

Substitution of (5.9) into the right-hand side of (5.10) gives X y(¯g0)λ(¯g0, g¯) g¯ 0 N !g ∞ ∞ ! !g0 Y p i X X X 1 − 2p p = γi i=1 1 − p g¯ g =0 0 1 − p 1 − p 0 g0=0 N N pgij × X Y Y ij   i=0 j=0 gij!  gij, i = 0, . . . , N, j = 0,...,N :     gij ≥ 0, gij = 0 if pij = 0, g00 = 0,  PN g = g0, i = 0,...,N,  j=0 ij i   PN  i=0 gij = gj, j = 0,...,N g0 g0 p ! i p ! 0 N γi ∞ ! Y 1 − p X 1 − 2p 1 − p = 0 0 i=1 gi! 0 1 − p g0! g0=0 g0 p ! i ∞ ! N γi X 1 − 2p Y 1 − p = 0 . (5.12) 0 1 − p i=0 gi! g0=0 Comparison of (5.11) and (5.12) proves (5.10) for the form (5.9). Thus the πy-part of equilibrium distribution is given by

N !ni Y p πy(¯n) = γi . i=1 1 − p

5.3 Upper limit blocking; reversible routing Assume that in each station a maximum number of customers is allowed, 0 say ni ≤ Mi at station i, and that the routing characteristics p(¯g, g¯ ;m ¯ ) have the form (5.1) with b(¯g, g¯0;m ¯ ) = 1{m¯ +g ¯0 ≤ M¯ }. Customers route in groups and are blocked as a total group too. When a group is blocked the total group returns to their originating stations. Now assume that the state-independent batch routing process is reversible, that is assume that the solution y(·) of the batch traﬃc equations (5.2) satisﬁes y(¯g)λ(¯g, g¯0) = y(¯g0)λ(¯g0, g¯). 244 Group-local-balance

Then y(¯g;m ¯ ) = y(¯g)1{m¯ +g ¯ ≤ M¯ }, is a solution of (3.6), and with Γ(·) satisfying (5.3) the πy-part of (3.8) is given by ¯ πy(¯n) = Γ(¯n), n¯ ≤ M. This example can be seen as a special case of the strongly reversible processes discussed in Chapter 8 and is inserted here to illustrate the blocking function b(·). Any example given in Chapter 8 can be inserted here. Then the blocking function b(¯g, g¯0;m ¯ ) will be a symmetrical function ofg, ¯ g¯0 for allg, ¯ g¯0 such thatm ¯ +g, ¯ m¯ +g ¯0 ∈ V . Note that the indicator blocking presented here is indeed a symmetrical function restricted to V .

5.4 Minimal workload blocking Suppose that in each transition a minimum number of customers must remain in the stations, say Mi at station i. Then a transition from state n¯ to staten ¯0, due tog ¯ customers leaving andg ¯0 customers entering the stations, is blocked when ni − gi < Mi for at least one i. In this case all customers must remain in their originating stations. Assume that p(·) has the form (5.1), then the blocking function has the form

b(¯g, g¯0;m ¯ ) = 1{m¯ ≥ M¯ }.

If y(·) satisﬁes the batch routing equations (5.2), then for allm ¯ and all i such thatm ¯ +g ¯ ∈ Vi(m ¯ ) a solution of (3.6) is given by

y(¯g;m ¯ ) = y(¯g)1{m¯ ≥ M¯ }.

0 If there exists a function Γ(·) such that for allm ¯ , i andm ¯ +¯g, m¯ +¯g ∈ Vi(m ¯ ) Γ(m ¯ +g ¯) y(¯g) = , Γ(m ¯ +g ¯0) y(¯g0) then the πy-part of (3.8) is given by ¯ πy(¯n) = Γ(¯n), n¯ ≥ M. 9.6 Dual processes 245

6 Dual processes

In Chapter 5 the notion of duality is introduced for stochastic processes in the setting of queueing networks in which single changes are allowed only. As is noted at the end of Chapter 8, this notion of duality can be generalized to queueing networks in which batch service and batch N routing is allowed. To this end, assume that a positive solution {ci}i=1 exists to the state-dependent batch routing equations for allg, ¯ g¯0, m¯ such thatm ¯ +g, ¯ m¯ +g ¯0 ∈ V

( 0 ) X Y gk 0 Y gk 0 γk p(¯g, g¯ ;m ¯ ) − γk p(¯g , g¯;m ¯ ) = 0. (6.1) g¯0 k k

Then the πy-part of the equilibrium distribution is given in (5.6), and the process allows an invariant measure Φ given by

Y nk Φ(¯n) = φ(¯n) ck , n¯ ∈ V. k

The dual state space Sd can immediately be generalized to batch service and routing queueing networks. The dual process can be deﬁned similar to Deﬁnition 5.3.1. The transition rates for the dual process are

φ(m ¯ +g ¯0) qd(m, ¯ m¯ +g ¯0 − g¯) = pd(¯g, g¯0;m ¯ +g ¯0). ψ(m ¯ )

N Furthermore, if a positive solution {di}i=1 exists to

 0  !g !gk  1 k 1  X Y pd(¯g, g¯0;m ¯ +g ¯0) − Y pd(¯g0, g¯;m ¯ +g ¯0) = 0, g¯  k γk k γk  (6.2) the dual process allows an invariant measure Ψ given by

N Y mk d Ψ(m ¯ ) = ψ(m ¯ ) dk , m¯ ∈ S . k=1 Thus, for batch service and routing queueing networks the dual process can be deﬁned. Furthermore, the results given in Chapter 5 can be generalized to batch routing queueing networks. 246 Group-local-balance

7 Speciﬁc examples

In principle, any example from Section 4 can be combined with any example from Section 5 to obtain the equilibrium distribution

π(¯n) = Bφ(¯n)πy(¯n), with φ(·) specified in Section 4 and πy(·) specified in Section 5. This section will not give such combinations, but gives some specific examples of interest in themselves. First, in Section 7.1 discrete-time queueing networks that can be included in the framework of this chapter are discussed. In the remaining sections some examples including blocking of transitions are presented. In Section 7.2, the cyclic three centre model introduced in Example 2.1 is reconsidered, and a blocking protocol sufficient for GLB is presented. In Section 7.3 this protocol is generalized to queueing networks with random independent routing. Finally, Section 7.4 considers queueing networks with stations that are grouped into clusters.

7.1 Discrete-time queueing networks The most realistic examples of queueing networks with multiple transitions that have been reported in the literature are discrete-time queueing networks. Most notable in this respect are [65], [81]. Below it is show that the results of these references can be concluded from the general framework presented in this chapter. To this end, the general setting of discrete-time queueing networks is briefly reviewed and an interpretation of the discrepancy between discrete-time and continuous-time queueing networks is given. This will also provide a justification for the discrepancy between the equilibrium distribution for discrete-time and continuous-time queueing networks. Other related papers on discrete-time queueing networks [23], [40], [41], [73] will be addressed briefly in Section 7.1.4. In discrete-time queueing networks the time-axis is segmented into time-intervals or time-slots of fixed length. In each time-interval the probability of serving a number of customers and the probability for the arrival of a number of customers is defined. In general, these probabilities depend on the number of customers that are present at the stations during a time-slot. As is discussed in [65], two conventions regarding the number of customers that can be served during a time-slot can be distinguished. 9.7 Specific examples: discrete-time queueing networks 247

Late arrivals: a customer that arrives at a station during a time-slot cannot be served in the same time-slot. Early arrivals: a customer that arrives at a station during a time-slot has a non-null probability to be served in the same time-slot.

In both cases departures take place at the end of a time-slot, and arrivals at the beginning of a time-slot. The state of the queueing network is the number of customers present at the beginning of a time-slot. In the case of late arrivals, ﬁrst customers are selected for service at the station with a state-dependent service-probability and arrivals occur after service is completed. As a consequence, customers arrive late for service, and cannot receive service in the time-slot in which they arrive at a station. In the case of early arrivals, ﬁrst customers that are served in a previous time-slot arrive at the stations. Then all customers present at the stations (including those who have just arrived) are considered for service. Thus customers arriving at a station during a time-slot have a non-null probability to depart from the station at which they have just arrived. In particular, if a discrete-time queueing network with early arrivals is a tandem line, then a customer has a non-null probability to move through all stations of the queueing network in one time-slot. Note that customers in transit between the stations of the queueing network are not taken into account in the state-description of the discrete-time queueing network. In both [65] and [81] it is shown that a discrete-time queueing network with early arrivals has a product-form equilibrium distribution similar to the product-form equilibrium distribution obtained for a standard continuous- time Jackson network with single changes. Below, in Sections 7.1.1 and 7.1.2 a continuous-time analogue of the discrete-time queueing networks studied in [65] and [81] is given. The discrepancy between the continuous-time model and the discrete-time model is discussed in Section 7.1.3.

7.1.1 Walrand’s discrete-time queueing network In [81] a discrete-time queueing network with early arrivals is studied. The probability that a number of arrivals into the network occurs is assumed to have a state-independent Poisson distribution, that is the probability that g0 arrivals occur in time-slot n is

g0 λ −λ P (g0) = e , g0 ≥ 0. g0! 248 Group-local-balance

In any given time-slot, station i will serve gi out of ni customers present at the station with probability Si(gi, ni) given by

Si(0, ni) = ci(ni), αi(ni)αi(ni − 1) ··· αi(ni − gi + 1) Si(gi, ni) = ci(ni) , 0 < gi ≤ ni gi! where αi(0) = 1, αi(k) > 0 for k > 0, and ci(ni) such that

n Xi Si(k, ni) = 1, i = 1,...,N. k=0 After completing service the customers are independently routed among the arcs of the queueing network. Therefore, the routing-characteristics of the continuous-time queueing network have the form (5.7). Thus, the continuous-time queueing network with transition rates equal to the transition probabilities described above is a combination of Example 4.1.1 and Example 5.2 for Poisson arrivals. Note that the binomial factors QN β(¯g) = i=1 gi! thus appear both in the service-characteristics and in the routing-characteristics. Therefore, the method used in these examples to incorporate the binomial factors in the complementary part of the transition rates is justiﬁed. The equilibrium distribution at V is then given by N 1 ni c π(¯n) = B Y Y i . (7.1) i=1 ci(ni) k=1 αi(k)

7.1.2 Pujolle’s network of “extended Bernoulli queues” In [65] a discrete-time queueing network consisting of “extended Bernoulli queues” is discussed. For stations in isolation, both the early arrival and the late arrival case is discussed. However, when these stations are incorporated into a queueing network, early arrivals are considered only. The arrivals into the queueing network occur due to a generalized Bernoulli process. The probability of g0 arrivals in a given time-slot then obtains the geometric form

p !g0 1 − 2p! P (g ) = , (p < 1 ). 0 1 − p 1 − p 2 The service-process is assumed to be a generalized Bernoulli process too. The probability of serving gi out of ni customers present at station i in a 9.7 Speciﬁc examples: discrete-time queueing networks 249 given time-slot is then given by

gi ni−gi pi (1 − pi) 1 Ri(gi, ni) = , (pi < ). ni ni−1 ni 2 (1 − pi) + (1 − pi) pi + ··· + pi It is assumed that customers route among the stations of the network independent of the other routing customers and independent of the state of the network. The continuous-time queueing network with transition rates equal to the transition probabilities described above is a combination of Exam- ple 4.1.1 and Example 5.2 for the case of Bernoulli arrivals. As is shown in Example 5.2, the πy-part of the equilibrium distribution is given by

N !ni Y pci πy(¯n) = , i=1 1 − p by which the equilibrium distribution is concluded to be

N 1 "p(1 − p )c #ni π(¯n) = B Y i i , i=1 ci(ni) (1 − p)pi with ni (1 − pi) ci(ni) = , ni ni−1 ni (1 − pi) + (1 − pi) pi + ··· + pi N and {ci}i=1 the solution of the traffic equations. QN Note that the binomial factors β(¯g) = i=1 gi! do appear neither in the service-function nor in the routing function. As a consequence, these factors need not be incorporated into the complementary part of the transition rates as in Example 7.1.1. However, since these binomial factors do not appear in the routing function, the routing function is not normalized, that is λ(¯g, g¯0) does not represent a routing probability. In the continuous- time queueing network this does not give rise to interpretational problems. However, in the discrete-time queueing network discussed in [65], [66] the routing probabilities are not normalized which immediately suggests that the result for discrete-time queueing networks with geometric release probabilities obtained in these references is incorrect. A similar observation is made in [43], a reaction to the publication of [66]. However, in this reference it is argued that the continuous-time analogue of the discrete-time queueing network is incorrect too. It is my opinion that the continuous- time result presented above is correct, but I do agree with the statement 250 Group-local-balance of [43] that one should be very careful when continuous-time analogues of discrete-time results are given. The result given in [65] for discrete-time queueing networks can easily QN be corrected. In this case incorporate the binomial factor β(¯g) = i=1 gi! both in the service function and in the routing function. Then the routing function is properly normalized. Note that the service function now reduces to a special case of the service function given in [81]. To obtain a solution to the batch traffic equations, in addition assume that an additional factor g0! is present in the arrival process too. Then the transition rates of the continuous-time analogue of this discrete-time queueing network are basically unaffected, but the discrete-time queueing network presented in [65] reduces to a special case of the discrete-time queueing network discussed in [81].

7.1.3 Relation between discrete-time and continuous-time results The equilibrium distribution obtained in Examples 7.1.1 and 7.1.2 above for the continuous-time analogue of the discrete-time queueing networks discussed in [65] and [81] is not identical to the equilibrium distribution obtained for the discrete-time queueing networks discussed in these refer- −1 ences. An additional factor ci(ni) appears in the equilibrium distribution of the continuous-time queueing network. The discrepancy between the discrete-time and continuous-time equilibrium distributions is a consequence of the state-description used in the discrete-time models. In a discrete-time queueing network with early arrivals, arrivals occur just after the beginning of a time-slot, while departures take place just before the end of a time-slot. The state of the network is the number of customers present at the stations at the beginning of a time-slot. Thus, form ¯ n = (m1, . . . , mN ) the number of customer present at the stations at 0 0 0 the beginning of time-slot n,g ¯n = (g1, . . . , gN ) the number of customers arriving at the stations in time-slot n,g ¯n+1 = (g1, . . . , gN ) the number of customers departing form the stations in time-slot n, the state at the beginning of time slot n + 1 is

0 m¯ n+1 =m ¯ n +g ¯n − g¯n+1.

As arrivals occur before departures, the transitionm ¯ n → m¯ n+1 passes 0 through statem ¯ n +g ¯n. In [81] this observation is explicitly used since the 0 probability that departuresg ¯n+1 occur depends onm ¯ n +g ¯n. Thus state 0 m¯ n +g ¯n is a dual state for the process recording the number of customers 9.7 Speciﬁc examples: discrete-time queueing networks 251 present at the stations at the beginning of time-slots. In the continuous- time queueing networks presented in the examples above the service-rate depends on the number of customers present at the queueing network, and the state of the queueing network includes all customers that are considered for service. As a consequence, staten ¯ for the continuous-time queueing network is a dual state for the discrete-time queueing network. Now reconsider the dual process discussed in Chapter 5 and Section 6. As is observed there, the equilibrium distribution of the dual process involves the service function ψ rather than the service function φ. Since the dualizing method can be reversed, the equilibrium distribution of the discrete-time process should be the backward dual of the equilibrium distributions presented in Examples 7.1.1 and 7.1.2. The service functions for the process are in the case of Example 7.1.1 (cf. Section 4.1.1)

N mi 1 N 1 ni 1 ψ(m ¯ ) = Y Y , φ(¯n) = Y Y . i=1 k=1 αi(k) i=1 ci(ni) k=1 αi(k) As a consequence, for the appropriate routing function, the equilibrium distribution of the discrete-time model should read N mi c π(m ¯ ) = B Y Y i , i=1 k=1 αi(k) as is reported in [81]. A similar interpretation for the discrepancy between the continuous-time and discrete-time equilibrium distributions is given in [41], however, in this reference the dual process is not defined, and as a consequence the process used in [81] could not be appropriately explained. Regarding the dual process for the discrete-time queueing network discussed in [65] and [81], the following observation is of interest too. To this end, call the process discussed in these references the dual process. With the transition structure imposed on the processes discussed in these references, in a transition for the discrete-time primal process first a departure occurs and subsequently arrivals take place. More specific, letn ¯n denote the intermediate state of the queueing network in time-slot n, that is the state of the queueing network between arrivals and departures in time- slot n, then the evolution of the primal process is given by

0 n¯n+1 =n ¯n − g¯n+1 +g ¯n+1. As a consequence the discrete-time primal process corresponding to the discrete-time dual process with early arrivals is a discrete-time queueing network with late arrivals. 252 Group-local-balance

7.1.4 Further references on discrete-time queueing networks In [23], [73] discrete-time queueing networks with so-called doubly stochastic queues are studied. The doubly stochastic service-protocol allows dis- tinguishing of diﬀerent service-positions such as in FCFS or PS-disciplines by analogy with continuous-time service-disciplines (e.g. BCMP-protocol). In [23] open networks with geometrical input streams are studied, whereas in [73] a speciﬁc computer model is analysed to show how to deal with closed queueing networks. In both references a transition is allowed only if there is just one customer requesting a transition. If more than one customer completes service during a time-slot then all these customers have to restart service. If there is exactly one customer completing service the transition is allowed. If the transition is allowed then the customer routes among the arcs of the queueing network and is not blocked, i.e. it arrives at some other station of the queueing network independent of the state of the queueing network. In [40], [41] a framework of continuous-time queueing networks with multiple customer transitions is given. These continuous-time models are used to model discrete-time queueing networks. In [40] closed queueing networks are studied. The service-characteristics have the form (3.2) and customers route independent of the state of the queueing network and independent of the other routing customers, i.e. the routing characteristics have the form given in Example 5.2. In [41] closed and open queueing networks are studied. The service-characteristics have the form (3.3) and the routing characteristics have the more general form described in Ex- ample 5.1.

7.2 Deterministic anticipative blocking Reconsider the cyclic three centre model of Figure 9.1, but now with M customers present at the queueing network. At station 3 at most N3 customers are allowed. The state space of this closed queueing network is given by 3 X V = {n¯ : ni = M, n3 ≤ N3}. i=1 A transition from staten ¯ to staten ¯0 occurs due to the following mechanism. First at each station a group gi, i = 1, 2, 3 is selected for service. Group g1 and g2 are allowed to leave station 1 and 2 if each of these groups independently is allowed to enter station 3 after the departure of group g3 9.7 Speciﬁc examples: deterministic anticipative blocking 253 from station 3, i.e. only if

m3 + g1 ≤ N3, and m3 + g2 ≤ N3.

If either g1 or g2 is not allowed to enter station 3 all transitions are blocked and at all stations a new group gi, i = 1, 2, 3, is reselected for service, that is the state of the system eﬀectively remains the same. Otherwise the groups are routed according to the arcs depicted in Figure 9.1. The service-mechanism described above is natural in discrete-time queueing networks with late arrivals. First at each station within a time- slot a group gi, i = 1, 2, 3, is served, where information on the number of customers that are served at the other stations is present, but the servers have no information on the routing structure of the queueing network. At the end of the time-slot the groups are released in the network and select a new station according to the routing rules. If at some place blocking occurs all groups return to their stations. Otherwise they arrive at a new station at the beginning of the following time-slot. The transition rates for this process are given by

0 q(¯g, g¯ ;m ¯ ) = f(¯g, m¯ +g ¯)1{m3 + gi ≤ N3, i = 1, 2, 3}. (7.2)

When single changes are allowed only, as a direct consequence of (7.2), service at station 1 and station 2 must be stopped if n3 = N3, that is for a single changes queueing network the blocking-protocol imposed by (7.2) reduces to the stop-protocol. The GLB-equations, which deﬁne the qy-process, now have the very simple form y(¯g)p(¯g, g¯0;m ¯ ) = y(¯g00)p(¯g00, g¯;m ¯ ), (7.3) 0 00 whereg ¯ = (g3, g1, g2), andg ¯ = (g2, g3, g1) ifg ¯ = (g1, g2, g3), and

0 p(¯g, g¯ ;m ¯ ) = 1{m3 + gi ≤ N3, i = 1, 2, 3}.

For allg, ¯ m¯ such thatm ¯ +g ¯ ∈ V (m ¯ ), a solution of (7.3) is given by

y(¯g;m ¯ ) = y(¯g) = 1.

If the service-characteristics have the form (3.2), then by Theorem 3.2 it immediately follows that the equilibrium distribution at V is given by

π(¯n) = Bφ(¯n). 254 Group-local-balance

The blocking-protocol described above, where also customers departing from station 1 are blocked is required for the equilibrium distribution to satisfy GLB. This can be seen when considering the qy-process in detail. The qy-process must be strongly reversible in order to satisfy the conditions of the decomposition theorem, Theorem 3.2. If a groupg ¯ can leave the stations to route among the stations of the queueing network in the original Markov chain, then this group can leave the stations to route among the stations in the qy-process too. Since the qy-process must be strongly reversible, this groupg ¯ must be allowed to both leave and enter the stations. More speciﬁc, if in the qy-process, a group is allowed to route from station 3 to station 1, then, in contrast with the original Markov chain, this group must be allowed to route from station 1 to station 3 too. The blocking-protocol imposed by (7.2) is based on this observation.

7.3 Random anticipative blocking Consider the closed queueing network depicted in Figure 9.2 in which M customers are present. Suppose that at station 6 at most N6 customers are allowed. The state space of this queueing network is

7 X V = {n¯ : ni = M, n6 ≤ N6}. i=1 The previous example shows that in a cyclic queueing network, that is a queueing network with deterministic routing, if customers are only allowed to leave a station when they can be accepted at the constrained station, then the equilibrium distribution is of product-form. When the routing is random, however, the protocol is to be chosen more carefully, in order to guarantee GLB. The blocking-protocol thus obtained is more general than the batch-routing-stop-protocol obtained in the previous example. In some stations, say at station i, service may still continue at arbitrary speed even though m6 + gi > N6, while at other stations service of groups gi is to be stopped as soon as m6 + gi > N6. This also illustrates that the service-blocking-protocol used for single changes queueing networks “stop all stations if the constraint station reaches its maximum” (cf. [26]) is too restrictive when batch routings are involved. Suppose that the routing part of the transition rates has the form (5.2) with λ(¯g, g¯0) of the form (5.4), i.e. a customer routes among the stations independent of the other routing customers, and that the routing equations 9.7 Speciﬁc examples: clusters of stations 255

1 - @ @ 4 - @ @ @ 2 - @ 6 - @ @ 5 - @ 3 - @ 7 -

Figure 9.2. Closed queueing network

7 possess a positive solution {ci}i=1. Then the following blocking function b(¯g, g¯0;m ¯ ) is suﬃcient for the equilibrium distribution to satisfy GLB

0 0 b(¯g, g¯ ;m ¯ ) = 1{m6 + g1 + g2 ≤ N6, m6 + g6 ≤ N6}.

0 Here the term m6 + g6 ≤ N6 reﬂects that from station 4 and station 5 no more than N6 − m6 customers are allowed to leave and route to station 6. Service at station 5 may continue for customers routing to station 7. If the service-characteristics have the form (3.2), then the equilibrium distribution at V is given by

7 Y ni π(¯n) = Bφ(¯n) γi . i=1 If the queueing network is separated into an upper half above the line through station 5 and a lower half below this line, then it may be concluded that in the upper half at each level at most N6 − g6 customers are allowed to leave, where the levels consist of station 1 plus station 2 for the ﬁrst level, station 4 plus the upper half of station 5 for the second level and station 6 for the third level. In the lower half service may continue at an arbitrary speed.

7.4 Clusters of stations The examples below consider clusters of stations with a constraint on the total number of customers allowed in a cluster. Recall the notation introduced in Example 4.2, and assume that the service-characteristics have the form (3.2). For simplicity, cyclic models are considered only, as in 256 Group-local-balance

!! aa !! aa 2 a! 3 !a 1 -- aa !! aa !!

Figure 9.3. Queueing network with one cluster general a characterization of the routing probabilities without considering the explicit structure of the queueing network or of the clusters when multiple changes are allowed is most complicated.

7.4.1 Anticipative upper limit blocking Consider the queueing network depicted in Figure 9.3, where 1 and 2 are single stations and 3 represents a cluster of stations. The routing among 1, 2 and 3 is assumed to be cyclic, i.e. along the arcs, whereas the internal routing in cluster 3 is arbitrary. Suppose that at cluster 3 at most T3 customers are allowed, while at station 1 and 2 all arriving customers are accepted. If M customers are present in the queueing network then the state space is given by

N X X V = {n¯ : ni = M, nj ≤ T3}. i=1 j∈C3 The blocking protocol used in the subsequent examples is similar to that of Example 7.2 and states that at station 1 and 2 customers are allowed to leave only if they could be accepted at cluster 3 after the departure of served customers from cluster 3.

Batch routing Assume that when a transition is allowed the customers route among the stations with batch routing probabilities λ(¯g, g¯0). The routing characteristics are given by

0 0 0 0 0 p(¯g, g¯ ;m ¯ ) = λ(¯g, g¯ )1{n3 − g1 + g2 ≤ T3, n3 − g1 + g2 ≤ T3}.

Assuming that there exists a solution Γ(m ¯ +g ¯) as introduced in Exam- ple 5.1, the equilibrium distribution of the queueing network at V now 9.7 Speciﬁc examples: clusters of stations 257 becomes π(¯n) = Bφ(¯n)Γ(¯n).

Independent routing Assume that customers route independent of the other routing customers while at each cluster ﬁrst a group ri is selected and then this group is distributed among the stations of the cluster to be served there. Then the transition rates have the form   R !  n gi!  0 Y i gi ni−gi  p(¯g, g¯ ;m ¯ ) =  pi (1 − pi) Y  × i=1 gi gj! j∈Ci 0 0 0 0 ×λ(¯g, g¯ )1{n3 − g1 + g2 ≤ T3, n3 − g1 + g2 ≤ T3} where λ(¯g, g¯0) has the form (5.4). The equilibrium distribution at V now becomes P R ! nj nj j∈Ci Y 1 1 Y Y nk π(¯n) = ck . P pi i=1 j∈Ci nj ! j∈Ci k=1

The interpretation of including the numerical factors β(·) is illustrated QR QN 0 here rather nicely since both i=1 gi! and j=1 gj! cancel in p(¯g, g¯ ;m ¯ ) above.

7.4.2 Anticipative minimal workload blocking Reconsider the queueing network depicted in Figure 9.3 where 1 and 2 are single stations and 3 represents a cluster of stations. Suppose in cluster 3 a minimal workload must be guaranteed. Example 5.4 considers the case in which after departure a minimum number of customers must be present at a station. This example considers the case in which after arrival a minimum number of customers has to be present. Note that there is an essential diﬀerence between the present example and Example 5.4. In this example the number of customers may fall below the minimal workload during a transition, whereas in Example 5.4 this is not allowed. Suppose in cluster 3 at least T3 customers have to be present after a transition, then a transition is allowed if

0 0 n3 − g1 + g2 ≥ T3, n3 − g1 + g1 ≥ T3. 258 Group-local-balance

If M customers are present in the queueing network then the state space is given by N X X V = {n¯ : ni = M, nj ≥ T3}. i=1 j∈C3 The routing characteristics are

0 0 0 0 p(¯g, g¯ ;m ¯ ) = λ(¯g, g¯ )1{n3 − g1 + g2 ≥ T3, n3 − g1 + g1 ≥ T3}.

Assuming that there exists a solution Γ(m ¯ +g ¯) as introduced in Exam- ple 5.1, the equilibrium distribution of the queueing network at V is given by π(¯n) = Bφ(¯n)Γ(¯n). Chapter 10

Negative customers

1 Introduction

Recently, motivated by neural networks, in [37] a queueing network with positive and negative customers was introduced. By analogy with neural networks a station of the queueing network represents a neuron. The neuron potential is represented by the queue-length at the station of the queueing network. A positive customer routing from one station to another represents an excitation signal, adding one unit to the neuron potential when it arrives, whereas a negative customer routing from one station to another represents an inhibition signal, reducing the potential of the neuron to which it arrives by one unit [35]. As the neuron potential increases or decreases by single units at a time, a single type of customers is sufficient to represent the neuron potential. Therefore, the queue at a station is assumed to consist of positive customers only. In queueing networks with positive and negative customers the positive customers behave similar to normal customers in a Jackson network. Negative customers behave quite differently. When a negative customer enters a station the queue-length is reduced by one. Upon leaving a station a customer either remains a positive customer or, with fixed probability, becomes a negative customer. The routing characteristics of customers depends on their sign and is assumed to be state-independent. In addition to customers routing among the stations, customers enter the queueing network according to a Poisson process. By analogy with Jackson networks, in [37] the queue-length at the stations of the queueing network is allowed to be non-negative only. This represents the additional assumption made in [35] that the neuron poten-

259 260 Negative customers tial is not allowed to become negative. If a negative customer arrives at a station with queue-length 0 it is lost. Another application of queueing networks with positive and negative customers is to multiple resource systems with requests for service (positive customers) and decisions to cancel a request (negative customers). In this application it is obvious that the number of requests can be non-negative only. Decisions to cancel a request are ignored if no requests are present. When negative customers are involved, however, it seems logical to allow a negative number of customers at a station too, for example to represent backlog. A negative number of customers represents that a number of negative customers has entered the station when no positive customers were present or that a number of service-completions has occurred when no customers were present. Due to these completions a shortage of customers is generated at the station, i.e. the number of customers has become negative. The following example illustrates the behaviour of a queueing network with positive and negative customers. A two station queueing network will be used throughout this chapter to illustrate the results and is discussed extensively in Section 5. The blocking results that will be obtained for this example are characteristic for the results for general queueing networks with positive and negative customers. Note that the results of this chapter are not limited to the case of a two station queueing network.

Example 1.1 (Two station queueing network) Consider the two station queueing network depicted in Figure 10.1. Both stations have external arrivals of positive customers only. Customers completing service at a station route to the other station as negative customers. If the service- rate at the stations is “fast enough” then this queueing network is stable and possesses a product-form equilibrium distribution (cf. Section 5). Note that there are no customers leaving the queueing network. This shows that queueing networks with negative customers are a very special type of queueing network. 2

By substitution into the global balance equations, in [37] it is shown that a queueing network with negative and positive customers and fixed routing probabilities as described above, possesses a product-form equilibrium distribution if a solution exists to a set of non-linear customer-flow- equations. The customer-flow-equations presented in [37] do not seem to express a notion of local balance. As a consequence, the queueing network with positive and negative customers introduced in [37] seems to be a counter-example to the generally believed statement for queueing net- 10.1 Introduction 261

positive

negative

@ @ ?? 1 2

66 @ @ negative positive

Figure 10.1. Two station queueing network works that a form of local balance is directly related to a product-form equilibrium distribution. This chapter investigates the relation between the product-form equilibrium distribution obtained in [37] and an appropriate notion of local balance. Indeed, this chapter shows that the product-form result obtained in [37] is not a consequence of any notion of standard local balance. How- ever, a new and non-standard type of local balance which is specially ded- icated to the specific model is introduced. The product-form equilibrium distribution obtained in [37] is shown to satisfy this new and non-standard type of local balance. The major difference between standard local balance and local balance for queueing networks with positive and negative customers is that in contrast to local balance appearing in the literature, local balance for queueing networks with positive and negative customers does not compare the probability flow into and out of one and the same state. As such it is shown that queueing networks with positive and negative customers do not fit into any class of queueing networks which satisfy a standard form of local balance. The literature on product-form queueing networks shows that a prod- Q ni uct-form equilibrium distribution π(¯n) = i qi satisfies local balance if the coefficients {qi} are a solution of the traffic equations. The traffic 262 Negative customers equations state that the average input rate of a station equals the average output rate of that station. It will be shown that the traffic equations for queueing networks with positive and negative customers introduced in this chapter do state this equivalence too. These traffic equations are a set of non-linear equations directly related to the customer-flow-equations of [37]. Note that for processes with product-form equilibrium distribution non-linearity of the traffic equations is not uncommon in the literature (e.g. polymerization processes [8], [55], [87], batch routing queueing networks [9], [40]). Therefore, in this respect local balance for queueing networks with positive and negative customers does fit into local balance as appearing in the literature. In contrast to a standard queueing network (Jackson network), where a unique solution to the traffic equations is guaranteed if the process is irreducible, for a queueing network with positive and negative customers a unique solution to the traffic equations is not a direct consequence of irreducibility. For the two station queueing network discussed in Example 1.1 it is shown that a unique positive solution exists to the traffic equations. For queueing networks consisting of more than two stations some classes of queueing networks for which a unique solution to the traffic equations exists are discussed. Finally it is argued that the traffic equations are characteristic equations to the queueing network. The traffic equations determine not only the equilibrium distribution, but also the shape of the state space and the blocking conditions used at the boundaries of the state space. Summarizing, this chapter incorporates queueing networks with positive and negative customers introduced in [37] into an extended class of queueing networks which satisfy local balance and extends local balance to processes for which the rate out of a state is not balanced by the rate into that state. Local balance gives insight into the behaviour of the Markov process describing the queueing network. In particular, local balance allows the introduction of boundaries to the state space (blocking phenomena). The chapter is organized as follows. Section 2 presents the model in a theoretical setup and gives an interpretation of the model as queueing network with positive and negative customers afterwards as this is math- ematically more elegant. In Section 3 blocking of transitions due to local balance is considered. In particular, lower and upper bounds to the state space are discussed extensively. Section 4 considers the traffic equations. Conditions guaranteeing a unique positive solution to the traffic equations 10.2 Model and local balance 263 are given. Furthermore, Section 4.5 argues that the traffic equations are characteristic equations to the queueing network. Section 5 discusses the two station queueing network introduced in Example 1.1.

2 Model and local balance

Consider a stable, regular continuous-time Markov chain, {N}, at state space S ⊂ ZN = {..., −2, −1, 0, 1, 2,...}N . A staten ¯ ∈ S is a vector with 0 0 components ni, i = 1,...,N. The transition rates, Q = (q(¯n, n¯ ), n,¯ n¯ ∈ ZN ), are given by

 Λ(i), n¯0 =n ¯ + e , i = 1,...,N,  i   0  r(i)d(i) + λ(i), n¯ =n ¯ − ei, i = 1,...,N,  0  + 0 q(¯n, n¯ ) = r(i)p (i, j), n¯ =n ¯ − ei + ej, i, j = 1,...,N,   − − 0  r(i)p (i, j) + r(j)p (j, i), n¯ =n ¯ − ei − ej, i, j = 1,...,N,    0, otherwise, (2.1) where Λ, r, d : {1,...,N} → [0, ∞), p+, p− : {1,...,N}2 → [0, ∞) and d(i), i = 1,...,N, p+(i, j), p−(i, j), i, j = 1,...,N, are such that for all i, i = 1,...,N,

N n o d(i) + X p+(i, j) + p−(i, j) = 1. (2.2) j=1

Throughout this chapter it will be assumed that p+(i, i) = p−(i, i) = 0, i = 1,...,N. The assumption p+(i, i) = 0 can be deleted without any complications to the theory. The assumption p−(i, i) = 0, however, is essential to the theory. If p−(i, i) > 0 then a customer at station i can cancel another customer at station i. The assumption p−(i, i) = 0 excludes this behaviour. The necessity of this assumption will become clear when the traﬃc equations are introduced and illustrated.

Interpretation 2.1 (Queueing network) The Markov process with transition rates (2.1) can be interpreted as representing a queueing network with positive and negative customers. To this end consider an open queueing network consisting of N stations. The state of a station represents the number of customers at the queue, i.e. ni is the number of customers at station i, i = 1,...,N. Customers at station i are served 264 Negative customers at negative-exponential rate r(i) independent of the state of the other stations. Upon departure from a station, say station i, for example due to external inﬂuence, a customer receives a positive sign with probability p+(i) and a negative sign with probability p−(i). Subsequently, a positive customer is routed to station j with probabilityp ˜+(i, j) and leaves the network with probability d+(i), and a negative customer is routed to station j with probabilityp ˜−(i, j) and leaves the network with probability d−(i). When a positive customer enters a station the number of customers at that station increases by one and when a negative customer enters a station the number of customers at that station decreases by one. Ex- ternal arrivals to the queueing network can either be positive customers which arrive at station i according to a Poisson process with rate Λ(i) or negative customers arriving at station i according to a Poisson process with rate λ(i). As negative customers are involved, it is natural to allow the number of customers at a station to be negative too. A negative number of customers represents the absence of a number of customers due to the arrival of negative customers or the departure of customers. The transition rates of the Markov chain representing this queueing network are given in (2.1), where p+(i, j) = p+(i)˜p+(i, j), p−(i, j) = p−(i)˜p−(i, j), d(i) = p+(i)d+(i) + p−(i)d−(i), and r(i) is the rate at which customers are expelled from station i. 2

The following theorem gives a suﬃcient condition for {N} to posses an invariant measure, i.e. a set of non-negative numbers, m = (m(¯n), n¯ ∈ ZN ), that satisﬁes the global balance equations

X {m(¯n)q(¯n, n¯0) − m(¯n0)q(¯n0, n¯)} = 0, n¯ ∈ ZN . (2.3) n¯06=¯n

N Theorem 2.2 If {qi}i=1 is a positive solution of

N N X − X + γir(i)+ γiγjr(j)p (j, i)+γiλ(i) = γjr(j)p (j, i)+Λ(i), i = 1,...,N, j=1 j=1 (2.4) then {N} possesses an invariant measure m at ZN given by

N Y nk N m(¯n) = qk , n¯ ∈ Z . (2.5) k=1 Proof Insertion of m in the global balance equations (2.3) gives for 10.2 Model and local balance 265 n¯ ∈ ZN 1 X {m(¯n)q(¯n, n¯0) − m(¯n0)q(¯n0, n¯)} n¯06=¯n m(¯n)   N  N n o = X Λ(i) + r(i)d(i) + λ(i) + X r(i)p+(i, j) + r(i)p−(i, j) i=1  j=1  N ( X 1 − Λ(i) + qir(i)d(i) + qiλ(i) i=1 qi N ( ) X qj + −  + r(i)p (i, j) + qiqjr(i)p (i, j) j=1 qi  N  N  (∗) X  X −  = Λ(i) − qiqjr(i)p (i, j) − qir(i)d(i) − qiλ(i) i=1  j=1    N  N q 1  + X r(i) + λ(i) − X j r(i)p+(i, j) − Λ(i) i=1  j=1 qi qi  N  N (∗∗) X  X + = Λ(i) + qjr(j)p (j, i) i=1  j=1 N  X −  −qir(i) − qiqjr(i)p (i, j) − qiλ(i) j=1  N  N X 1  X − + qir(i) + qiqjr(i)p (j, i) + qiλ(i) i=1 qi  j=1 N  X +  − qjr(j)p (j, i) − Λ(i) j=1  (∗∗∗) = 0, where in (∗) terms are rearranged such that the global balance equations resemble (2.4), and (∗∗) is obtained from (2.2). 2

The invariant measure given in (2.5) cannot be normalized at ZN . Thus, although {N} possesses a product-form invariant measure at ZN , {N} does not posses a product-form equilibrium distribution at ZN . If {N} is truncated to a subset of ZN which is bounded from above or bounded from below, m can be normalized. This truncation is obtained from local balance. Local balance is discussed below and the truncation 266 Negative customers of {N} is discussed in Sections 3.1 and 3.2. Equation (2.4) is the generalization of the traffic equations to networks that allow negative customers too. This can be seen by setting p−(i, j) = 0, i, j = 1,...,N, λ(i) = 0, i = 1,...,N. This removes the influence of negative customers from (2.4). For ci = γir(i), i = 1,...,N, (2.4) now reduces to the standard traffic equations

N X + ci = Λ(i) + cjp (j, i), i = 1,...,N. (2.6) j=1 Therefore, henceforth (2.4) will be called traffic equations. As can be seen from the proof of Theorem 2.2, the assumption that N N {qi}i=1 satisfies the traffic equations (2.4) can be relaxed to {qi}i=1 is such that (∗∗∗) is satisfied. However, similar to the assumptions in a Jackson network, where it is assumed that a solution exists to the standard N traffic equations (2.6), the assumption that {qi}i=1 satisfies the traffic equations (2.4) implies that the invariant measure m given in (2.5) satisfies ∗ a type of local balance. To this end, define qi , the transition rates for negative customers leaving station i to route to a station of the queueing network:

∗ − qi (¯n, n¯ − ei − ej) = r(i)p (i, j), i, j = 1,...,N. ∗ ∗ Then q(¯n, n¯ − ei − ej) = qi (¯n, n¯ − ei − ej) + qj (¯n, n¯ − ei − ej). With this definition, the traffic equations imply that m satisfies local balance for queueing networks with positive and negative customers. Definition 2.3 A Markov chain satisfies local balance for queueing networks with positive and negative customers if a measure m exists such that for all n¯ ∈ S, i = 1,...,N

N X m(¯n)q(¯n, n¯ − ei + ej) j=0 N X n ∗ ∗ o + m(¯n)qi (¯n, n¯ − ei − ej) + m(¯n + ej)qj (¯n + ej, n¯ − ei) j=1 N X = m(¯n − ei + ej)q(¯n − ei + ej, n¯). (2.7) j=0 Local balance for queueing networks with positive and negative customers expresses that the rate out of station i due to the arrival of negative customers to station i from the outside (first term in the left-hand side), due 10.2 Model and local balance 267 to the departure of customers from station i (first and second term in the left-hand side of (2.7)), and due to the arrival of negative customers from other stations (third term on the left-hand side) balances with the rate into station i due to the arrival of positive customers. In the case of lower or upper bounds, the traffic equations are necessary and sufficient for the existence of a product-form invariant measure as is shown in Sections 3.1 and 3.2. Therefore, the assumption that a solution exists to the traffic equations (2.4) is a very natural assumption. Local balance for queueing networks with positive and negative customers as defined in (2.7) expresses that the rate out of station i due to the departure of a customer due to either a service-completion or the arrival of a negative customer balances with the rate into station i due to the arrival of a positive customer. However, note that the third term in the left-hand side does not correspond to the rate out of station i in state n¯ but in state n¯ + ej. Therefore, local balance as expressed by (2.7) is not of the standard type, where global balance decomposes into local balance for each state separately. As a consequence, if a measure satisfies local balance it remains to be shown that this measure satisfies global balance. This cannot be concluded by summation of the local balance equations (2.7) over the stations as is the case for a standard queueing network. Also, the result of Theorem 2.2 is not a consequence of standard local balance, but is obtained by applying (2.4) for several states, once for staten ¯ and once for staten ¯ − ei, i = 1,...,N. Therefore, the result of Theorem 2.2 is not a consequence of standard local balance. In fact, the notion of local balance introduced in (2.7) does not correspond to notions of local balance as reported in the literature:

• Standard Jackson network: Transitions in which customers leave at two stations simultaneously, i.e. transitions of the typen ¯ → n¯ − ei − ej, are not allowed in a single changes queueing network. Therefore (2.7) cannot be concluded from local balance as appearing in the literature on single changes queueing networks.

• Batch routing network: In batch routing queueing networks, for the network to posses a product-form equilibrium distribution, it is assumed that a batchg ¯ = (g1, . . . , gN ) that can leave is allowed to enter too (cf. [9]). This can be seen when considering the batch traﬃc equations. For a batch routing queueing network with state- independent routing, for the process to posses a product-form equi- N librium distribution it is necessary and suﬃcient that for allg ¯ ∈ IN0 268 Negative customers

N a non-negative solution {qi}i=1 exists to

( N N 0 ) X Y gk 0 Y gk 0 γk p(¯g, g¯ ) − γk p(¯g , g¯) = 0. (2.8) g¯06=¯g k=1 k=1

As a transitionn ¯ → n¯ − ei − ej in which a customer leaves at both station i and station j is allowed, it must be the case that p(ei + ej, 0)¯ > 0. However, a transition in which a customer arrives at both station i and station j is not allowed in a queueing network with positive and negative customers, thus for allg ¯0 it must be the 0 case that p(¯g , ei + ej) = 0. Therefore, for a queueing network with positive and negative customers (2.8) cannot have a positive solution N {qi}i=1 as is required in Theorem 2.2. • Multiple types of customers: In single changes queueing networks with multiple types of customers a transitionn ¯ → n¯ − ei − ej is not allowed. In batch routing queueing networks with multiple types of customers, similar to the arguments given above, (2.8) generalized to also include multiple customer types cannot posses a positive N solution {qi}i=1. The main diﬀerence between local balance appearing in the literature and local balance for queueing networks with positive and negative customers is the following. Standard local balance states: rate out due to the departure ofg ¯ = rate in due to the arrival ofg ¯. In contrast, local balance for queueing networks with positive and negative customers states:

rate out due to the departure of ei and ei + ej, j = 1,...,N = rate in due to the arrival of ei. Comparison of these statements shows that (2.4) and (2.7) cannot be obtained from standard local balance as appearing in the literature unless p−(i, j) = 0 for all i, j. This is formalized in Lemma 3.4, where it is shown that a product-form equilibrium distribution for queueing networks with positive and negative customers satisﬁes standard local balance if and only if the inﬂuence of negative customers is deleted from the model. 10.2 Model and local balance 269

Example 2.4 (Two station queueing network) This example illustrates local balance for queueing networks with positive and negative customers as expressed by (2.4) and (2.7) for the special case of a two station queueing network. In Figure 10.2a for station 1 (i.e. i = 1), in Figure 10.2b for i = 2 and in Figure 10.2c for the outside, both (2.4) and (2.6) are visualized. Note that in a standard queueing network local balance for the outside is obtained by comparing the rate out of all stations to the rate into all stations, e.g. by summing (2.6) over i = 1, 2. This construction is used to obtain the case i = 0 for a queueing network with negative and positive customers too. This is justified since for p−(1, 2) = p−(2, 1) = λ(1) = λ(2) = 0 the queueing network with negative customers reduces to a standard queueing network. Global balance at staten ¯ for a standard queueing network is obtained by adding local balance for i = 0, 1, 2 as depicted in Figure 10.3a. For a queueing network with positive and negative customers adding local balance for i = 0, 1, 2 does not give global balance. As is shown in Figure 10.3b the transitions n¯ + e1 → n¯ − e2 andn ¯ + e2 → n¯ − e1 destroy this decomposition of global balance into local balance. Furthermore, from these figures it can be seen that the assumption p−(i, i) = 0 is necessary to the theory. If this assumption is not satisfied then an additional arcn ¯ + 2ei → n¯ must be added in these figures which will cause the behaviour of the Markov chain to be substantially more complex. 2 270

r r r

©I I R R

- n¯ - n¯ r r rr

Figure 10.2a. i = 1

n¯ n¯

r r r

r r r rr Figure 10.2b. i = 2

- n¯ n¯ - rr r rr

Figure 10.2c. i = 0

positive and negativer customers positive customers only 271

I ? R 6

- - r

? I 6 R

sum of local balance atn ¯ = global balance atn ¯

Figure 10.3a. standard queueing network

sum of local balance atn ¯ 6= global balance atn ¯

Figure 10.3b. queueing network with positive and negative customers 272 Negative customers

r r r r

©I ©I R R

- - n¯ I n¯ r r r r

In Section 2 an invariant measure for {N} with transition rates (2.1) is derived. As a direct consequence of the unrestricted state space, ZN , chosen in Section 2 this invariant measure cannot be normalized. Also, in applications it is more natural to constrain the number of customers at the stations, say −∞ < Ni ≤ ni ≤ Mi < ∞ for some Ni,Mi ∈ Z, i = 1,...,N. For example if Ni = 0, i = 1,...,N, the number of customers at the stations is allowed to be non-negative only. This gives the model discussed in [37]. As can be seen from Figure 10.3b, for {N} to be truncated to a state space with lower and upper bounds the transition rates at the boundaries must be modified since transitions leaving the state space are not allowed. As is shown in Theorem 2.2, m as given in (2.5) satisfies the global balance equations at the interior of the state space. For m to satisfy the global balance equations at the boundaries, the modification of the transition rates at the boundaries must be chosen very carefully. As the proof of Theorem 2.2 is based on local balance it is sufficient to consider local balance when modifying the transition rates. Section 3.1 considers lower bounds, and Section 3.2 considers upper bounds.

3.1 Lower bounds This section considers {N} truncated to S ⊂ ZN , where

S = {n¯ : −∞ < Ni ≤ ni, i = 1,...,N}. 10.3 State space truncation: lower bounds 273

The transition rates at the boundary of S are modified such that local balance at the boundary is preserved. To this end consider Example 2.4. As can be seen from Figures 10.2a, 10.2b for n1 ≥ N1, n2 = N2 local balance for i = 2 does not cross this boundary. In contrast, for i = 1 the transitionn ¯ → n¯ − e1 − e2 leaves S. For local balance to be preserved this transition must be modified. A straightforward modification is to simply replace the transitionn ¯ → n¯ − e1 − e2 byn ¯ → n¯ − e1 with the same rate. This is depicted in Figure 10.4. The physical interpretation of this modification is that a negative customer leaving station 1 leaves the queueing network since it cannot enter station 2 or that a negative customer entering station 2 when n2 = N2 is lost. For the general process this modification gives that the transition rates Q = (q(¯n, n¯0), n,¯ n¯0 ∈ S) are

N X − q(¯n, n¯ − ei) = r(i)d(i) + λ(i) + r(i)p (i, j)1{nj = Nj}, i = 1,...,N, j=1 (3.1) and q(¯n, n¯0), n,¯ n¯0 ∈ S, as given in (2.1) otherwise. The following theorem shows that under this modification the traffic equations are necessary and sufficient for {N} to have a product-form equilibrium distribution at S. Theorem 3.1 {N} possesses a unique, positive, product-form equilibrium distribution π at S = {n¯ : ni ≥ Ni, i = 1,...,N,Ni ∈ Z, i = 1,...,N} given by N Y 1 − qk π(¯n) = qnk , n¯ ∈ S, (3.2) Nk k k=1 qk N if and only if {qi}i=1 is the unique positive solution of (2.4) such that qi < 1, i = 1,...,N.

N Proof Assume that {qi}i=1 is a positive solution of (2.4). For m(¯n) = QN nk k=1 qk this gives X {m(¯n)q(¯n, n¯0) − m(¯n0)q(¯n0, n¯)} n¯06=¯n N  X  = m(¯n) q(¯n, n¯ + ei) + q(¯n, n¯ − ei) i=1  N  X  + {q(¯n, n¯ − ei + ej) + q(¯n, n¯ − ei − ej)} j=1  274 Negative customers

N  X  − m(¯n − ei)q(¯n − ei, n¯) + m(¯n + ei)q(¯n + ei, n¯) i=1  N X + {m(¯n − ei + ej)q(¯n − ei + ej, n¯) j=1   +m(¯n + ei + ej)q(¯n + ei + ej, n¯)} 

N  X  = m(¯n) Λ(i) + λ(i)1{ni > Ni} + r(i)d(i)1{ni > Ni} i=1  N X − + r(i)p (i, j)1{nj = Nj, ni > Ni} j=1 N X + + r(i)p (i, j)1{ni > Ni} j=1 N X − + r(i)p (i, j)1{nj > Nj, ni > Ni)} j=1 1 − Λ(i)1{ni > Ni} − qiλ(i) − qir(i)d(i) qi N X − − qir(i)p (i, j)1{nj = Nj} j=1 N N  X qj + X −  − r(j)p (j, i)1{ni > Ni} − qiqjr(i)p (i, j) j=1 qi j=1  N  N (∗) X  X − = m(¯n) Λ(i) − qir(i)d(i) − qir(i)p (i, j)1{nj = Nj} i=1  j=1 N  X −  − qiqjr(i)p (i, j) − qiλ(i) j=1  N  N  X  1 X qj +  +m(¯n) r(i) + λ(i) − Λ(i) − r(j)p (j, i) 1{ni > Ni} i=1  qi j=1 qi  N  N (∗∗) X  X + = m(¯n) Λ(i) + qjr(j)p (j, i) i=1  j=1 10.3 State space truncation: lower bounds 275

N  X −  −qir(i) − qiqjr(j)p (j, i) − qiλ(i) j=1  N  N X  X − + m(¯n − ei) qir(i) + qiqjr(j)p (j, i) + qiλ(i) i=1  j=1 N  X +  − qjr(j)p (j, i) − Λ(i) 1{ni > Ni} j=1  = 0, where (∗) is obtained by rearranging terms using (2.2) and (∗∗) by adding PN + PN − and subtracting i,j=1 qjr(j)p (j, i) and i,j=1 qjr(j)p (j, i)1(ni > Ni). QN nk Now assume that m(¯n) = k=1 qk is a positive invariant measure at S. Then, as m(¯n) > 0 for alln ¯ ∈ S, forn ¯ ∈ S and i = 1,...,N m(¯n + e ) i = q . m(¯n) i

Thus, qi > 0, i = 1,...,N. Inserting m(¯n) into the global balance equations gives, forn ¯ = (N1,...,NN ), the left lower corner of the state space,

N N  N  X X  X −  m(¯n) Λ(i) = m(¯n + ei) r(i)d(i) + r(i)p (i, j) + λ(i) i=1 i=1  j=1  N X − + m(¯n + ei + ej)r(i)p (i, j), i,j=1 which implies that

N N N N X X + X X − Λ(i) + qir(i)p (i, j) = qi {r(i) + λ(i)} + qiqjr(i)p (i, j). i=1 i,j=1 i=1 i,j=1 (3.3) Forn ¯ = (N1,...,NN ) + ek, k = 1,...,N,

( N ) m(¯n) X Λ(i) + λ(k) + r(k) i=1 N  N  X  X −  = m(¯n + ei) r(i)d(i) + r(i)p (i, j) + λ(i) i=1  j6=k  N X − +m(¯n − ek)Λ(k) + m(¯n + ei + ej)r(i)p (i, j) i,j=1 276 Negative customers

N X + + m(¯n − ek + ej)r(j)p (j, k). j=1 From (3.3) and rearranging terms

N X λ(k) + r(k) + qir(i) i=1 N N  N  X + X  X −  = qir(i)p (i, j) + qi r(i)d(i) + r(i)p (i, j) i,j=1 i=1  j6=k  1 N q + Λ(k) + X j r(j)p+(j, k), k = 1,...,N. qk j=1 qk

N QN nk Using (2.2) this gives that {qi}i=1 satisfies (2.4). Thus m(¯n) = k=1 qk N is a positive invariant measure if and only if {qi}i=1 is a positive solution of (2.4). N If {qi}i=1 is the unique positive solution of (2.4) such that qi < 1, i = 1,...,N, then, as is shown above, π(¯n) as given in (3.2) is an invariant measure. As 0 < qi < 1, i = 1,...,N, π is a positive product-form equilibrium distribution. Now assumeπ ˜ is another positive product-form N equilibrium distribution, i.e. for a set of constants {q˜i}i=1,π ˜ can be written QN nk π˜(¯n) = B k=1 q˜k ,n ¯ ∈ S, and there exists ann ¯ ∈ S such thatπ ˜(¯n) 6= π(¯n) andπ ˜ satisfies the global balance equations at S. Then, asπ ˜ is a positive probability distribution, it must be the case that 0 < q˜i < 1, N i = 1,...,N. Furthermore, as is shown above, {q˜i}i=1 satisfies (2.4). As π˜(¯n) 6= π(¯n) for somen ¯ ∈ S, it must be the case thatq ˜i 6= qi for some i which is in contradiction to the assumption on the uniqueness of the solution to (2.4). To prove the reversed statement, note that since π given in (3.2) is a positive probability distribution, it must be that 0 < qi < 1, i = 1,...,N. N N As is shown above, {qi}i=1 satisfies (2.4). Now assume {q˜i}i=1 such that 0 < q˜i < 1, i = 1,...,N, also satisfies (2.4) andq ˜i 6= qi for some i. Then QN nk π˜ given byπ ˜(¯n) = B k=1 q˜k ,n ¯ ∈ S, is an equilibrium distribution. As {N} possesses a unique product-form equilibrium distribution, forn ¯ ∈ S and i = 1,...,N π(¯n + e ) π˜(¯n + e ) q = i = i =q ˜ . i π(¯n) π˜(¯n) i

N Thus {qi}i=1 is the unique solution to (2.4) such that 0 < qi < 1, i = 1,...,N. 2 10.3 State space truncation: lower bounds 277

The statement of Theorem 3.1 is that {N} possesses a unique positive product-form equilibrium distribution given in (3.2). This does not imply that π is the unique equilibrium distribution. For π to be the unique equilibrium distribution, it must be assumed that {N} is irreducible. Suﬃcient conditions for {N} to be irreducible are, for example, that 1. Λ(i) > 0, i = 1,...,N, λ(i) > 0, i = 1,...,N, 2. Λ(i) > 0, i = 1,...,N, r(i)p−(i, j) > 0, i, j = 1,...,N, 3. Λ(i) > 0, i ∈ I ⊂ {1,...,N}, r(i)p+(i, j) > 0, i ∈ I, j ∈ {1,...,N}\I, λ(i) > 0, i = 1,...,N,

N 4. there exists a unique, positive solution {ci}i=1 to  N  N  X +  X + γi d(i) + p (i, j) = Λ(i) + γjp (j, i), i = 1,...,N.  j=1  j=1 This condition states that if the network with positive customers only is irreducible, then the network with positive and negative customers is irreducible too. N If {N} is irreducible and a solution {qi}i=1 such that 0 ≤ qi < 1, i = 1,...,N, exists to the traffic equations then π as given in (3.2) is the unique equilibrium distribution. Under the product-form equilibrium distribution given in (3.2) the traffic equations (2.4) state that the expected probability flow out of station i equals the expected probability flow into station i, i = 1,...,N. This can be seen by computing these expectations. The average output of station i due to the departure of positive customers from station i routing to station j is given by X Eqij = π(¯n)q(¯n, n¯ − ei + ej) n¯∈S N Y X 1 − qk nk + = q r(i)p (i, j)1(ni > Ni) Nk k k=1 nk≥Nk qk + = qir(i)p (i, j). The average output of station i due to the arrival of negative customers from station j is given by N − Y X 1 − qk nk − Eq = q r(j)p (j, i)1{nj > Nj, ni > Ni} ij Nk k k=1 nk≥Nk qk − = qiqjr(j)p (j, i). 278 Negative customers

Similarly the other terms can be computed. This gives: qir(i) = average output of station i due to the departure of negative and positive customers PN − j=1 qiqjr(j)p (j, i) = average output of station i due to the arrival of negative customer from other stations qiλ(i) = average output of station i due to the arrival of negative customers from the outside Λ(i) = average arrival rate at station i due to the arrival of positive customers from the outside PN + j=1 qjr(j)p (j, i) = average arrival rate at station i due to the arrival of positive customers from the network

The left-hand side of (2.4) represents the average rate of departures of customers from station i and the right-hand side represents the average rate of arrivals at station i, i = 1,...,N. Thus, the traffic equations are natural equations for {N} at S. Moreover, this gives a second motivation for calling (2.4) the traffic equations. In [37] the queueing network with positive and negative customers discussed above is introduced for the case Ni = 0, i = 1,...,N. A sufficient condition for {N} to posses a product-form equilibrium distribution is given only. The following lemma relates the sufficient condition given in [37], (3.4a)–(3.4c), to the traffic equations. In Remark 3.3 Gelenbe’s result is related to the result obtained here.

N Lemma 3.2 {qi}i=1 is a solution to

N + X + λ (i) = γjr(j)p (j, i) + Λ(i), i = 1,...,N, (3.4a) j=1

N − X − λ (i) = γjr(j)p (j, i) + λ(i), i = 1,...,N, (3.4b) j=1 λ+(i) γ = , i = 1,...,N, (3.4c) i r(i) + λ−(i) N if and only if {qi}i=1 is a solution to

N N X − X + γir(i)+ γiγjr(j)p (j, i)+γiλ(i) = γjr(j)p (j, i)+Λ(i), i = 1,...,N. j=1 j=1 (3.5) 10.3 State space truncation: lower bounds 279

N Proof If {qi}i=1 is a positive solution of (3.4a) – (3.4c). Substitution of (3.4c) into (3.4a) gives for i = 1,...,N

N − X + qi r(i) + λ (i) = qjr(j)p (j, i) + Λ(i). (3.6) j=1 N Substitution of (3.4b) into (3.6) gives that {qi}i=1 satisfies (3.5). N + − Now assume {qi}i=1 satisfies (3.5). Define λ (i), λ (i), i = 1,...,N as N N + X + − X − λ (i) = qjr(j)p (j, i) + Λ(i), λ (i) = qjr(j)p (j, i) + λ(i). j=1 j=1 Substitution of λ+(i), λ−(i) into (3.5) gives for i = 1,...,N − + qir(i) + qiλ (i) = λ (i). N Thus {qi}i=1 satisfies (3.4a)–(3.4c) . 2 Remark 3.3 (Gelenbe’s result) In [37] it is proven by substitution into N the global balance equations, that if {qi}i=1 is a positive solution to (3.4a)– (3.4c), then {N} possesses a product-form equilibrium distribution given in (3.2). Lemma 3.2 and Theorem 3.1 show that this result is reproduced here. The result obtained in Theorem 3.1 is more general than Gelenbe’s result as Theorem 3.1 gives a necessary and sufficient condition for π to be of product-form. Furthermore, the result of Theorem 3.1 is obtained by considering local balance. The insight obtained through local balance is an essential contribution to the theory on queueing networks with positive and negative customers. Local balance allows the introduction of boundaries as is shown in the present section, Section 3.2 and Section 5 and incorporates blocking phenomena into the theory. The coefficients λ+(i), λ−(i) defined in (3.4a), (3.4b) are related to the average arrival rate of customers at station i, i = 1,...,N. At first glance one suspects λ+, λ− to represent the average arrival rate of positive and negative customers respectively as is argued in [37]. Be aware, however, that λ+, λ− are indeed related to these arrival rates, but as follows: λ+(i) = average arrival rate of positive customers at station i; − qiλ (i) = average arrival rate of negative customers at station i. As the average output rate of customers from station i is the average output rate due to the arrival of negative customers plus the average − output rate due to service, i.e. qiλ (i) + qir(i), (3.4c) expresses that the average input rate equals the average output rate. 2 280 Negative customers

To conclude this section, the following lemma shows that the results obtained in this section cannot be obtained from standard local balance. Lemma 3.4 A queueing network with positive and negative customers and product-form equilibrium distribution (3.2) satisﬁes standard local balance equating for each state n¯ ∈ S the rate out of a station to the rate into a station for each station separately if and only if r(j)p−(j, i) = 0 for all i, j, i, j = 1,...,N. Proof Standard local balance reads forn ¯ ∈ S, i = 1,...,N

 N N   X X  π(¯n) q(¯n, n¯ − ei) + q(¯n, n¯ − ei + ej) + q(¯n, n¯ − ei − ej)  j=1 j=1  N X = π(¯n − ei)q(¯n − ei, n¯) + q(¯n − ei + ej, n¯). (3.7) j=1 Insertion of the transition rates (2.1), (2.2) and the equilibrium distribu- N tion (3.2) into (3.7) gives that {qi}i=1 satisﬁes for i = 1,...,N

N N X − X + qir(i) + qir(j)p (j, i) + qiλ(i) = Λ(i) + qjr(j)p (j, i). (3.8) j=1 j=1

N By Theorem 3.1, {qi}i=1 is a positive solution of (2.4) too. Comparison of N (2.4) and (3.8) shows that {qi}i=1 is a solution of both (2.4) and (3.8) if and only if r(j)p−(j, i) = 0 for all i, j. 2

Lemma 3.4 implies that the product-form equilibrium distribution (3.2) satisfies standard local balance only if the influence of negative customers is deleted from the model. To this end, first observe that the condition r(j)p−(j, i) = 0 for all i, j implies that p−(j, i) = 0 for all i, j since p−(j, i) > 0 implies that r(j) = 0. If r(j) = 0 then customers do not receive service at station j. In this case, since negative customers cannot enter station j from any other station of the queueing network as r(k)p−(k, j) = 0 for all k, the queue-length at station j reduces only if a negative customer enters station j from the outside. Then λ(j) effec- tively reduces to a service-rate for positive customers and these customers can leave the queueing network only. Therefore, it may be assumed that p−(i, j) = 0 for all i, j. The arrival of negative customers to a station from the outside can now be interpreted as an additional service-effort for positive customers. As a consequence, the service-function will become 10.3 State space truncation: upper bounds 281 more complex, but the notion of negative customers does not contribute to the model. Therefore, the results obtained in this chapter cannot be concluded from standard local balance as appearing in the literature on product-form queueing networks.

3.2 Upper bounds In Section 3.1 the transition rates are modiﬁed such that {N} possesses a product-form equilibrium distribution at S = {n¯ : ni ≥ Ni, i = 1,...,N}. This section addresses the complementary problem: upper limit blocking at the stations. Based on local balance, the transition rates are modiﬁed such that {N} possesses a product-form equilibrium distribution at

N S = {n¯ ∈ Z : ni ≤ Mi < ∞, i = 1,...,N}, i.e. the number of customers at station i is constrained not to exceed Mi. For standard queueing networks it is known that in the case of upper limit blocking special blocking protocols, such as the stop-protocol, must be applied to preserve a product-form equilibrium distribution (cf. [9], [26], [76]). For a queueing network with positive and negative customers these protocols do not preserve a product-form equilibrium distribution. To show this reconsider Example 2.4. For a standard queueing network the stop-protocol stops all transitions in which a customer leaves station 2 if station 1 is saturated, i.e. if n1 = M1. In Figure 10.2b this comes down to blocking the transitionsn ¯ ↔ n¯ ± e2. As can immediately be seen from Figures 10.2 and 10.3, at this boundary global balance reduces to local balance for i = 1. For a queueing network with positive and negative customers the stop protocol does not preserve product-form. This can be seen from Figure 10.2b. Blocking of transitions in which a customer leaves station 2 if n1 = M1 also stops the transitionn ¯ → n¯ − e1 − e2. As a consequence local balance for i = 1 atn ¯ − e2 is destroyed. Thus the stop protocol does not preserve product-form. A similar argument can be used for other blocking protocols for standard queueing networks. From local balance we obtain that the following modiﬁcation of the transition rates Q = (q(¯n, n¯0), n,¯ n¯0 ∈ S) preserves product-form:

N X + q(¯n, n¯ + ei) = Λ(i) + qkr(k)p (k, i)1{nk = Mk}, i = 1,...,N, k=1 N X − q(¯n, n¯ − ei) = r(i)d(i) + λ(i) + qkr(k)p (k, i)1{nk = Mk} (3.9) k=1 282 Negative customers

n¯ n¯

r r r

r Figurer 10.5. Modiﬁcationr r at upper boundr

N X + + r(i)p (i, k)1{nk = Mk}, i = 1,...,N, k=1 N where {qi}i=1 are arbitrary numbers such that qi > 1, i = 1,...,N and q(¯n, n¯0) as given in (2.1) otherwise. For the two station queueing network described in Example 2.4 this modification is illustrated in Figure 10.5. The additional terms have the following interpretation. If nk = Mk, then customers leaving station i to route to station k are lost, which explains PN + k=1 r(i)p (i, k)1{nk = Mk}. Furthermore, if nk = Mk, at station i + additional arrivals from the outside are triggered at rate qkr(k)p (k, i) for − positive customers and at rate qkr(k)p (k, i) for negative customers. The following theorem shows that {N} thus modified possesses a product-form equilibrium distribution at S. At first glance, both Theorem 3.5 and its proof are very similar to Theorem 3.1 and its proof. However, proving global balance in the proof of these theorems differs substantially. (For example see equation (∗∗) below.) Theorem 3.5 {N} possesses a unique, positive, product-form equilibrium distribution π at S = {n¯ : ni ≤ Mi, i = 1,...,N,Mi ∈ Z, i = 1,...,N} given by N Y qk − 1 π(¯n) = qnk , n¯ ∈ S, (3.10) Mk+1 k k=1 qk N if and only if {qi}i=1 is the unique positive solution of (2.4) such that qi > 1, i = 1,...,N. N Proof Assume that {qi}i=1 is a positive solution to the traffic equa- QN nk tions (2.4). Insertion of m(¯n) = k=1 qk into the global balance equations at S gives X {m(¯n)q(¯n, n¯0) − m(¯n0)q(¯n0, n¯)} n¯06=¯n 10.3 State space truncation: upper bounds 283

N  X  = m(¯n) Λ(i)1{ni < Mi} i=1  N X + + qkr(k)p (k, i)1{nk = Mk, ni < Mi} k=1 +r(i)d(i) + λ(i) N X − + qkr(k)p (k, i)1{nk = Mk} k=1 N X + + r(i)p (i, k)1{nk = Mk} k=1 N N X + X − + r(i)p (i, j)1{nj < Mj} + r(i)p (i, j) j=1 j=1 N 1 X qk + − Λ(i) − r(k)p (k, i)1{nk = Mk} qi k=1 qi −qir(i)d(i)1{ni < Mi} − qiλ(i)1{ni < Mi} N X − − qiqkr(k)p (k, i)1{nk = Mk, ni < Mi} k=1 N X + − qir(i)p (i, k)1{nk = Mk, ni < Mi} k=1 N X qi + − r(i)p (i, j)1{ni < Mi} j=1 qj N  X −  − qiqjr(i)p (i, j)1{ni < Mi, nj < Mj} j=1  N  (∗) X  = m(¯n) Λ(i) − qir(i)d(i) − qiλ(i) i=1  N ! X − − qiqkr(k)p (k, i) 1{ni < Mi} k=1 1 N q +r(i) + λ(i) − Λ(i) − X k r(k)p+(k, i) qi k=1 qi N X − + qkr(k)p (k, i)1{nk = Mk} k=1 284 Negative customers

N X + + qkr(k)p (k, i)1{nk = Mkni < Mi} k=1 N ) X + − qir(i)p (i, k)1{nk = Mk, ni < Mi} k=1 N ( N (∗∗) X X + = m(¯n) Λ(i) + qkr(k)p (k, i) i=1 k=1 N ) X − −qir(i) − qiqkr(k)p (k, i) − qiλ(i) 1{ni < Mi} k=1 N ( N X X − + m(¯n − ei) qir(i) + qiqkr(k)p (k, i) + qiλ(i) i=1 k=1 N ) X + −Λ(i) − qkr(k)p (k, i) k=1 = 0, where (∗) is obtained by rearranging terms, and (∗∗) is obtained from (2.2) and the following identity

N N X + X + qir(i)p (i, j)1{ni < Mi} + qir(i)p (i, j)1{nj < Mj, ni = Mi} i,j=1 i,j=1 N N X + X + = qir(i)p (i, j)1{nj < Mj} + qir(i)p (i, j)1{ni < Mi, nj = Mj}. i,j=1 i,j=1

QN nk Now assume that m(¯n) = k=1 qk is a positive invariant measure at S. Then, the assumption that m(¯n) > 0 for alln ¯ ∈ S implies that qi > 0, i = 1,...,N. Insertion of m(¯n) into the global balance equations gives forn ¯ = (M1,...,MN ), the right upper corner of the state space

N ( N ) X X − r(i) + qkr(k)p (k, i) + λ(i) i=1 k=1 N ( N ) X 1 X + = Λ(i) + qkr(k)p (k, i) . (3.11) i=1 qi k=1

Forn ¯ = (M1,...,MN ) − es

N  X  X − X + r(i)d(i) + λ(i) + qkr(k)p (k, i) + r(i)p (i, k) i=1  k6=s k6=s 10.4 Traﬃc equations 285

 X − X + +  + r(i)p (i, j) + Λ(s) + qkr(k)p (k, s) + r(i)p (i, s) j k6=s 

N  X  1 X qk + = Λ(i) + r(k)p (k, i) + qsr(s)d(s) + qsλ(s) i=1 qi k6=s,i qi  X − X + qs +  +qs qkr(k)p (k, s) + qs r(s)p (s, k) + r(s)p (s, i) . k6=s k6=s qi  N Rearranging terms and insertion of (3.11) gives that {qi}i=1 satisﬁes the traﬃc equations. The remainder of the proof is similar to the proof of Theorem 3.1. 2

N Note that the coefficients {qi}i=1 appearing in the modification (3.9) are appearing in the traffic equations too. Thus, in Theorem 3.5 it is implicitly assumed that the coefficients used in the modification also satisfy the traffic equations. This problem is discussed in Section 4.5.

4 Traﬃc equations

In the previous sections it is assumed that a unique positive solution exists to the traffic equations (2.4) such that either qi < 1, i = 1,...,N, (Sec- tion 3.1) or qi > 1, i = 1,...,N, (Section 3.2). This section addresses the problem of existence and uniqueness of a positive solution to the traffic equations. As is shown in Section 4.1 for the queueing network introduced in Example 2.4, this problem is not easy to solve. For the general queueing network consisting of more than two stations, for some very special cases existence and uniqueness of a positive solution to the traffic equations can be proven. In these cases, discussed in Sections 4.2 and 4.3, it can easily be shown that it may be the case that qi < 1, qj > 1, and qk = 1 for i 6= j 6= k. In Section 4.5 the traffic equations are discussed. It is argued that the traffic equations are characteristic equations for the process.

4.1 Uniqueness of the solution This section reconsiders the two station queueing network discussed in Examples 1.1 and 2.4. For this queueing network Theorem 4.1 shows that the traffic equations possess a unique positive solution. The first condition (irreducibility) is sufficient for the existence of a positive solution of the 286 Negative customers traffic equations in a standard queueing network. The second condition stating that the queueing network is not pathological is discussed in more detail in Section 4.4 and states that positive customers can enter each station. Theorem 4.1 (Uniqueness of solution) If the Markov chain, {N}, describing a two station queueing network is irreducible and not pathological, which means that

Λ(1) > 0 or r(2)p+(2, 1) > 0, (4.1a) and Λ(2) > 0 or r(1)p+(1, 2) > 0, (4.1b) and Λ(1) > 0 or Λ(2) > 0, (4.1c) then the traﬃc equations possess a unique positive solution q1, q2. Proof The traﬃc equations (2.4) for this special case read:

− + γ1r(1) + γ1γ2r(2)p (2, 1) + γ1λ(1) = γ2r(2)p (2, 1) + Λ(1), (4.2a)

− + γ2r(2) + γ2γ1r(1)p (1, 2) + γ2λ(2) = γ1r(1)p (1, 2) + Λ(2). (4.2b) First, consider the case where all parameters r(1), r(2), etc. are positive. Then (4.2a) can be written

+ γ2r(2)p (2, 1) + Λ(1) γ1 = − . (4.2c) r(1) + γ2r(2)p (2, 1) + λ(1)

Substitution of (4.2c) into (4.2b) and rearranging terms gives that q2 must satisfy

2 2 − + − − γ2 {r(2) p (2, 1) + r(1)r(2)p (2, 1)p (1, 2) + λ(2)r(2)p (2, 1)} − +γ2 {r(1)r(2) + λ(1)λ(2) + Λ(1)r(1)p (1, 2) + r(1)λ(2) + λ(1)r(2) −r(1)r(2)p+(1, 2)p+(2, 1) − Λ(2)r(2)p−(2, 1)} + {−Λ(1)r(1)p+(1, 2) − r(1)Λ(2) − λ(1)Λ(2)} = 0. (4.3) 2 As all parameters are positive the coefficient of γ2 is positive and the 0 coefficient of γ2 is negative. This implies that (4.3) possesses two distinct solutions, one positive and one negative solution. Thus (4.2a), (4.2b) 10.4 Traffic equations 287

∗ possess a unique positive solution q2. By symmetry, (4.2a), (4.2b) posses ∗ ∗ ∗ a unique positive solution q1 too. Insertion of q2 into (4.2c) gives that q1 ∗ corresponds to q2 which implies that the traffic equations possess a unique ∗ ∗ positive solution q1 = q1, q2 = q2. Second, consider the case r(2) = 0, i.e. no service at station 2. The irreducibility assumption implies that λ(2) > 0 or r(1)p−(1, 2) > 0 and r(1) > 0 or λ(1) > 0 as both at station 1 and station 2 in some staten ¯ the number of customers must be able to decrease. Then, (4.2a) and (4.1a) Λ(1) show that q1 = r(1)+λ(1) > 0. Insertion of q1 into (4.2b) gives that q2 > 0 − + since q1r(1)p (1, 2)+λ(2) > 0 by irreducibility and q1r(1)p (1, 2)+Λ(2) > 0 from (4.1b). Thus the traffic equations posses a unique positive solution q1, q2. Similarly, if r(1) = 0 the traffic equations possess a unique positive solution q1, q2 too. Now assume that r(1) > 0 and r(2) > 0. Then (4.2c) is well-defined 2 1 0 and q2 must satisfy (4.3). If the coefficients of γ2 , γ2 and γ2 are non-zero then (4.3) possesses a unique positive solution q2. Insertion of γ2 > 0 into (4.2c) gives that γ1 > 0 since the network is assumed to be non- pathological. Insertion of γ2 < 0 into

+ γ1r(1)p (1, 2) + Λ(2) γ2 = − , r(2) + γ1r(1)p (1, 2) + λ(2) which is obtained from (4.2b) and is well-defined since r(2) > 0, shows that γ1 < 0. As the traffic equations possess at most two distinct solutions q1, the traffic equations possess a unique positive solution q1, q2 if the 2 0 coefficients in (4.3) are non-zero. If the coefficients of γ2 or γ2 vanish, however, this must be shown separately. 2 − + − The coefficient of γ2 is zero if both p (2, 1) = 0 and p (2, 1)p (1, 2) = − − 1 0. If p (2, 1) = 0 and p (1, 2) = 0 then the coefficient of γ2 is non- 0 negative. As the coefficient of γ2 is non-positive, it must be that q2 ≥ 0 and from (4.2c) that q1 ≥ 0. Insertion of q1 = 0 into (4.2a) implies that + q2r(2)p (2, 1) + Λ(1) = 0. As q2 ≥ 0 this implies that Λ(1) = 0. Then, since the network is assumed to be non-pathological, r(2)p+(2, 1) > 0 and thus q2 = 0 which implies that Λ(2) = 0 which contradicts (4.1c). − + 1 Similarly, if p (2, 1) = 0 and p (2, 1) = 0 the coefficient of γ2 is non- negative and (4.2a), (4.2b) possess a unique positive solution q1, q2. From 0 (4.1c) the coefficient of γ2 vanishes only if r(1) = λ(1) = 0. 2

Note that all combinations (q1 > 1, q2 > 1), (q1 > 1, q2 < 1), etc. are possible. This can be seen in the following example. Assume that 288 Negative customers

λ(1) = λ(2) = p+(1, 2) = p+(2, 1) = d(1) = d(2) = 0 and r(1) = r(2) = 1, then (4.3) reads

q1(1 + q2) = Λ(1), q2(1 + q1) = Λ(2).

The following table gives some possible combinations.

Λ(1) Λ(2) q1 q2 3 3 1 1 4 4 2 2 3 1 2 1 1 2 3 1 3 2 2 2

As is shown in the simple case of a two station queueing network it is very difficult to prove that a unique positive solution to the traffic equations exists. Moreover, in this special case it must be assumed that the queueing network is non-pathological. Thus, in general, it does not seem to be possible to prove the existence and uniqueness of a positive solution to the traffic equations without further assumptions. In some special cases this problem is solved (cf. [37]). These cases are reviewed below in Section 4.2, where feedforward networks are discussed, and Section 4.3, which considers balanced networks. As is shown in Section 4.4, it is necessary for the existence of a positive solution to the traffic equations that the queueing network is not pathological.

4.2 Feedforward networks As is argued above, general conditions for the existence and uniqueness N of a positive solution {qi}i=1 to the traffic equations are hard to give. For an important class of networks, so-called feedforward networks, such a solution exists (cf. [37]). A network is said to be feedforward if the stations can be numbered such that p+(i, j) = 0 if j < i and p−(i, j) = 0 if j < i, i.e. if customers can go from station i to stations with a larger number only. For such a N network {qi}i=1 can be calculated recursively (cf. [37]). From the traffic equations Λ(1) q = . 1 r(1) + λ(1) 10.4 Traffic equations 289

If qi, i = 1, . . . , s − 1, are known then qs can be calculated from

X + qjr(j)p (j, s) + Λ(s) j

~~As all terms r(1), r(2), etc. are non-negative, and the calculating procedure is unique, this gives a unique positive solution to the traﬃc equations.~~

4.3 Balanced networks A very special class of networks considered in [37] is the class of balanced networks. A network with negative and positive customers is balanced if a constant solution qi = q, i = 1,...,N, exists to the traffic equations. If such a solution exists, then a positive constant solution exists and this positive constant solution is the unique positive constant solution to the traffic equations (cf. [37]). Be aware, however, that this does not imply that it is the unique positive solution to the traffic equations. Assume that the network is balanced. Insertion of qi = q, i = 1,...,N, into the traffic equations gives (cf. [37])

N X r(j)p+(j, i) − λ(i) − r(i) q = j=1 N 2 X r(j)p−(j, i) j=1 1  2  2  N  N  X + X −   r(j)p (j, i) − λ(i) − r(i) + 4 r(j)p (j, i)Λ(i) j=1 j=1 + . N 2 X r(j)p−(j, i) j=1

~~Note that the right-hand side is independent of i as q is independent of i. Further note that q can take arbitrary positive values as can be seen by considering the right-hand side. If~~

~~N N 2 X r(j)p−(j, i) < X r(j)p+(j, i) − λ(i) − r(i), j=1 j=1 290 Negative customers then q > 1, otherwise if~~

N N r(i) + X r(j)p−(j, i) + λ(i) > Λ(i) + X r(j)p+(j, i) (4.4) j=1 j=1 then q < 1, if (4.4) holds with equality then q = 1, and if > is replaced by < then q > 1. Note that (4.4) expresses that the departure rate at station i is larger than the arrival rate at station i. In all cases, {N} possesses an invariant measure m given by

~~PN n m(¯n) = q i=1 i .~~

4.4 Pathological network The assumption that the queueing network is not pathological made in Section 4.1 is a very natural assumption. Assume that the network is pathological, then positive customers cannot enter the stations, i.e. Λ(i) = 0, for all i, and r(j)p+(j, i) = 0, for all i, j. If this is the case then the number of customers at the stations cannot increase, thus in case of a lower bound π((N1,...,NN )) = 1, and in case of no lower bounds the process is transient, i.e. {N} does not possess an equilibrium distribution. Further note that the assumption that the network is non-pathological is necessary for the existence of a positive solution to the traﬃc equations. To this end, assume that the network is pathological. Then the traﬃc equations read

 N  X − qi r(i) + qjr(j)p (j, i) + λ(i) = 0, i = 1,...,N, j=1 with solution N X − qi = 0 or r(i) + qjr(j)p (j, i) + λ(i) = 0, i = 1,...,N. j=1 Note that r(i) = λ(i) = 0 for all i implies that there are no customers routing in the network, therefore it must be the case that r(i) + λ(i) > 0 for some i, say i0. Then for i0 N X − qjr(j)p (j, i0) = −r(i0) − λ(i0) < 0. j=1 10.4 Traﬃc equations 291

~~If there exists a non-zero solution, it must be the case that qk < 0 for some k.~~

4.5 Characteristic equations As is illustrated above, for the Markov chain to possess a product-form equilibrium distribution, the traffic equations completely characterize the process. In Sections 4.1–4.4 some insight is given in the behaviour of the traffic equations and more specifically in the existence of a unique positive solution to the traffic equations. As is observed in these sections, some special conditions, additional to the conditions imposed in a standard queueing network are necessary for the existence of a unique positive solution to the traffic equations. This section argues that the traffic equations are characteristic equations for the Markov chain describing the queueing network in the sense that they determine not only the equilibrium distribution, but also determine the shape of the state space and the blocking conditions at the boundaries of the state space. To this end first note that the Markov chain possesses a product-form invariant measure N only if the traffic equations possess a non-negative solution, {qi}i=1, as the invariant measure must satisfy the global balance equations at the interior of the state space. Second, the value of qi, i = 1,...,N, characterizes the state space at which the invariant measure can be normalized. QN nk To this end, note that for m(¯n) = k=1 qk to be normalizable, if qi < 1 the state space must be such that −∞ < Ni ≤ ni, if qi > 1 the state space must be such that ni ≤ Mi < ∞, and if qi = 1 it must be the case that −∞ < Ni ≤ ni ≤ Mi < ∞. Third, for the Markov chain to possess a product-form equilibrium distribution at a truncated state space, the transition rates must be modified as given in (3.1) and (3.9). These mod- N ifications involve {qi}i=1, the solution to the traffic equations. Therefore, the traffic equations are characteristic equations for the Markov chain as they determine the state space and the equilibrium distribution. This is formalized in Theorem 4.2 below. The proof is omitted since it is a combination of the proofs of Theorems 3.1, and 3.5. Note that the corner points n¯ = (n1, . . . , nN ) such that ni = Ni for some i and nj = Mj for some j 6= i must be checked explicitly since these points are not considered in Theorems 3.1 and 3.5.

~~Theorem 4.2 (Characteristic result) Assume a non-negative solution 292 Negative customers~~

~~N {qi}i=1 exists to~~

~~N N X − X + γir(i)+ γiγjr(j)p (j, i)+γiλ(i) = γjr(j)p (j, i)+Λ(i), i = 1,...,N, j=1 j=1 (4.5) such that with I<,I=,I> a partition of {1,...,N}~~

~~qi < 1 if i ∈ I<, qi = 1 if i ∈ I=, qi > 1 if i ∈ I>.~~

~~For Ni,Mi ∈ Z, i = 1,...,N, such that~~

~~Ni > −∞ for i ∈ I< ∪ I=, Mi < ∞ for i ∈ I> ∪ I=, the Markov chain, {N}, with transition rates Q = (q(¯n, n¯0), n,¯ n¯0 ∈ ZN ) deﬁned as~~

 Λ(i) + PN q r(k)p+(k, i)1{n = M }, n¯0 =n ¯ + e ,  k=1 k k k i    ni < Mi,    r(i)d(i) + λ(i)    PN −  + j=1 r(i)p (i, j)1{nj = Nj}   N  + P q r(k)p−(k, i)1{n = M }  k=1 k k k   PN + 0  + k=1 r(i)p (i, k)1{nk = Mk}, n¯ =n ¯ − ei, q(¯n, n¯0)= n > N ,  i i    r(i)p+(i, j), n¯0 =n ¯ − e + e ,  i j    ni > Ni, nj < Mj,    − − 0  r(i)p (i, j) + r(j)p (j, i), n¯ =n ¯ − ei − ej,    ni > Ni, nj > Nj,    0, otherwise, (4.6) possesses an equilibrium distribution π at state space

~~N S = {n¯ ∈ Z : Ni ≤ ni ≤ Mi, i = 1,...,N} 10.5 Example: two station queueing network 293 given by~~

~~N Y 1 − qi Y 1 − qi Y 1 − qi Y π(¯n) = qnk , n¯ ∈ S. Ni Ni Mi+1 Mi+1 k i∈I< qi i∈I= qi − qi i∈I> −qi k=1~~

~~5 Two station queueing network~~

Reconsider the two station queueing network discussed in Example 1.1 and depicted in Figure 10.1. At both stations external arrivals of positive customers at rate Λi at station i, i = 1, 2, occur only. At the stations service is provided in FCFS order by a single server at negative-exponential rate ri at station i, i = 1, 2. Customers completing service at station 1 route to station 2 as negative customers and customers completing service at station 2 route to station 1 as negative customers. The parameters for this queueing network are

~~Λ(1) = Λ1 > 0, Λ(2) = Λ2 > 0, λ(1) = 0, λ(2) = 0,~~

~~r(1) = r1 > 0, r(2) = r2 > 0, p−(1, 2) = 1, p−(2, 1) = 1, p+(1, 2) = 0, p+(2, 1) = 0, d(1) = 0, d(2) = 0, and the traﬃc equations read~~

~~γ1r(1) + γ1γ2r(2) = Λ(1), (5.1a)~~

γ2r(2) + γ1γ2r(1) = Λ(2). (5.1b) The traffic equations represent generalized station balance for stations 1 and 2. To this end note that (5.1a) states that the rate into station 1 due to external arrivals, Λ(1), equals the rate out of station 1 due to service-completions at station 1, γ1r(1), and due to arrivals from station 2, γ1γ2r(2). In Figure 10.6 the traffic equations are expressed graphically. For i = 1 and i = 2 local balance at staten ¯ and local balance at state n¯ − ei are depicted. Global balance at staten ¯ is obtained by adding these 4 local balance relations. Combining local balance at staten ¯ for i = 1 and local balance at staten ¯ − e2 for i = 2 gives “half of” global balance. This is depicted in the bottom line of Figure 10.6. Thus an additional 294 Negative customers type of local balance for this specific model is obtained. This type of local balance is used below to further reduce the state space, i.e, to introduce additional boundaries to the state space. Since all parameters Λ1, Λ2, r1, r2 are positive, the process is irreducible. Theorem 3.5 implies that the traffic equations posses a unique positive solution q1, q2. As a consequence the queueing network possesses a unique invariant measure m at state space S = ZN given by

~~n1 n2 m(¯n) = q1 q2 . (5.2)~~

~~(4.3) implies that q2 < 1 if and only if~~

~~r(2)2 + r(1)r(2) + Λ(1)r(1) − Λ(2)r(2) − Λ(2)r(1) > 0, (5.3a) and by symmetry that q1 < 1 if and only if~~

~~r(1)2 + r(1)r(2) + Λ(2)r(2) − Λ(1)r(1) − Λ(1)r(2) > 0, (5.3b) 10.5 Example: two station queueing network 295~~

~~q1q2r(2) q2r(2)~~

~~d d~~

~~i = 1, staten ¯ i = 1, staten ¯ − e1~~

~~Λ(1) = q1r(1) + q1q2r(1)r(2)~~

~~= 2, staten ¯ − e2 i = 2, stated ni ¯~~

~~Λ(2) = q2r(2) + q1q2r(1)r(1)~~

~~q1q2r(2) Λ(2) 6 q1q2r(1)~~

~~Figure 10.6. Local balance for queueing network of Figure 10.1 296 Negative customers~~

~~b S~~

~~(0,0)~~

~~Figure 10.7. State space~~

If the coeﬃcients Λ(1), Λ(2), r(1), r(2) satisfy (5.3a), (5.3b) then Theo- rem 3.1 gives that for N1,N2 ∈ Z the process possesses a unique equilibrium distribution at S = {n¯ : n1 ≥ N1, n2 ≥ N2}. For N1 = N2 = 0

~~n1 n2 π(¯n) = (1 − q1)q1 (1 − q2)q2 , n1, n2 ≥ 0.~~

If (5.3a), (5.3b) are satisﬁed with > replaced by < then q1 > 1 and q2 > 1, and Theorem 4.1 can be applied to produce a unique equilibrium distribution at a state space that is bounded from above. In addition to these types of blocking, observation of local balance depicted in Figure 10.6 shows that, without additional stringent assumptions on the transition rates, the state space can be reduced to a band as depicted in Figure 10.7. To this end note that combination of local balance for i = 1 with reference staten ¯ and i = 2 with reference state n¯ − e2 shows that for transitions on the right-hand side of the diagonal the rate inton ¯ balances with the rate out ofn ¯. Similarly for the left-hand side of the diagonal. At boundary a service at station 2 is stopped and at boundary b service at station 1 is stopped to preserve a product-form equilibrium distribution. Chapter 11

~~Concluding remarks~~

This monograph has discussed the queue-length distribution for queueing networks. Assuming that a single type of customers is present and that customers require a negative-exponentially distributed amount of service at the stations, the class of queueing networks that possess a product-form queue-length distribution is characterized. For queueing networks not in equilibrium it is shown that the queue-length distribution is of product- form if and only if the queueing network consists of inﬁnite-server queues only. For queueing networks in equilibrium under general service characteristics but restrictive routing characteristics the equilibrium distribution is shown to be of product-form. Moreover, these conditions on the routing characteristics are shown to be necessary for the queueing network to have a product-form equilibrium distribution. The product-form results obtained in this monograph can immediately be generalized to include diﬀerent customer-types by simply adding components to the state-vector for each additional customer-type. Furthermore, as is discussed in various references, in the case of single changes queueing networks both service- disciplines and, for some service disciplines, generally distributed service- requirements can be included as a direct consequence of local balance. Other than the obvious generalizations presented above, the product-form results obtained in this monograph cannot be further generalized. Al- though service and routing characteristics used in this monograph include batch routing and batch service, blocking of transitions, cluster-dependent routing and service, congestion-dependent routing, the class of queueing networks that possess a product-form equilibrium distribution is a limited class. A queueing network has a product-form equilibrium distribution if for

297 298 Concluding remarks each station separately the physical rate in is balanced by the physical rate out, that is the probability flow for customers leaving a station equals the probability flow for customers entering that station. This flow balance is usually characterized by the appropriate set of underlying traffic equations, also if blocking of transitions is involved. The following equivalence statements basically cover all product-form results.

~~product-form physical interpretation of ⇐⇒ equilibrium distribution rate in = rate out appropriate notion of ⇐⇒ local balance appropriate set of ⇐⇒ traﬃc equations~~

~~The following table summarizes the results on product-form queueing networks obtained in this monograph. References 299~~

CH3 Transient behaviour General queueing networks Lemmas 2.1, 2.2 Canonical forms Theorems 3.1, 4.3, 5.2 Transient distribution is of product-form if and only if the network consists of infinite-server queues only CH4 Transient behaviour Engset loss model Corollary 3.4 Transient distribution is the sum of two product-form distributions CH5 Equilibrium behaviour Single changes: dual processes Definition 3.1 Definition of the dual process Theorem 3.2 Dual equilibrium distribution Theorem 5.4 Arrival theorem CH6 Equilibrium behaviour Single changes: Norton’s theorem Assumption 4.1 Decomposition into clusters of stations Conditions 4.2, 4.3 Global and local traffic equations Theorem 4.4 The routing characteristics decompose into a global part and a local part Theorems 5.1, 5.2 Norton’s theorems: the process decomposes into a global and a local process CH7 Equilibrium behaviour Single changes: amalgamation of processes Definition 2.1 Definition of the amalgamated process Definition 2.3 Cross-balance Theorem 2.5 Equilibrium distribution CH8 Equilibrium behaviour Batch routing: strong reversibility Definition 2.1 Symmetry property Definition 2.3 Strong reversibility Theorem 2.4 Characterization of strongly reversible processes with a product-form equilibrium distribution CH9 Equilibrium behaviour Batch routing: group-local-balance Definition 2.2 Definition of group-local-balance Definition 2.5 Definition of theq ¯-process Theorem 2.7 Characterization of the equilibrium distribution in terms of the equilibrium distribution of theq ¯-process Theorem 3.2 Decomposition theorem: the equilibrium distribution factorizes into a service part and a routing part CH10 Equilibrium behaviour Negative customers Theorem 2.2 Product-form invariant measure and introduction of the traffic equations Definition 2.3 Definition of local balance for queueing networks with negative customers Theorems 3.1, 3.5 Product-form equilibrium distribution at restricted state spaces Lemma 3.4 Product-form result cannot be obtained from standard local balance 300 References References

~~[1] Asmussen, S. (1987) Applied probability and queues. Wiley.~~

~~[2] Baccelli, F. and Br´emaud,P. (1987) Palm probabilities and stationary queues. Lecture Notes in Statistics 41, Springer-Verlag.~~

~~[3] Balsamo, S. and Iazeolla, G. (1982) An extension of Norton’s theorem for queueing networks. IEEE Trans. on Software Eng. 8, 298-305.~~

~~[4] Baskett, F., Chandy, K. M., Muntz, R. R. and Palacios, F. G. (1975) Open, closed and mixed networks of queues with diﬀerent classes of customers. J. A. C. M. 22, 248-260.~~

~~[5] Bean, N. (1988) Honours thesis. The University of Western Australia.~~

~~[6] Belevitch, V. (1968) Classical network theory. Holden-Day.~~

~~[7] Bertsekas, D. P. and Gallager, R. G. (1987) Data networks. Prentice- Hall.~~

~~[8] Boucherie, R. J. and Van Dijk, N. M. (1990) Spatial birth-death processes with multiple changes and applications to batch service networks and clustering processes. Adv. Appl. Prob. 22, 433-455.~~

~~[9] Boucherie, R. J. and Van Dijk, N. M. (1991) Product forms for queueing networks with state-dependent multiple job transitions. Adv. Appl. Prob. 23, 152-187.~~

~~[10] Brandt, A. (1990) On Norton’s theorem for multi-class queueing networks of quasi-reversible nodes. Preprint Nr. 256 Humboldt- Universit¨atzu Berlin, Sektion Mathematik.~~

~~301 302 References~~

[11] Brockmeyer, E., Halstrøm, H. L. and Jensen, A. (1948) The life and works of A. K. Erlang. Transactions of the Danish Academy of Tech- nical Sciences. 2. [12] Burke, P. J. (1956) The output of a queueing system. Operations Research 4, 699-704. [13] Chandy, K. M., Herzog, U. and Woo, L. (1975) Parametric analysis of queueing networks. IBM J. Res. Develop. 19, 36-42. [14] Chandy, K. M., Howard, J. H. and Towsley, D. F. (1977) Product form and local balance in queueing networks. J. A. C. M. 24, 250- 263. [15] Chandy, K. M. and Martin, A. J. (1983) A characterization of product-form queueing networks. J. A. C. M. 30, 286-299. [16] Chung, K. L. (1967) Markov chains with stationary transition probabilities. 2nd ed. Springer-Verlag. [17] Cohen, J. W. (1982) The single server queue. 2nd ed. North-Holland. [18] Cohen, J. W. and Boxma, O. J. (1985) A survey of the evolution of queueing theory. Statistica Neerlandica 39, 143-158. [19] Conway, A. E. and Georganas, N. D. (1986) Decomposition and aggregation by class in closed queueing networks. IEEE Trans. on Soft- ware Eng. 12, 1025-1040. [20] Conway, A. E. (1990) A new method of parametric analysis for product-form queueing networks. Performance ’90 (Eds. P. J. B. King, I. Mitrani and R. J. Pooley), North-Holland. [21] Cooper, R. B. (1981) Introduction to queueing theory. 2nd ed. North- Holland. [22] Courtois, P. J. (1977) Decomposability: queueing and computer system applications. Academic Press. [23] Daduna, H. and Schassberger, R. (1983) Networks of queues in discrete time. Zeitschrift f¨urOperations Research 27, 159-175. [24] Van Dijk, N. M. (1988) On Jackson’s product form with ‘jump-over’ blocking. Operations Research Letters 7, 233-235. References 303

~~[25] Van Dijk, N. M. (1991) Product forms for random access schemes. Computer Networks and ISDN-systems 22, 303-317.~~

~~[26] Van Dijk, N. M. (1991) “Stop = recirculate” for exponential queueing networks with departure blocking. Operations Research Letters 10, 343-351.~~

~~[27] Van Dijk, N. M. (1991) Product forms for queueing networks with limited clusters. Research Report, Free University, Amsterdam~~

~~[28] Van Dongen, P. G. J. and Ernst, M. H. (1984) Kinetics of reversible polymerization. J. Stat. Phys. 37, 301-324.~~

~~[29] Van Dongen, P. G. J. and Ernst, M. H. (1987) Fluctuations in coag- ulating systems. J. Stat. Phys. 49, 879-926.~~

~~[30] Engset, T. (1918) Die Wahrscheinlichkeitsrechnung zur Bestimmung der W¨ahleranzahlin automatischen Fernsprech¨amtern. Elektrotech Z. 31, 304-306.~~

[31] Ernst, M. H. (1984) Kinetic theory of clustering. In: International Summer School on Fundamental Problems in Statistical Mechanics, VI Trondheim, Norway, June 1984 (Eds. E. G. D. Cohen) North- Holland.

~~[32] Foley, R. D. (1982) The non-homogeneous M/G/∞ queue. Opsearch 19, 40-48.~~

~~[33] Foschini, G. J. and Gopinath, B. (1983) Sharing memory optimally. IEEE Trans. Commun. 31, 352-360.~~

~~[34] Foster, F. G. (1953) On the stochastic matrices associated with certain queueing processes. Ann. Math. Statist. 24, 355-360.~~

~~[35] Gelenbe, E. (1989) Random neural networks with negative and positive signals and product form solution. Neural Computation 1, 502- 510.~~

~~[36] Gelenbe, E., Glynn, P. and Sigman, K. (1991) Queues with negative arrivals. J. Appl. Prob. 28, 245-250.~~

~~[37] Gelenbe, E. (1991) Product-form queueing networks with negative and positive customers. J. Appl. Prob. 28, 656-663. 304 References~~

~~[38] Gordon, W. J. and Newell, G. F. (1967) Closed queueing systems with exponential servers. Operations Research 15, 254-265.~~

~~[39] Harrison, J. H. and Lemoine, A. J. (1981) A note on networks of inﬁnite-server queues. J. Appl. Prob. 18, 561-567.~~

~~[40] Henderson, W., Pearce, C. E. M., Taylor, P. G. and Van Dijk, N. M. (1990) Closed queueing networks with batch services. Queueing Sys- tems 6, 59-70.~~

~~[41] Henderson, W. and Taylor, P. G. (1990) Product form in networks of queues with batch arrivals and batch services. Queueing Systems 6, 71-88.~~

~~[42] Henderson, W. and Taylor, P. G. (1991) Some new results on queueing networks with batch movement. J. Appl. Prob. 28, 409-421.~~

~~[43] Henderson, W. and Taylor, P. G. (1991) Discrete-time queueing networks with geometric release probabilities. Research Report~~

~~[44] Hordijk, A. and Van Dijk, N. M. (1981) Networks of queues with blocking. Performance ’81 (Ed. F. J. Kylstra), 51-65, North-Holland.~~

[45] Hordijk, A. and Van Dijk, N. M. (1982) Stationary probabilities for networks of queues. Applied Probability - Computer Science: The Interface. (Eds. L. Disney and T. J. Ott), Birkh¨auser,Vol. II, 423- 451.

[46] Hordijk, A. and Van Dijk, N. M. (1983) Networks of queues. Part I: Job-local-balance and the adjoint process. Part II: General routing and service characteristics. Lecture notes in Control and Informa- tional Sciences. Springer-Verlag. Vol. 60, 158-205.

~~[47] Hordijk, A. and Van Dijk, N. M. (1983) Adjoint processes, job local balance and insensitivity for stochastic networks. Bull. Inst. Int. Statist. 50, 776-788.~~

~~[48] Jackson, J. R. (1957) Networks of waiting lines. Operations Research 5, 518-521.~~

~~[49] Jackson, J. R. (1963) Jobshop-like queueing systems. Management Science 10, 131-142. References 305~~

~~[50] Van Kampen, N. G. (1981) Stochastic processes in physics and chemistry. North-Holland.~~

~~[51] Kaufman, J. S. (1981) Blocking in a shared resource environment. IEEE Trans. Commun. 29, 1474-1481.~~

~~[52] Kelly, F. P. (1975) Networks of queues with customers of diﬀerent types. J. Appl. Prob. 12, 542-554.~~

~~[53] Kelly, F. P. (1975) Markov processes and Markov random ﬁelds. Bull. Inst. Int. Statist. 46, 397-404.~~

~~[54] Kelly, F. P. (1976) Networks of queues. Adv. Appl. Prob. 8, 416-432.~~

~~[55] Kelly, F. P. (1979) Reversibility and stochastic networks. Wiley.~~

~~[56] Kingman, J. F. C. (1969) Markov population processes. J. Appl. Prob. 6, 1-18.~~

~~[57] Kritzinger, P. S., Van Wyk, S. and Krzesinski, A. E. (1982) A gener- alisation of Norton’s theorem for multiclass queueing networks. Per- formance Evaluation 2, 98-107.~~

~~[58] Lam, S. S. (1977) Queueing networks with population size constraints. IBM J. Res. Develop. 21, 370-378.~~

~~[59] Lushnikov, A. A. (1978) Coagulation in ﬁnite systems. J. Coll. Interf. Sci. 65, 276-285.~~

~~[60] Pittel, B. (1979) Closed exponential networks of queues with satu- ration: the Jackson-type stationary distribution and its asymptotic analysis. Math. Operations Research 4, 357-378.~~

~~[61] Pollett, P. K. (1986) Connecting reversible Markov processes. Adv. Appl. Prob. 18, 880-900.~~

~~[62] Pollett, P. K. (1990) A note on the classiﬁcation of Q-processes when Q is not regular. J. Appl. Prob. 27, 278-290.~~

~~[63] Pollett, P. K. (1991) Invariant measures for Q-processes when Q is not regular. Adv. Appl. Prob. 23, 277-292. 306 References~~

[64] Pujolle, G., Claude, J. P. and Seret, D. (1985) A discrete tandem queueing system with a product form solution. Proc. Intern. Semi- nar on Computer Networking and Performance Evaluation. North- Holland, Kyoto, 139-147.

~~[65] Pujolle, G. (1988) Discrete-time queueing systems with a product form solution. MASI Research Report, July 1988~~

[66] Pujolle, G. (1991) Discrete-time queueing systems for data networks performance evaluation. Queueing performance and control in ATM (ITC-13) (Eds. J. W. Cohen and C. D. Pack) North-Holland, 239-244

~~[67] Reuter, G. E. H. (1957) Denumerable Markov processes and the associated contraction semigroups on l. Acta Math. 97, 1-46.~~

~~[68] Riordan, J. (1962) Stochastic service systems. Wiley.~~

~~[69] Sanchez, D. A. (1968) Ordinary diﬀerential equations and stability theory: an introduction. W. H. Freeman and Company.~~

~~[70] Schassberger, R. (1977) Insensitivity of steady-state distributions of generalised semi-Markov processes. Part I. Ann. Prob. 5, 87-99.~~

~~[71] Schassberger, R. (1978) Insensitivity of steady-state distributions of generalised semi-Markov processes. Part II. Ann. Prob. 6, 85-93.~~

~~[72] Schassberger, R. (1978) The insensitivity of stationary probabilities in networks of queues. Adv. Appl. Prob. 10, 906-912.~~

~~[73] Schassberger, R. and Daduna, H. (1983) A discrete-time technique for solving closed queueing models of computer systems. Technical Report. Technische Universit¨atBerlin.~~

~~[74] Schrijver, A. (1986) Theory of linear and integer programming. Wiley.~~

~~[75] Seneta, E. (1981) Non-negative matrices and Markov chains. Springer-Verlag.~~

~~[76] Serfozo, R. F. (1989) Markovian network processes: congestion dependent routing and processing. Queueing Systems 5, 5-36.~~

~~[77] Stockmayer, W. H. (1943) Theory of molecular size distributions and gel formation in branched chain polymers. J. Chem. Phys. 11, 45-55. References 307~~

~~[78] Strang, G. (1988) Linear algebra and its applications. Harcourt Brace Javonovich, Inc.~~

~~[79] Vantilborgh, H. (1978) Exact aggregation in exponential queueing networks. J. A. C. M. 25, 620-629.~~

~~[80] Walrand, J. and Varaiya, P. (1980) Interconnections of Markov chains and quasi-reversible queueing networks. Stoch. Proc. Appl. 10, 209- 219.~~

~~[81] Walrand, J. (1983) A discrete-time queueing network. J. Appl. Prob. 20, 903-909.~~

~~[82] Walrand, J. (1983) A note on Norton’s theorem for queueing networks. J. Appl. Prob. 20, 442-444.~~

~~[83] Walrand, J. (1988) An introduction to queueing networks. Prentice- Hall.~~

~~[84] Whittle, P. (1967) Nonlinear migration processes. Bull. Inst. Int. Statist. 42, 642-647.~~

~~[85] Whittle, P. (1968) Equilibrium distributions for an open migration process. J. Appl. Prob. 5, 557-571.~~

~~[86] Whittle, P. (1985) Partial balance and insensitivity. J. Appl. Prob. 22, 168-176.~~

~~[87] Whittle, P. (1986) Systems in stochastic equilibrium. Wiley.~~

~~[88] Wolﬀ, R. W. (1989) Stochastic modeling and the theory of queues. Prentice-Hall.~~

~~[89] Yao, D. D. and Buzacott, J. A. (1987) Modeling a class of ﬂexible manufacturing systems with reversible routing. Operations Research 35, 87-93. 308 References Samenvatting~~

~~Produktvorm in netwerken van wachtrijen~~

In een groot aantal realistische systemen moeten klanten of taken wachten op bediening of gaan klanten verloren als het bedieningssysteem bezet is. Voorbeelden van dergelijke systemen zijn een supermarkt waar klanten in de rij bij de kassa moeten wachten tot ze aan de beurt zijn, een telefooncentrale waar een oproep verloren gaat als de centrale bezet is, een machine waar pallets wachten om bewerkt te worden en een centrale processor die zijn tijd verdeelt tussen de aanwezige computeropdrachten. Een netwerk van wachtrijen is het wiskundige model voor een systeem waarin opeen- hoping van klanten kan optreden. Het basiselement van een netwerk van wachtrijen is een wachtrij of station. Klanten komen aan bij een station om een bepaalde dienst te ontvangen. In het wiskundige model worden zowel de aankomststroom van klanten als de gevraagde hoeveelheid bediening stochastisch verondersteld, dat wil zeggen dat toevallige eﬀecten een rol kunnen spelen. Na bediening vertrekt de klant naar een andere wachtrij in het netwerk of verlaat de klant het netwerk. Vele praktische systemen kunnen als netwerk van wachtrijen gemodelleerd worden. Voor- beelden hiervan zijn een telefoonnetwerk, een produktielijn bestaande uit meerdere machines en een computer bestaande uit een centrale processor, geheugenbanken en I/O-poorten. Typische grootheden die van belang zijn om het gedrag van praktische systemen te analyseren zijn de rijlengte (het aantal klanten dat staat te wachten bij een station), de verblijftijd (de tijd die een klant door- brengt bij een station), en de doorzet (het aantal klanten dat per tijds- eenheid geholpen wordt bij een station). Deze prestatiematen kunnen berekend worden uit de verdeling van de rijlengte. In deze monograﬁe ligt de nadruk op de bepaling van exacte analytische uitdrukkingen voor de rijlengteverdeling.

~~309 310 Samenvatting~~

De evolutie van een netwerk van wachtrijen wordt bepaald door de overgangsmatrix. Uit de overgangsmatrix kan in principe de rijlengteverdeling worden gekarakteriseerd: gegeven de beginverdeling (de toestand van het netwerk als het wordt opgestart), kan de verdeling op ieder volgend tijdstip uit de overgangsmatrix bepaald worden. Echter, voor algemene netwerken van wachtrijen is het aantal rekenoperaties dat uitgevoerd moet worden om de tijdsafhankelijke rijlengteverdeling te bepalen astronomisch groot. Om deze reden ligt het voor de hand om te zoeken naar gesloten analytische uitdrukkingen, d.w.z. formules die op relatief eenvoudige wijze uit de parameters van het netwerk van wachtrijen bepaald kunnen worden. Over netwerken van wachtrijen is bekend dat voor een grote klasse van netwerken de evenwichtsverdeling van de rijlengte een produktvorm heeft, d.w.z. als de effecten van de begintoestand uitgewerkt zijn, dan heeft de verdeling van de rijlengte een produktvorm. Deze produktvorm is een eenvoudige uitdrukking voor de evenwichtsverdeling, die in principe alleen bestaat uit een produkt van termen, éénvoor iedere wachtrij in het netwerk. Bijgevolg kunnen prestatiematen voor netwerken in de klasse met een produktvorm voor de evenwichtsverdeling eenvoudig geanalyseerd worden. In hoofdstuk 3 van deze monografie wordt aangetoond dat voor netwerken van wachtrijen die niet in evenwicht zijn een produktvorm verdeling voor de rijlengte dan en alleen dan verkregen kan worden als de klanten in het netwerk elkaar niet be¨ınvloeden, d.w.z. als het netwerk alleen bestaat uit wachtrijen die aan iedere aanwezige klant een aparte bediende toe- wijzen. In hoofdstuk 4 wordt het tijdsafhankelijke gedrag van het Engset verlies model geanalyseerd. Het Engset verlies model beschrijft het gedrag van een telefooncentrale met s lijnen en N abonnees. Op stochastische tijdstippen besluit een abonnee op te bellen. Zolang niet alle s lijnen bezet zijn wordt het telefoongesprek door de centrale doorgeschakeld en blijft de abonnee gedurende een stochastische tijd in gesprek. Als alle s lijnen bezet zijn dan hoort de abonnee de ingesprektoon en verbreekt de verbinding. Voor dit model wordt in hoofdstuk 4 aangetoond dat in een zeer speciaal geval de tijdsafhankelijke rijlengteverdeling een som is van twee produktvormen. De resterende hoofdstukken van deze monografie beschrijven het evenwichtsgedrag van netwerken van wachtrijen. Een netwerk van wachtrijen is in evenwicht als in iedere toestand van het netwerk de waarschijnlijkheidsstroom uit die toestand gelijk is aan de waarschijnlijkheidsstroom in die toestand. Het evenwichtsconcept voor netwerken van wachtrijen is Produktvorm in netwerken van wachtrijen 311 gebaseerd op een meer intu¨ıtieve vorm van evenwicht, namelijk lokaal evenwicht of stationsevenwicht: in evenwicht is in iedere toestand de stroom van klanten uit een station even groot als de stroom van klanten in een station. Deze vorm van evenwicht blijkt nodig en voldoende te zijn om een produktvorm voor de evenwichtsverdeling te verkrijgen. Lokaal evenwicht en de produktvorm voor de evenwichtsverdeling worden bestudeerd in hoofdstukken 5 – 10. In het bijzonder worden de grenzen van de klasse van netwerken van wachtrijen die een produktvorm voor de evenwichtsverdeling hebben afgetast en vastgelegd. De nadruk ligt hierbij op het optreden van blokkering: tengevolge van capaciteitsbeperkingen bij de stations kunnen aankomende klanten niet toegelaten worden tot het station van hun keuze. In hoofdstuk 5 wordt de transitiestructuur van netwerken van wachtrijen bestudeerd. De toestand van een netwerk van wachtrijen wordt gegeven door het aantal klanten bij de wachtrijen in het netwerk. In een transitie springt een klant van een wachtrij naar een andere wachtrij. Gedurende deze transitie loopt de klant een moment vrij rond in het netwerk en neemt de toestand van het netwerk waar. Deze waargenomen toestand wordt een duale toestand genoemd, en het proces dat de evolutie van de duale toestanden beschrijft wordt een duaal proces genoemd. Dit duale proces wordt geanalyseerd in hoofdstuk 5. Als belangrijkste toepassing van dit duale proces wordt een generalisatie van de aankomststelling gegeven. Deze stelling houdt ruwweg in dat een klant die van een wachtrij naar een andere wachtrij springt het netwerk waarneemt als- of het in evenwicht is en hij zelf niet in het netwerk aanwezig is. De aankomststelling is van bijzonder belang voor de kwantitatieve analyse van netwerken van wachtrijen, omdat verschillende benaderingstechieken hierop gebaseerd zijn. De grootte van netwerken van wachtrijen beperkt vaak de praktische bruikbaarheid. Voor veel systemen zijn de prestatiematen daarom niet direct uit de evenwichtsverdeling van het bijbehorende netwerk van wachtrijen te berekenen. Een methode om toch redelijke uitspraken te doen over deze prestatiematen is gebaseerd op de aankomststelling. Een andere methode is gebaseerd op Norton’s stelling uit de electriciteitsleer. Bij deze methode wordt het netwerk van wachtrijen onderverdeeld in clusters van wachtrijen, zodanig dat het proces uiteenvalt in lokale processen, die het gedrag binnen de clusters beschrijven, en een globaal proces, dat het gedrag tussen de clusters beschrijft. In hoofdstuk 6 worden voor netwerken van wachtrijen met een produktvorm voor de evenwichtsverde- 312 Samenvatting ling algemene criteria afgeleid waaronder een dergelijke decompositie mogelijk is, zodanig dat de lokale processen en het globale proces onafhankelijk van elkaar geanalyseerd kunnen worden. Deze decompositie heeft vele praktische toepassingen: indien alleen het globale gedrag van het netwerk van wachtrijen geanalyseerd moet worden, dan geeft de decompositie een aanzienlijke besparing van de rekeninspanning; indien het gedetailleerde gedrag van een of enkele clusters bepaald moet worden, dan volgt uit de decompositie dat van de overige clusters slechts het globale gedrag van invloed is, ook dit kan gunstig zijn voor de totale rekeninspanning die nodig is om prestatiematen te bepalen. Norton’s stelling voor netwerken van wachtrijen is tevens een basis voor de benadering van prestatiematen. Hoofdstukken 5 en 6 tonen aan dat de klasse van netwerken met een produktvorm voor de evenwichtsverdeling vrij ruim is en dat er goede tech- nieken bestaan om in dergelijke netwerken de relevante prestatiematen te bepalen. Ook al is de klasse van netwerken van wachtrijen met een produktvorm voor de evenwichtsverdeling een vrij ruime klasse, en zijn er voor netwerken van wachtrijen die niet in deze klasse vallen verschillende benaderingstechnieken gebaseerd op produktvorm verdelingen, toch is de groep van netwerken die m.b.v. produktvormen geanalyseerd kan worden beperkt. Hierom is het nodig om meer algemene uitdrukkingen voor de evenwichtsverdeling te onderzoeken. Een eerste stap in deze richting vormen netwerken van wachtrijen waarvoor de evenwichtsverdeling geen produktvorm heeft, maar een som is van produktvormen. In hoofdstuk 7 wordt deze stap gemaakt. Hiertoe wordt een verzameling van netwerken van wachtrijen beschouwd, waarbij ieder netwerk in deze verzameling een netwerk is met een produktvorm voor de evenwichtsverdeling. Vervolgens worden de overgangsmatrices van deze netwerken bij elkaar opgeteld. In- dien de overgangsmatrices van de netwerken in de verzameling voldoen aan ‘cross-balance’, dan is de evenwichtsverdeling van het somproces precies de som van de evenwichtsverdelingen van de netwerken in de verzameling. De techniek van ‘cross-balance’ geeft een algemeen bruikbare methode om netwerken van wachtrijen te analyseren. Een tweede generalisatie van de standaard produktvorm netwerken zoals beschreven in hoofdstukken 5 en 6 vormen netwerken van wachtrijen waarin een aantal klanten tegelijkertijd van wachtrij kan wisselen. Deze netwerken worden geanalyseerd in hoofdstukken 8 en 9. In hoofdstuk 8 wordt een algemeen model opgesteld voor processen waarin iedere transitie naar twee kanten kan worden doorlopen. Dit model is sterk gerelateerd aan modellen voor chemische reacties en wordt gebruikt om zowel Produktvorm in netwerken van wachtrijen 313 netwerken van wachtrijen als polymerisatieprocessen te beschrijven. De karakteristieke eigenschap van het model van hoofdstuk 8 is dat het proces sterk reversibel is, d.w.z. dat de waarschijnlijkheidsstroom voor een specifieke transitie (reactie) even groot is als de waarschijnlijkheidsstroom voor de omgekeerde transitie. Onder deze voorwaarde wordt het bestaan van een produktvorm voor de evenwichtsverdeling gerelateerd aan een symmetrievoorwaarde voor de overgangsmatrix. In netwerken van wachtrijen kan een transitie veelal slechts in een richting plaatsvinden, bijvoorbeeld in een produktielijn (lopende band) bewegen alle pallets in een richting. Aan de voorwaarde van sterke re- versibiliteit uit hoofdstuk 8 is daarom voor veel netwerken niet voldaan. In hoofdstuk 9 wordt deze voorwaarde losgelaten. Gebaseerd op groep- lokaal-evenwicht, een speciale vorm van lokaal evenwicht voor netwerken van wachtrijen met meervoudige transities, worden nodige en voldoende voorwaarden voor een produktvorm voor de evenwichtsverdeling gegeven. Verder wordt aangetoond dat voor netwerken die voldoen aan groep- lokaal-evenwicht de analyse van de bediening en routering onafhankelijk kan geschieden. De resultaten voor netwerken met meervoudige transities worden in dit hoofdstuk toegepast op netwerken van wachtrijen met een discrete tijdsparameter, een klasse van netwerken die bijvoorbeeld belang- rijk is voor de analyse van computers. In de netwerken uit de hoofdstukken 3 t/m 9 wordt de lengte van de wachtrij vergroot als er een klant bij een wachtrij aankomt. In hoofdstuk 10 wordt een derde generalisatie van netwerken van wachtrijen beschreven. Hier is het mogelijk dat een aankomende klant de rijlengte bij een wachtrij met éénverkleint. Dit correspondeert bijvoorbeeld met een neuraal netwerk, waar binnenkomende signalen de potentiaal van een neuron kunnen vergroten of verkleinen, of met een systeem waarin een verzoek om bediening kan worden ingetrokken. Voor dergelijke systemen wordt een speciale vorm van lokaal evenwicht afgeleid, die weer een produktvorm voor de evenwichtsverdeling impliceert. Deze vorm van lokaal evenwicht is niet direct gerelateerd aan lokaal evenwicht voor standaard netwerken van wachtrijen. Uit de hoofdstukken over het evenwichtsgedrag van netwerken van wachtrijen blijkt dat éénverzameling vergelijkingen een centrale rol speelt. Deze verzameling van vergelijkingen, de zogenaamde verkeersvergelijkingen, is gerelateerd aan de gemiddelde stroom van klanten door het netwerk en is relatief eenvoudig op te lossen. Een netwerk van wachtrijen heeft alleen een produktvorm voor de evenwichtsverdeling als de fysieke uit- 314 Samenvatting stroom van klanten uit een station (de hoeveelheid klanten dat het station verlaat) gelijk is aan de fysieke instroom van klanten in een station. Deze eigenschap is sterk verwant met het bestaan van een oplossing van de verkeersvergelijkingen en met het bestaan van een geschikte vorm van lokaal evenwicht. Dit leidt tot de volgende conclusie: een netwerk van wachtrijen heeft een produktvorm voor de evenwichtsverdeling dan en slechts dan als een van de volgende uitspraken geldt (vgl. hoofdstuk 11):

~~(a) de fysieke uitstroom bij ieder station is gelijk aan de instroom; (b) er bestaat een geschikte vorm van lokaal evenwicht; (c) de geschikte set van verkeersvergelijkingen heeft een oplossing. Index~~

absorbing 17 batch routing equations adjoint process 36 state-dependent 243 aggregated process 142 batch routing probabilities aggregation coefficients 208 state-independent 236 amalgamated process 8, 158 BCMP-network 33 amalgamation coefficients 158 Bernoulli-service 37, 246 arrivals birth-and-death process 65 Bernoulli batch 239 blocking blocking of 30 anticipative 250 early 245 of arrivals 30 late 245 communication 34, 107 Poisson batch 238 indicator 80 triggered 30 jump-over 34 arrival theorem 103 minimal workload 242 assumptions 38 reversible 80 ASTA 103 upper limit 241 balance canonical form cross- 8, 161 closed network 44 detailed 25 open network 42 deterministic detailed 189 characteristic equations 290 global 5, 19, 25 characteristic result 290 group-local- 9, 28, 219 classification of methods 2 job-local 35, 219 closed-form expression 3 local 3, 28, 79, 264 closed queueing network 22 partial 3, 27 closed set 17 station 3, 28 clustering process 187, 205ff strong detailed 28 communicating states 17 transition 25 communication-blocking 34, 107 balanced network 287 complementary slackness 105 basic assumptions 4 concentration 207 batch routing 37 concentration equations 206

315 316 Subject index configuration-potential 82, 212 initial distribution 16 conservative rate matrix 13 insensitivity 33 cross-balance 8, 161 intercommunicating states 17 customer position 21 invariant measure 18, 19 irreducibility condition decomposition theorem 227 negative customers 275 detailed balance 25 irreducible set 17 deterministic 189 discrete time 37 Jackson network 29 distribution job-local-balance 35, 219 equilibrium 18, 19 jump-over blocking 34 stationary 18, 19 dualizing method Kirchoff’s laws 118 backward 87 Kolmogorov equations forward 87 backward 14 dual process 6, 84, 96, 213, 243, 249 forward 4, 14, 16, 47 dual state space 78 Kolmogorov’s criterion 26, 224 dummy transition 218 Kronecker-delta 13 early arrivals 245 late arrivals 245 electrical circuit 117 Lévy’sdichotomy 13, 47 Engset loss model 6, 63ff local balance 3, 28 equilibrium distribution 18, 19 for negative customers 264 ergodic 18 for the dual process 79 excitation signal 257 for the primal process 79 exit-time 14 local irreducible set 219 local process 139 feedforward networks 287 local solution 221 fraction of time 18 local state space 219 fragmentation coefficients 208 Markov chain 13 GLB 219 continuous-time 13, 21, 22 global balance 5, 19, 25 regular 15 global process 139, 141 uniformizable 15 global recirculation protocol 131 Markov process 12 group-local-balance 9, 28, 219 time-homogeneous 12 Markov property 12 hole, see vacancy measure 17 indicator 18 invariant 18, 19 inhibition signal 257 stationary 17 Subject index 317

MUSTA 103 q¯- 221 qy- 227 negative customers 257ff self-dual 111 neural networks 257 stationary 12 neuron potential 257 stochastic 11 Norton’s theorem 7, 116ff time-reversed 168 applications of 146 truncated 167 in strong form 141 product-form 4 in weak form 138 classical 31 notation for clusters 123 equilibrium distribution 30 null-recurrent state 17 insensitive 33 numerical factors 227 transient distribution 40 occupation numbers 20 pseudoinverse 194 open queueing network 22 pure-jump process 14 outside 21 q¯-process 221 Palm probability 100 – 103 qy-process 227 partial balance 3, 27 quasirandom input 63 pathological network 288 queueing network 19ff polymerization process 207 batch routing 37 positive-recurrent state 17 closed 22 potential-interpretation 82 discrete-time 37, 244ff primal process 78, 96 Walrand’s 245 probability mass 16 Pujolle’s 246 probability flow 16, 19 open 22 probability flux 16 queue-length 1 process rate matrix 13 adjoint 36 conservative 13 aggregated 142 stable 13 amalgamated 8, 158 recirculation protocol 131 birth-and-death 65 recurrence-time 17 clustering 187, 205ff recurrent 17 dual 6, 84, 96, 213, 243, 249 regular 15, 22 global 139, 141 reversibility 26 local 139 strong 8, 28, 190, 222 Markov 12 routing decomposition 127 polymerization 207 primal 78, 96 self-dual process 111 pure-jump 14 service deletions 30 318 Subject index service-potential 34 global 126 service-protocol 33 local 127 (φ, γ, δ)-protocol 33 negative customers 283ff service-time 20 state-dependent 121 simultaneous jumps 23 time-dependent 41 singleton 221 transient distribution 16 site-potential 83, 213 transient state 17 sojourn-time 1 transition balance 25 stable 13, 22 transition matrix 12 standard transition matrix 12 standard 12 state 11 transition rate 13, 21 absorbing 17 potential-interpretation 82 recurrent 17 triangular set 179 transient 17 basic 182 state space 11 proper 182 dual 78 triggered arrivals 30 stationary distribution 18, 19 truncated process 167 stationary measure 17 stationary process 12 uniformizable 15 station balance 3, 28 unit-vector 21 stochastic process 11 vacancy 94, 95, 113 continuous-time 11 vector discrete-time 11 non-negative 12 stop-protocol 92, 107, 184 positive 12 batch-routing- 252 unit- 21 dual- 109 strong detailed balance 28 weak coupling 155 strong reversibility 8, 28, 190, 222 symmetrical station 33 symmetry-property 189

~~Thevenin’s theorem 7, 117 throughput 1 time-dependent distribution 16 time-homogeneous 12 time-reversed process 168 traﬃc equations 30 also batch routing equations batch 235 Symbol index~~

~~0 null-element, e.g. vector consisting of 0’s only 1{A} p.18, indicator: 1{A} = 1 if A occurs, 0 otherwise~~

~~β(¯g) p.226, numerical factors B normalizing constant Bd p.82, dual normalizing constant~~

~~γ in Chapter 4: input-rate Cr, r ∈ R p.123, cluster r~~

δ(¯n, n¯0) p.13, Kronecker-delta, δ(¯n, n¯0) = 1 ifn ¯ =n ¯0, 0 otherwise d(i) p.261, probability that customer leaves the network D+ p.194, pseudoinverse of D Dij(m ¯ ) p.83, site-potential diﬀerence d Dkj(¯n) p.84, dual site-potential diﬀerence ei p.21, ith unit vector, e0 = 0 (r) ei p.123, ith unit vector at cluster r e¯r p.123, unit vector for cluster r E expectation fi(¯n) p.20, service-rate at station i g¯ p.23, served group of customers gi p.23, number of customers served at station i

~~H(¯n) p.121, solution to the state-dependent traffic equations HR(n¯) p.126, solution to the global traffic equations H(r)(¯n(r)) p.127, solution to the local traffic equations~~

~~319 320 Symbol index~~

λi p.29, Poisson arrival rate to station i λ(i) p.261, Poisson arrival rate for negative customers λ(¯g, g¯0) p.188, batch routing probabilities Λ(i) p.261, Poisson arrival rate for positive customers

µ in Chapter 4: service-rate −1 µi p.19, required amount of service at station i µi(¯n) pp.20, 120, departure rate from station i in staten ¯ (s) (s) µi (¯n ) p.139, local service-rate µ0(¯n) p.20, arrival rate in staten ¯ µ(¯g, n¯) p.24, service-rate m(¯n) p.17, measure, invariant measure m(k)(¯n) p.160, invariant measure of process k Mr(n¯) p.139, global service-rate n¯ state of the processn ¯ = (n1, . . . , nN ) ni ith component of staten ¯ number of customers present at station i n¯(r) p.123, state at cluster r n¯ p.123, global state nr p.123, rth component of n¯ N number of stations of the queueing network in Chapter 4: number of sources N p.78, N = {0,...,N} for open queueing networks p.78, N = {1,...,N} for closed queueing networks IN natural numbers {1, 2, 3,...} IN0 IN ∪ {0} {N(t)} stochastic process {N d(t)} p.84, dual process o(h) p.13, function g(h) such that g(h)/h → 0 as h → 0

π(¯n) p.18, equilibrium distribution, stationary distribution πd(m ¯ ) p.82, dual equilibrium distribution (s) πn p.65, equilibrium distribution for the Engset loss model πy(¯n) p.227, equilibrium distribution of the qy-process πR(n¯) p.139, equilibrium distribution of the global process ΠR(n¯) p.141, global equilibrium distribution Symbol index 321

pij p.21, routing probabilities p(i, j;m ¯ ) p.21, state-dependent routing probabilities p(¯g, g¯0;m ¯ ) p.23, state-dependent batch routing probabilities pij(m ¯ ) p.78, primal state-dependent routing probabilities d pij(¯n) p.84, dual state-dependent routing probabilities (r) (r) pij (¯n ) p.125, state-dependent routing probabilities at cluster r prs(n¯) p.125, global state-dependent routing probabilities p+(i, j) p.261, routing probability for positive customers p−(i, j) p.261, routing probability for negative customers P0(¯n) p.16, initial distribution P{A} probability of event A P{A|B} conditional probability of event A given event B P (¯n, n¯0; s) p.12, transition probability P (t) p.12, transition matrix (s) Pn (t) p.65, transient distribution for the Engset loss model φ(¯n) p.23, service function φR(n¯) p.134, global service function φ(r)(¯n(r)) p.134 local service function Φ(¯n) p.79, invariant measure ΦR(n¯) p.141, global service function ψ(¯n) p.23, service function ψR(n¯) p.135, global service function ψ(r)(¯n(r)) p.135, local service function ψ(¯g, g¯0;m ¯ ) p.188, service and routing function Ψ(¯n) p.79, invariant measure ΨR(n¯) p.141, global service function q(¯n, n¯0) p.13, transition rate from staten ¯ to staten ¯0 q(¯g, g¯0;m ¯ ) p.23, transition rates qd(m, ¯ m¯ 0) p.84, dual transition rates 0 qR(n¯, n¯ ) p.139, global transition rates q(r)(¯n(r), n¯0(r)) p.139, local transition rates at cluster r q(k)(¯n, n¯0) p.158, transition rates of process k Q p.13, rate matrix Q(n¯, n¯0) p.141, transition rates of the global process

~~ρ(¯n) p.14, exit-time from staten ¯ r(i) p.261, service-rate at station i r(k) p.158, amalgamation coeﬃcient 322 Symbol index~~

~~IR real numbers (−∞, ∞) IR+ positive real numbers (0, ∞) s time variable in Chapter 4: number of servers S state space Sd p.78, dual state space S− p.86~~

~~τ(¯n) p.17, recurrence-time to staten ¯ t time variable T p.11, time-axis~~

~~U p.82, conﬁguration potential~~

V irreducible set p.82, in Chapter 5: conﬁguration potential V (k) p.162, irreducible set of kth process x(¯g;m ¯ ) p.221, local solution to GLB-equations y(¯g;m ¯ ) p.227, local solution to batch traﬃc equations

~~Z {..., −2, −1, 0, 1, 2,...}~~