Notes for ECE 567 Communication Network Analysis Bruce Hajek May 31, 2021 c 2006 by Bruce Hajek All rights reserved. Permission is hereby given to freely print and circulate copies of these notes so long as the notes are left intact and not reproduced for commercial purposes. Email to [email protected], pointing out errors or hard to understand passages or providing comments, is welcome. Contents 1 Countable State Markov Processes 3 1.1 Example of a Markov model . 3 1.2 Definition, Notation and Properties . 5 1.3 Pure-Jump, Time-Homogeneous Markov Processes . 6 1.4 Space-Time Structure . 8 1.5 Poisson Processes . 12 1.6 Renewal Theory . 14 1.6.1 Renewal Theory in Continuous Time . 16 1.6.2 Renewal Theory in Discrete Time . 17 1.7 Classification and Convergence of Discrete State Markov Processes . 17 1.7.1 Examples with finite state space . 17 1.7.2 Discrete Time . 20 1.7.3 Continuous Time . 22 1.8 Classification of Birth-Death Processes . 24 1.9 Time Averages vs. Statistical Averages . 26 1.10 Queueing Systems, M/M/1 Queue and Little's Law . 27 1.11 Mean Arrival Rate, Distributions Seen by Arrivals, and PASTA . 30 1.12 More Examples of Queueing Systems Modeled as Markov Birth-Death Processes . 32 1.13 Method of Phases and Quasi Birth-Death Processes . 34 1.14 Markov Fluid Model of a Queue . 36 1.15 Problems . 37 2 Foster-Lyapunov stability criterion and moment bounds 45 2.1 Stability criteria for discrete time processes . 45 2.2 Stability criteria for continuous time processes . 53 2.3 Problems . 57 3 Queues with General Interarrival Time and/or Service Time Distributions 61 3.1 The M/GI/1 queue . 61 3.1.1 Busy Period Distribution . 63 3.1.2 Priority M/GI/1 systems . 64 3.2 The GI/M/1 queue . 66 3.3 The GI/GI/1 System . 68 3.4 Kingman's Bounds for GI/GI/1 Queues . 69 3.5 Stochastic Comparison with Application to GI/GI/1 Queues . 71 iii 3.6 GI/GI/1 Systems with Server Vacations, and Application to TDM and FDM . 73 3.7 Effective Bandwidth of a Data Stream . 75 3.8 Problems . 78 4 Multiple Access 83 4.1 Slotted ALOHA with Finitely Many Stations . 83 4.2 Slotted ALOHA with Infinitely Many Stations . 85 4.3 Bound Implied by Drift, and Proof of Proposition 4.2.1 . 87 4.4 Probing Algorithms for Multiple Access . 90 4.4.1 Random Access for Streams of Arrivals . 92 4.4.2 Delay Analysis of Decoupled Window Random Access Scheme . 92 4.5 Problems . 93 5 Stochastic Network Models 99 5.1 Time Reversal of Markov Processes . 99 5.2 Circuit Switched Networks . 101 5.3 Markov Queueing Networks (in equilibrium) . 104 5.3.1 Markov server stations in series . 104 5.3.2 Simple networks of MS stations . 105 5.3.3 A multitype network of MS stations with more general routing . 107 5.4 Problems . 109 6 Calculus of Deterministic Constraints 113 6.1 The (σ; ρ) Constraints and Performance Bounds for a Queue . 113 6.2 f-upper constrained processes . 115 6.3 Service Curves . 117 6.4 Problems . 120 7 Graph Algorithms 123 7.1 Maximum Flow Problem . 123 7.2 Problems . 124 8 Flow Models in Routing and Congestion Control 127 8.1 Convex functions and optimization . 127 8.2 The Routing Problem . 128 8.3 Utility Functions . 133 8.4 Joint Congestion Control and Routing . 134 8.5 Hard Constraints and Prices . 135 8.6 Decomposition into Network and User Problems . 136 8.7 Specialization to pure congestion control . 139 8.8 Fair allocation . 141 8.9 A Network Evacuation Problem . 142 8.10 Braess Paradox . 142 8.11 Further reading and notes . 144 8.12 Problems . 144 iv 9 Dynamic Network Control 149 9.1 Dynamic programming . 149 9.2 Dynamic Programming Formulation . 150 9.3 The Dynamic Programming Optimality Equations . 152 9.4 Problems . 156 10 Solutions 161 v vi Preface This is the latest draft of notes I have used for the graduate course Communication Network Analy- sis, offered by the Department of Electrical and Computer Engineering at the University of Illinois at Urbana-Champaign. The notes describe many of the most popular analytical techniques for design and analysis of computer communication networks, with an emphasis on performance issues such as delay, blocking, and resource allocation. Topics that are not covered in the notes include the Internet protocols (at least not explicitly), simulation techniques and simulation packages, and some of the mathematical proofs. These are covered in other books and courses. The topics of these notes form a basis for understanding the literature on performance issues in networks, including the Internet. Specific topics include • The basic and intermediate theory of queueing systems, along with stability criteria based on drift analysis and fluid models • The notion of effective bandwidth, in which a constant bit rate equivalent is given for a bursty data stream in a given context • An introduction to the calculus of deterministic constraints on traffic flows • The use of penalty and barrier functions in optimization, and the natural extension to the use of utility functions and prices in the formulation of dynamic routing and congestion control problems • Some topics related to performance analysis in wireless networks, including coverage of basic multiple access techniques, and transmission scheduling • The basics of dynamic programming, introduced in the context of a simple queueing control problem • The analysis of blocking and the reduced load fixed point approximation for circuit switched networks. Students are assumed to have already had a course on computer communication networks, al- though the material in such a course is more to provide motivation for the material in these notes, than to provide understanding of the mathematics. In addition, since probability is used exten- sively, students in the class are assumed to have previously had two courses in probability. Some prior exposure to the theory of Lagrange multipliers for constrained optimization and nonlinear optimization algorithms is desirable, but not necessary. I'm grateful to students and colleagues for suggestions and corrections, and am always eager for more. Bruce Hajek, December 2006 1 2 Chapter 1 Countable State Markov Processes 1.1 Example of a Markov model Consider a two-stage pipeline as pictured in Figure 1.1. Some assumptions about it will be made in order to model it as a simple discrete time Markov process, without any pretension of modeling a particular real life system. Each stage has a single buffer. Normalize time so that in one unit of time a packet can make a single transition. Call the time interval between k and k + 1 the kth \time slot," and assume that the pipeline evolves in the following way during a given slot. If at the beginning of the slot, there are no packets in stage one, then a new packet arrives to stage one with probability a, independently of the past history of the pipeline and of the outcome at state two. If at the beginning of the slot, there is a packet in stage one and no packet in stage two, then the packet is transfered to stage two with probability d1. If at the beginning of the slot, there is a packet in stage two, then the packet departs from the stage and leaves the system with probability d2, independently of the state or outcome of stage one. These assumptions lead us to model the pipeline as a discrete-time Markov process with the state space S = f00; 01; 10; 11g, transition probability diagram shown in Figure 1.2 (using the notationx ¯ = 1 − x) and one-step transition probability matrix P given by 0 a¯ 0 a 0 1 ¯ ¯ B ad¯ 2 a¯d2 ad2 ad2 C P = B ¯ C @ 0 d1 d1 0 A 0 0 d2 d¯2 d d a 1 2 Figure 1.1: A two-stage pipeline 3 -- ---- -- ad ad a 2 2 00 01 d 1 -- a ad ad 2 2 -- 10 11 d d 2 2 -- d1 Figure 1.2: One-step transition probability diagram for example. The rows of P are probability vectors. (In these notes, probability vectors are always taken to be row vectors, and more often than not, they are referred to as probability distributions.). For example, the first row is the probability distribution of the state at the end of a slot, given that the state is 00 at the beginning of a slot. Now that the model is specified, let us determine the throughput rate of the pipeline. The equilibrium probability distribution π = (π00; π01; π10; π11) is the probability vector satis- fying the linear equation π = πP . Once π is found, the throughput rate η can be computed as follows. It is defined to be the rate (averaged over a long time) that packets transit the pipeline. Since at most two packets can be in the pipeline at a time, the following three quantities are all clearly the same, and can be taken to be the throughput rate. The rate of arrivals to stage one The rate of departures from stage one (or rate of arrivals to stage two) The rate of departures from stage two Focus on the first of these three quantities. Equating long term averages with statistical averages yields η = P [an arrival at stage 1] = P [an arrival at stage 1jstage 1 empty at slot beginning]P [stage 1 empty at slot beginning] = a(π00 + π01): Similarly, by focusing on departures from stage 1, obtain η = d1π10.