Performance Evaluation of Queueing Networks - Outline Introduction - networks of queues are the example family of systems to be studied • Deterministic models including network calculus • Dynamic routing • Review of elements of probability and statistics & statistical confidence, overview of simu- • lation Stationary models • Markovian models in continuous and discrete time • Caching • Fork-join queues • Markov decision processes • Constrained optimization and duality with examples • November 9, 2020 George Kesidis 1 Performance Evaluation of Queueing Networks - Outline (cont) Queueing system models have been used in a wide range of applications including com- • puter/communication networking, computation, supply chain and logistics. The focus of this course will be (unambiguous) theoretical derivations of performance ob- • jectives based on models of queueing system and their workloads. To this end, we will review the basic, relevant elements of probability theory. • We will also discuss performance evaluation based on simulation. • Simulation is useful when system or workload complexity precludes simple models that lead • to close-form analytical results for the performance objectives. We also will review the use of statistical confidence when reporting the results of a simulation • study. November 9, 2020 George Kesidis 2 Performance Evaluation of Queueing Networks - Outline (cont) In the following, our approach to performance evaluation will be to will consider models of • increasing detail: 1. deterministic, including worst-case analysis 2. stationary and ergodic 3. stationary Markovian We will demonstrate how increased model complexity (assumedsuitableforthephysical • system under consideration) leads to more refined and detailed performance results. November 9, 2020 George Kesidis 3 Deterministic models of queues and queuing networks Arrivals, departures and queue occupancy • Trafficshaping-tokenbuckets,servicecurves • Flow scheduling • Network calculus • November 9, 2020 George Kesidis 4 Queues - preliminaries Aqueueorbuffer is simply a waiting room with an identified arrival process and departure • (completed ”jobs”) process. Work is performed on jobs by servers according to a service policy. • In some applications, jobs arriving to the queue will be packets of information; in others, • the arrivals will represent calls attempting to be set-up in the network. Some jobs may be blocked from entering the queue (if the queue’s waiting room is full) or • join the queue and be expelled from the queue before reaching the server. For jobs reaching the server, their queueing delay plus service time is called their sojourn • time, i.e., the time between the arrival of the job to the queue and its departure from the server. We will consider queues that serve jobs in the order of their arrival known as first come, • first serve (FCFS) or first in, first out (FIFO). November 9, 2020 George Kesidis 5 Arrivals, departures, and queue occupancy Over the time interval (0,t],thecountingprocess • – A(0,t] t R+ represents the number of jobs arriving at the queue, { | ∈ } – D(0,t] t R+ represents the number of departures from the queue, { | ∈ } – L(0,t] t R+ represents the number of jobs lost (blocked upon arrival or pushed- out/preempted).{ | ∈ } Let Q(t) be the number of jobs in the queueing system at time t; i.e., • – the occupancy of the queue plus the number of jobs being servedattimet; – including the arrivals at t but not the departures at t. We assume no jobs with zero sojourn time. • November 9, 2020 George Kesidis 6 Arrivals, departures, and queue occupancy (cont) Clearly, a previously ”arrived” job is either queued or has departed or has been blocked, i.e., • Q(0) + A(0,t]=Q(t)+D(0,t]+L(0,t]. If we take the origin of time to be ,wecansimplywrite • −∞ Q(t)=A( ,t] D( ,t] L( ,t]. −∞ − −∞ − −∞ November 9, 2020 George Kesidis 7 Basic assumptions We’ll typically assume that: • – Servers are nonidling (or ”work conserving”) in that they are busy whenever Q(t) > 0. – Ajob’sservicecannotbepreemptedbyanotherjob. – Jobs may only be blocked upon arrival to a queue. – All servers associated with a given queue work at the same, constant rate (otherwise, need to define the work each job brings). th Thus, we can unambiguously define Si to be the service time required by the i job. • In addition, each job i will have the following two quantities associated with it: • – its arrival time to the queueing system Ti,assumedtobeanondecreasingsequencein i ( i, T T +1), and ∀ i ≤ i – its departure (service completion) time from the server Vi if the job is not lost. November 9, 2020 George Kesidis 8 Queue workload (not blocked jobs) th Let Ri(t) be the residual amount service time required by the i job at time t. • Clearly, 0 Ri(t) Si for all i, t; Ri(t)=0for t>Vi; Ri(t)=Si for t<Vi Si. • ≤ ≤ − The total work-to-be-done (or workload)attimet, W (t),issimplythesumoftheservice • times of all queued jobs and residual service times of all jobsbeingservedattimet. For jobs i that are not lost, let Vi be the departure time of the job from the server. • th Clearly, Vi Si is the time at which the i job enters a server and, for all t and i, • − Ri(t)=Vi t. − Clearly, a job i is in the queue but not in service if Ti t<Vi Si. • ≤ − Ri(t) Si Ti Vi Si t Vi − November 9, 2020 George Kesidis 9 Parameterizing queue arrival and departure processes The arrival process A is parameterized above as Ti,Si i Z or Z+ . • { } ∈ The queueing discipline determines how jobs are enqueued andinwhichordertheyare • served (dequeued), i.e., the dynamics of queue Q and workload W processes. The departure process D,parameterizedby Vi,Si is determined by both the queueing • discipline and the arrival process. { } November 9, 2020 George Kesidis 10 Lossless queues Now assume the queue we have just introduced is lossless, i.e., L( ,t]=0for all t. • −∞ Define the indicator 1B =1if B is true, else =0.Since, • A(s, t]= 1 T (s, t] , and D(s, t]= 1 V (s, t] , { i ∈ } { i ∈ } i i ! ! we get (by recalling i, Vi >Ti by assumption) that ∀ Q(t)=A( ,t] D( ,t]= 1 T t<V . −∞ − −∞ { i ≤ i} i ! th The sojourn time is the total delay experienced by the i job, Vi Ti, i.e., the departure • time minus the arrival time. − Again, this sojourn time consists of two components: the queueing delay, Vi Ti Si, • − − plus the service time, Si. Expressions will be derived for quantities of interest such as the number of jobs in the queue, • the workload, and job sojourn times. The objective is to express quantities of interest in terms ofthejobarrivaltimesandservice • times alone. November 9, 2020 George Kesidis 11 The case of no waiting room Suppose the queueing system consists only of the servers and no waiting room. • Thus, if the job flow is demultiplexed (demux’ed) to one of K servers, the queueing system • can only hold K jobs at any given time. Since the system is assumed lossless: for all jobs i, • Vi = Ti + Si If there are infinitely many servers (K = ), the system is always lossless and so the • number of jobs queued and the workload are∞ Q(t)= 1 Ti t<Ti + Si = 1 Ti (t Si,t] { ≤ } { ∈ − } i i ! ! W (t)= 1 T (t S ,t] R (t) { i ∈ − i } i i ! where R (t)=V t. i i − In the following figure, note how the negative slope of the workload sample path is propor- • tional to the number of jobs currently queued. November 9, 2020 George Kesidis 12 The case of no waiting room - example sample path W (t) S2 S3 S1 ... t S1 Q(t) S2 4 3 2 1 ... t T1 T2 T3V2 V1 November 9, 2020 George Kesidis 13 The case of a lossless single-server queue waiting room server arriving jobs departing jobs Now suppose that the queue has a waiting room and only a single server. • Clearly, if the waiting room was infinite in size, the queue would be lossless irrespective of • the job arrival and service times. For the following example sample path, note that upon arrivaloftheith job at time T , Q • i increases by 1 and W increases by Si. The process Q is piecewise constant and, due to the action of the server, W (t) has zero • time derivative if Q(t)=0(i.e., W is constant) and otherwise has time derivative 1 for any t that is not a job arrival time. − Upon departure of the ith job, Q decreases by 1. • November 9, 2020 George Kesidis 14 The case of a lossless single-server queue - example sample path W (t) ... S3 S1 S2 t Q(t) 4 3 2 ... 1 t T1 T2 T3 V1 November 9, 2020 George Kesidis 15 Lossless single-server queue: Departure-times recursion Theorem: For a work-conserving, single-server, lossless FIFO queue,theith job’s departure • time Vi =maxVi 1,Ti + Si { − } + for all jobs i Z ,whereV0 0. ∈ ≡ Proof: For the ith job arriving at the lossless queue, there are two cases. • If Ti >Vi 1,then: • − – job i 1 has already departed the queue by time Ti. − – So, Q(T )=0and, i− – when the ith job joins the queue, it immediately enters the server. – So, it departs Si seconds after it arrives, i.e., Vi = Ti + Si. On the other hand, if Ti Vi 1, • ≤ − – job i 1 is present in the queue (and immediately ahead of the ith job) when the ith job joins− the queue. th – Thus, the i job will depart the queue Si seconds after job i 1, i.e., Vi = Vi 1 +Si. − − November 9, 2020 George Kesidis 16 Lossless single-server queue: Departure-times recursion (cont) Note that, by subtracting T from both sides of the departure-times recursion, we get a • i statement involving the sojourn times Vi Ti and the interarrival times Ti Ti 1: − − − Vi Ti =maxVi 1 Ti, 0 + Si − { − − } =max(Vi 1 Ti 1) (Ti Ti 1), 0 + Si, { − − − − − − } where T0 0.