DOCSLIB.ORG

Absorbing State in Markov Decision Process, 330 Absorption, 466

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Absorbing State in Markov Decision Process, 330 Absorption, 466 Index Absorbing state Average-cost criterion, 326, 360 in Markov decision process, 330 Back-substitution, 182-183 Absorption, 466 Bailey's bulk queue, 209, 247-248 Accelerating convergence, 273 Balance equations, 412 Acceleration, 268, 270 global, 130, 411-412, 414 Action space, 326 job-class, 423 Adjoint, 215, 230 local, 130, 422 Agarwal, 289, 298 partial, 411-414, 418, 422 Aggregate station, 437 station, 411, 413-414, 418, 422 Aggregation, 57 and blocking, 427 Aggregation matrix, 102 failure of, 425 Aggregation step, 103 restored, 427, 429, 435 Aliased sequence, 277 Barthez, 272 Aliasing, 266 BASTA, 373 See also Error, aliasing Benes, 282, 284, 287 Alternating series, 268 Bernoulli arrivals, 373 Analytic, 206, 208, 210, 216, 235, 238 Bernoulli process, 387 Analyticity condition, 294 Bertozzi, 304 And gate, 462 Binomial average, 270 Approximation, 375, 377-378, 380-381, 388, Binomial distribution, 270 392-393, 396 Block elimination, 175, 179, 183 of transition matrix, 180, 357 and paradigms of Neuts, 187-189 Arbitrary-epoch tail probabilities, 381 Block iterative methods, 93 Arbitrary service time, 381 Block Jacobi, 94 Argument principle, 207, 239 Block SOR, 94 Arnoldi's method, 53, 83, 97 Block-splitting, 93 Arrival-first, 366 Blocking, 158, 425-427 Asmussen, 293, 299, 301 Blocking probability, 410 Assembly line, 415 Erlang,283 Asymptotic behavior, 243 time dependent, 280, 282 Asymptotic formulas, 282 Borovkov, 281 Asymptotic parameter, 294 Boundary probabilities, 206, 210, 218 Asymptotics, 303 Bounding methodology, 428-429 Automata: see Stochastic automata Bounds, 351, 429, 432, 434-435, 437 Availability, 46, 465 and optimal design, 436 instantaneous, 447 for waiting time, 396 interval, 447 intuitively obvious, 430 limiting interval, 448 on loss probability, 429, 432, 434 point availability, 69 on minimal costs, 436 steady-state, 448 on throughput, 429, 432, 439 482 COMPUTATIONAL PROBABILITY on utilization, 429 rate of, 89, 358 Branch point, 238 Convex, 243-244 Breakdown, 426, 434-437, 439 Convolution algorithm, 304, 423, 428 Bremmer, 260 Convolution, 206, 259, 285, 315 Bromwich inversion integral, 263 smoothing, 272 Buffer, 411, 437-438 n-fold, 261 Busy cycle, 291, 367 Conway, 304 Busy period, 261, 367, 387 Costs, 325, 328 in MIGll queue, 280 terminal, 329 time-homogeneous, 335 Call center, 194, 433 uniformly bounded, 357 bilingual, 199 Coupling matrix, 102 Capacity constraint, 411, 427 Covariance function, 281, 287 Cauchy contour integral, 276 Cramer-Lundberg approximation, 294 Censored chain, 214, 221, 230 Crout method, 184, 191 Censoring, 173, 175, 177, 180, 182, 189-190 CTMC, 154, 412 Central limit theorem, 259 See also Continuous-time Markov chain Characteristic function, 258 Cumulants, 299 Chaudhry, 289, 298 Cumulative distribution function, 262 QPACK software package, 381, 397 Cumulative reward, 45, 48, 60-61 QROOT software package, 376, 381, 390 cumulative impulse reward, 46 Chebyshev-Subspace iteration, 97 cumulative operational time, 46, 69 Closed queueing network: see Queueing cumulative rate reward, 45, 64, 71 network Curse of dimensionality, 7, 410 Coefficient of variation, 457 Cutset, 141 Cohen, 257, 290, 301 Combinatorial explosion, 424 DA (Delayed Access), 367 Communication systems, 365 Damped sequence, 277 Compact storage schemes, 82, 85 Damping, 266 Companion matrix, 222 Darling, 286 Complementary cdf, 262 Davies, 272 Complementation Davis, 263, 265 of transition matrix, 180 Decision, 325 Complex-conjugate, 377, 392 Decision rule, 326 Complex Fourier series, 266 greedy, 353 Complex-valued functions, 262 Decomposable, 157, 174 Complex variables, 263 Decomposition Computation time, 114 for dimension reduction, 309-311 Computational complexity, 7, 139, 309 of a network, 439 Computer networking, 410 Decompositional methods, 97 Computer systems, 365 Decreasing failure rate, 27 Conditional decomposition, 309 Delay, 410 Congestion, 432 average, 197-198 Conservation of probability, 206, 209-210, Delayed access, 367 212, 218, 226, 228, 231, 249 Departure-first, 366 Continuous, 210, 216, 238, 243 Dependability, 65, 445-446, 469-473 Continuous-time control, 359 Depth-first search, 105 Continuous-time Markov chain, 154, 219, DES: see Discrete event system 232, 412, 453 Descriptor, 123 Continuous-time Markov decision chain, Determinant, 216-218, 238, 240, 282 358-359 Deterministic arrivals, 397 Contour integral, 263, 289 Deterministic service time, 381, 422 Contour, 276 Diagonally dominant, 240 Contraction mapping, 342, 349, 358 irreducibly, 240 Control strictly, 240 continuous-time, 359 Difference equations, 225 Convergence Difference operator, 264 pointwise, 358 Diffusion process, 281 INDEX 483 Dimension reduction truncation, 265, 270, 274 by decomposition, 309-311 Euler summation, 268, 270, 273, 278, Directed graph, 105 314-315 Direct methods: see Solution method Euler's constant, 301 Disaggregation step, 103 Event, 3, 16 Discount factor, 330 immediate, 15, 29-33 Discounted-cost problem, 330 schedule of, 3, 14 Discrete event Exchangeability, 60, 64 simulation, 3 Expected level, 193 system, 3, 14 Exponential damping, 300 Discrete-time Markov chain, 12, 14, Exponential polynomial, 457 154-155,207 Exponential service time, 412 Discretization error, 265-266, 268 Distribution Fault tree, 451, 462-463 binomial, 270 with repeated events, 451 gamma, 260, 297-298 Feldmann, 301 geometric, 397 Feller, 259, 268 infinitely divisible, 247 Fill-in, 82, 85, 92 marginal, 420-421, 424 Finite differences, 264 normal,281 Finite-horizon problem, 329, 332, 334 phase-type, 5, 27 Finite-state approximations, 357 stationary, 81, 206, 214, 219, 229 First passage time, 47, 284, 387 steady-state, 281 Fixed point, 341, 349 Divide and conquer, 99 iteration, 91, 471 Doetsch, 260, 262 Flannery, 296 Drift, 212, 219 Flow equations, 412 DTMC, 12 Fluid flow models, 71 See also Discrete-time Markov chain Fork and join queue, 427 Dubner,272 Fourier Duffield, 280 coefficients, 266 Durbin, 272 series, 265-266 Dynamic programming, 332, 336 method,266 transform, 168, 258 EAS (early arrival system), 367-368, 397 discrete, 277 Eigenvalues, 51, 207, 217, 222, 234, 238, 243, FT,451 282 See also Fault tree distinct, 239 FTRE,451 multiplicity of, 239 See also Fault tree with repeating Eigenvectors, 207, 217, 243, 282 events Elimination of non-optimal actions, 355 Functional dependency cycle, 140 Emptiness Functional rates, 135 probability of, 29 Functional transitions, 118, 123 Entropy, 412 Erlang, 365 G matrix, 221 Erlang blocking probability, 281, 283 Gamma distribution, 260, 297-298 Erlang loss model, 279 GASTA,373 Error, 265 Gaussian elimination, 86-87, 340 absolute, 293 Gauss-Seidel, 91 aliasing, 266-267, 269-272, 277-279, 303, backward, 82, 89 311-312 forward, 82, 89 bound, 265, 272, 275 ordering of states, 90 discretization, 265-268, 276, 312 Gaver, 258, 272, 301 balancing with roundoff error, 271 Gaver-Stehfest procedure, 264 estimate, 270 Generalized eigenvector, 222 in Euler summation, 272, 274 Generalized Erlang, 292 relative, 293 Generalized tensor product, 135, 137-139 roundoff, 265, 268, 271-272, 278-279, 312 Generating function, 206, 208, 275, 303, 307 control of, 268, 279 Generator, 219, 232 484 COMPUTATIONAL PROBABILITY and synchronizing events, 119 Infinitesimal generator, 81 Geometric arrivals, 373 Ir,itial approximation, 83 Geometric, 160, 366, 377-378, 381, 392, 394 Initial value theorem, 269 Geometric distribution, 397 Insensitivity, 422 Georganas, 304 of bounds, 430, 435 Giffin, 260 Instantaneous state, 31 GI/G/1 paradigm, 156, 191-193 preservation, 31 GI/G/1 queue: see Queue, GI/G/1 Interarrival time GI/G/1 type process, 156-158 rational transform, 290 GI/M/1 paradigm, 168, 187-189 Interdependence graph, 310 GI/M/1 type process, 155-157, 159, 164 Inverse of a matrix, 174-175, 185 GI/M/1 queue: see Queue, GI/M/l Inversion dimension, 310 Global balance, 130,411-412,418,422 Inversion of generating functions, 375 Global generator, 118 Irreducible, 207, 210, 217, 226 G/M/l type process, 206, 223, 228-229, Iterative aggregation/disaggregation, 103 232-233 Iterative methods, 82 See also GI/M/l type process Jacobs, 301 G/M/l queue: see Queue, GI/M/l Jackson, 421 GMRES, 98, 115 Jackson networks, 24, 417 Graph, 310 Jagerman, 264, 272, 281-282, 284 GSPN (Generalized Stochastic Petri Net) Jagerman-Stehfest procedure, 263 17-22 Jensen's method, 54, 233 colored 22 See also Randomization, Uniformization GTH advantage, 86, 182 Job, 305, 422-423 GTP: see Generalized tensor product class, 305 Harmonic, 217, 249 job-class balance, 423 Heavy-traffic, 281 job-local balance, 422 approximation, 293 priority, 427 Hessenberg matrix, 98 type, 423 lower, 155 Johnsonbaugh, 273 upper, 156 Jordan chain, 217, 222, 234, 246 High-level formalism, 14 Jordan form, 222, 234 History, 326 Jump chain, 220, 232 Homogeneous in time, 455-456 Kao,259 Homers's method, 169 Keilson, 283, 286 Hosono, 272 Kendall functional equation, 261 Hyperexponential, 291 Kernel of the MRS, 459 Hypoexponential,291 Kingman, 299 Kleinrock, 260 IA (Immediate Access), 367 Knessl,282 Ideal decomposition, 309 Kobayashi, 259, 304 Idle time, 291 Kolmogorov's equations, 44, 48, 52, 56 Ill-conditioned, 87-88 Krylov subspace, 98 ILU: see Incomplete LU factorization Krylov subspace method, 53, 55 ILU preconditioning, 90 Kwok,272 ILU(O), 92 ILUK,93 Laguerre ILUTH,92 coefficients, 264 Imbedded Markov chain, 159, 173, 180, 365 functions, 264 Immediate access, 367 generating functions, 264 Incomplete LU factorization, 92 polynomials, 264 Increasing failure rate, 27 series, 264 Indicator function, 6 series
Load more

Recommended publications
  • Performance Analysis of Multiclass Queueing Networks Via Brownian Approximation
    Performance Analysis of Multiclass Queueing Networks via Brownian Approximation by Xinyang Shen B.Sc, Jilin University, China, 1993 A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY in THE FACULTY OF GRADUATE STUDIES (Faculty of Commerce and Business Administration; Operations and Logisitics) We accept this thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA May 23, 2001 © Xinyang Shen, 2001 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Faculty of Commerce and Business Administration The University of. British Columbia Abstract This dissertation focuses on the performance analysis of multiclass open queueing networks using semi-martingale reflecting Brownian motion (SRBM) approximation. It consists of four parts. In the first part, we derive a strong approximation for a multiclass feedforward queueing network, where jobs after service completion can only move to a downstream service station. Job classes are partitioned into groups. Within a group, jobs are served in the order of arrival; that is, a first-in-first-out (FIFO) discipline is in force, and among groups, jobs are served under a pre-assigned preemptive priority discipline.
    [Show full text]
  • Markovian Queueing Networks
    Richard J. Boucherie Markovian queueing networks Lecture notes LNMB course MQSN September 5, 2020 Springer Contents Part I Solution concepts for Markovian networks of queues 1 Preliminaries .................................................. 3 1.1 Basic results for Markov chains . .3 1.2 Three solution concepts . 11 1.2.1 Reversibility . 12 1.2.2 Partial balance . 13 1.2.3 Kelly’s lemma . 13 2 Reversibility, Poisson flows and feedforward networks. 15 2.1 The birth-death process. 15 2.2 Detailed balance . 18 2.3 Erlang loss networks . 21 2.4 Reversibility . 23 2.5 Burke’s theorem and feedforward networks of MjMj1 queues . 25 2.6 Literature . 28 3 Partial balance and networks with Markovian routing . 29 3.1 Networks of MjMj1 queues . 29 3.2 Kelly-Whittle networks. 35 3.3 Partial balance . 39 3.4 State-dependent routing and blocking protocols . 44 3.5 Literature . 50 4 Kelly’s lemma and networks with fixed routes ..................... 51 4.1 The time-reversed process and Kelly’s Lemma . 51 4.2 Queue disciplines . 53 4.3 Networks with customer types and fixed routes . 59 4.4 Quasi-reversibility . 62 4.5 Networks of quasi-reversible queues with fixed routes . 68 4.6 Literature . 70 v Part I Solution concepts for Markovian networks of queues Chapter 1 Preliminaries This chapter reviews and discusses the basic assumptions and techniques that will be used in this monograph. Proofs of results given in this chapter are omitted, but can be found in standard textbooks on Markov chains and queueing theory, e.g. [?, ?, ?, ?, ?, ?, ?]. Results from these references are used in this chapter without reference except for cases where a specific result (e.g.
    [Show full text]
  • An Overflow Loss Network Model for Capacity Plan
    CORE Metadata, citation and similar papers at core.ac.uk Provided by STORE - Staffordshire Online Repository An Overflow Loss Network Model for Capacity Plan- ning of a Perinatal Network Md Asaduzzaman and Thierry J. Chaussalet University of Westminster, London, UK. Summary. In this paper, a model framework is developed to solve capacity planning problems faced by many perinatal networks in the UK. We propose a loss network model with overflow based on a continuous-time Markov chain for a perinatal network with specific application to a network in London. We derive the steady state expressions for overflow and rejection probabilities for each neonatal unit of the network based on a decomposition approach. Results obtained from the model are very close to observed values. Using the model, decisions on number of cots can be made for specific level of admission acceptance probabilities for each level of care at each neonatal unit of the network and specific levels of overflow to temporary care. Keywords: Queueing network model; Decomposition; Rejection; Continuous-time Markov chain 1. Introduction Every year over 80,000 (approximately 10%) neonates are born premature, very sick, or very small and require some form of specialist support in England (DH, 2003; RCPCH, 2007). Neonatal ser- vices aim to offer high quality care for these vulnerable babies. Over a six month period in 2006-07, neonatal units were shut to new admissions for an average of 24 days. One in ten units exceeded its capacity for intensive care for more than 50 days during a six month period (Bliss, 2007). The Na- tional Audit Office reported that capacity and staffing problems at unit level continue to constrain neonatal service (NAO, 2007).
    [Show full text]
  • Lecture Notes on Stochastic Networks
    Lecture Notes on Stochastic Networks Frank Kelly and Elena Yudovina Contents Preface page ix Overview 1 Queueing and loss networks 2 Decentralized optimization 4 Random access networks 5 Broadband networks 6 Internet modelling 8 Part I 11 1 Markov chains 13 1.1 Definitions and notation 13 1.2 Time reversal 16 1.3 Erlang’s formula 18 1.4 Further reading 21 2 Queueing networks 22 2.1 An M/M/1 queue 22 2.2 A series of M/M/1 queues 24 2.3 Closed migration processes 26 2.4 Open migration processes 30 2.5 Little’s law 36 2.6 Linear migration processes 39 2.7 Generalizations 44 2.8 Further reading 48 3 Loss networks 49 3.1 Network model 49 3.2 Approximation procedure 51 3.3 Truncating reversible processes 52 v vi Contents 3.4 Maximum probability 57 3.5 A central limit theorem 61 3.6 Erlang fixed point 67 3.7 Diverse routing 71 3.8 Further reading 81 Part II 83 4 Decentralized optimization 85 4.1 An electrical network 86 4.2 Road traffic models 92 4.3 Optimization of queueing and loss networks 101 4.4 Further reading 107 5 Random access networks 108 5.1 The ALOHA protocol 109 5.2 Estimating backlog 115 5.3 Acknowledgement-based schemes 119 5.4 Distributed random access 125 5.5 Further reading 132 6 Effective bandwidth 133 6.1 Chernoff bound and Cramer’s´ theorem 134 6.2 Effective bandwidth 138 6.3 Large deviations for a queue with many sources 143 6.4 Further reading 148 Part III 149 7 Internet congestion control 151 7.1 Control of elastic network flows 151 7.2 Notions of fairness 158 7.3 A primal algorithm 162 7.4 Modelling TCP 166 7.5 What is being
    [Show full text]
  • Lecture Notes on Stochastic Networks
    Lecture Notes on Stochastic Networks Frank Kelly and Elena Yudovina Contents Preface page viii Overview 1 Queueing and loss networks 2 Decentralized optimization 4 Random access networks 5 Broadband networks 6 Internet modelling 8 Part I 11 1 Markov chains 13 1.1 Definitions and notation 13 1.2 Time reversal 16 1.3 Erlang’s formula 18 1.4 Further reading 21 2 Queueing networks 22 2.1 An M/M/1 queue 22 2.2 A series of M/M/1 queues 24 2.3 Closed migration processes 26 2.4 Open migration processes 30 2.5 Little’s law 36 2.6 Linear migration processes 39 2.7 Generalizations 44 2.8 Further reading 48 3 Loss networks 49 3.1 Network model 49 3.2 Approximation procedure 51 v vi Contents 3.3 Truncating reversible processes 52 3.4 Maximum probability 57 3.5 A central limit theorem 61 3.6 Erlang fixed point 67 3.7 Diverse routing 71 3.8 Further reading 81 Part II 83 4 Decentralized optimization 85 4.1 An electrical network 86 4.2 Road traffic models 92 4.3 Optimization of queueing and loss networks 101 4.4 Further reading 107 5 Random access networks 108 5.1 The ALOHA protocol 109 5.2 Estimating backlog 115 5.3 Acknowledgement-based schemes 119 5.4 Distributed random access 125 5.5 Further reading 132 6 Effective bandwidth 133 6.1 Chernoff bound and Cramer’s´ theorem 134 6.2 Effective bandwidth 138 6.3 Large deviations for a queue with many sources 143 6.4 Further reading 148 Part III 149 7 Internet congestion control 151 7.1 Control of elastic network flows 151 7.2 Notions of fairness 158 7.3 A primal algorithm 162 7.4 Modelling TCP 166 7.5 What is being
    [Show full text]
  • Delay Asymptotics and Bounds for Multitask Parallel Jobs
    Queueing Systems (2019) 91:207–239 https://doi.org/10.1007/s11134-018-09597-5 Delay asymptotics and bounds for multitask parallel jobs Weina Wang1,2 · Mor Harchol-Balter2 · Haotian Jiang3 · Alan Scheller-Wolf4 · R. Srikant1 Received: 10 November 2018 / Published online: 16 January 2019 © Springer Science+Business Media, LLC, part of Springer Nature 2019 Abstract We study delay of jobs that consist of multiple parallel tasks, which is a critical performance metric in a wide range of applications such as data file retrieval in coded storage systems and parallel computing. In this problem, each job is completed only when all of its tasks are completed, so the delay of a job is the maximum of the delays of its tasks. Despite the wide attention this problem has received, tight analysis is still largely unknown since analyzing job delay requires characterizing the complicated correlation among task delays, which is hard to do. We first consider an asymptotic regime where the number of servers, n, goes to infinity, and the number of tasks in a job, k(n), is allowed to increase with n. We establish the asymptotic independence of any k(n) queues under the condition k(n) = o(n1/4). This greatly generalizes the asymptotic independence type of results in the literature, where asymptotic indepen- dence is shown only for a fixed constant number of queues. As a consequence of our independence result, the job delay converges to the maximum of independent task delays. We next consider the non-asymptotic regime. Here, we prove that inde- pendence yields a stochastic upper bound on job delay for any n and any k(n) with k(n) ≤ n.
    [Show full text]
  • Heavy-Traffic Optimality of a Stochastic
    Heavy-Traffic Optimality of a Stochastic Network under Utility-Maximizing Resource Control Heng-Qing Ye∗ Dept of Decision Science National University of Singapore, Singapore David D. Yaoy Dept of Industrial Engineering and Operations Research Columbia University, New York, USA December 2005 Abstract We study a stochastic network that consists of a set of servers processing multiple classes of jobs. Each class of jobs requires a concurrent occupancy of several servers while being processed, and each server is shared among the job classes in a head-of-the-line processor- sharing mechanism. The allocation of the service capacities is a real-time control mechanism: in each network state, the control is the solution to an optimization problem that maximizes a general utility function. Whereas this resource control optimizes in a \greedy" fashion, with respect to each state, we establish its asymptotic optimality in terms of (a) deriving the fluid and diffusion limits of the network under this control, and (b) identifying a cost function that is minimized in under the diffusion limit, along with a characterization of the so-called fixed point state of the network. Keywords: stochastic processing network, concurrent resource occupancy, utility-maximizing resource control, fluid limit, diffusion limit, resource pooling, heavy-traffic optimality, Lyapunov function. ∗Supported in part by the grant R-314-000-061-112 of National University Singapore. ySupported in part by NSF grant CNS-03-25495, and by HK/RGC Grant CUHK4179/05E. 1 1 Introduction We study a class of stochastic networks with concurrent occupancy of resources, which, in turn, are shared among jobs.
    [Show full text]
  • Economies-Of-Scale in Many-Server Queueing Systems: Tutorial and Partial Review of the QED Halfin-Whitt Heavy-Traffic Regime
    Economies-of-scale in many-server queueing systems: tutorial and partial review of the QED Halfin-Whitt heavy-traffic regime Johan S.H. van Leeuwaarden Britt W.J. Mathijsen Bert Zwart July 30, 2019 Abstract Multi-server queueing systems describe situations in which users require service from multiple parallel servers. Examples include check-in lines at airports, waiting rooms in hospitals, queues in contact centers, data buffers in wireless networks, and delayed service in cloud data centers. These are all situations with jobs (clients, patients, tasks) and servers (agents, beds, processors) that have large capacity levels, rang- ing from the order of tens (checkouts) to thousands (processors). This survey investigates how to design such systems to exploit resource pooling and economies-of-scale. In particular, we review the mathematics behind the Quality-and-Efficiency Driven (QED) regime, which lets the system operate close to full utiliza- tion, while the number of servers grows simultaneously large and delays remain manageable. Aimed at a broad audience, we describe in detail the mathematical concepts for the basic Markovian many-server system, and only provide sketches or references for more advanced settings related to e.g. load balancing, overdispersion, parameter uncertainty, general service requirements and queueing networks. While serving as a partial survey of a massive body of work, the tutorial is not aimed to be exhaustive. Contents 1 Introduction 2 5.1 Bounds . 19 5.2 Optimality gaps . 21 2 Example models 6 5.3 Refinements . 22 2.1 Many exponential servers . .6 2.2 Bulk-service queue . 11 6 Extensions 23 arXiv:1706.05397v2 [math.PR] 28 Jul 2019 3 Key QED properties 14 6.1 Abandonments .
    [Show full text]
  • A Heavy Traffic Limit Theorem for a Class of Open Queueing Networks With
    Queueing Systems 32 (1999) 5±40 5 A heavy traf®c limit theorem for a class of open queueing networks with ®nite buffers ∗ J.G. Dai a and W. Dai b,∗∗ a School of Industrial and Systems Engineering, and School of Mathematics, Georgia Institute of Technology, Atlanta, GA 30332-0205, USA E-mail: [email protected] b School of Mathematics, Georgia Institute of Technology, Atlanta, GA 30332-0160, USA Submitted 1 February 1998; accepted 1 December 1998 We consider a queueing network of d single server stations. Each station has a ®nite capacity waiting buffer, and all customers served at a station are homogeneous in terms of service requirements and routing. The routing is assumed to be deterministic and hence feedforward. A server stops working when the downstream buffer is full. We show that a properly normalized d-dimensional queue length process converges in distribution to a d-dimensional semimartingale re¯ecting Brownian motion (RBM) in a d-dimensional box under a heavy traf®c condition. The conventional continuous mapping approach does not apply here because the solution to our Skorohod problem may not be unique. Our proof relies heavily on a uniform oscillation result for solutions to a family of Skorohod problems. The oscillation result is proved in a general form that may be of independent interest. It has the potential to be used as an important ingredient in establishing heavy traf®c limit theorems for general ®nite buffer networks. Keywords: ®nite capacity network, blocking probabilities, loss network, semimartingale re¯ecting Brownian motion, RBM, heavy traf®c, limit theorems, oscillation estimates 1.
    [Show full text]
  • Continuous-Time Markov Chains
    CONTINUOUS-TIME MARKOV CHAINS by Ward Whitt Department of Industrial Engineering and Operations Research Columbia University New York, NY 10027-6699 Email: [email protected] URL: www.columbia.edu/∼ww2040 December 4, 2013 c Ward Whitt Contents 1 Introduction 1 2 Transition Probabilities and Finite-Dimensional Distributions 2 3 Modelling 4 3.1 A DTMC with Exponential Transition Times . ...... 6 3.2 TransitionRatesandODE’s. .... 7 3.3 Competing Clocks with Exponential Timers . ........ 11 3.4 Uniformization: A DTMC with Poisson Transitions . .......... 14 4 Birth-and-Death Processes 16 5 Stationary and Limiting Probabilities for CTMC’s 23 6 Reverse-Time CTMC’s and Reversibility 31 7 Open Queueing Networks (OQN’s) 36 7.1 TheModel ...................................... 36 7.2 The Traffic Rate Equations and the Stability Condition . .......... 36 7.3 The Limiting Product-Form Distribution . ......... 37 7.4 Extensions: More Servers or Different Service SchedulingRules ......... 39 7.5 Steady State in Discrete and Continuous Time . ........ 40 8 Closed Queueing Networks (CQN’s) 41 8.1 Why Are Closed Models Interesting? . ...... 41 8.2 A Normalized Product-Form Distribution . ........ 42 8.3 Computing the Normalization Constant: The Convolution Algorithm . 44 9 Stochastic Loss Models 45 9.1 TheErlangLossModel .............................. 45 9.2 StochasticLossNetworks . 47 9.3 InsensitivityintheErlangLossModel . ........ 48 10 Regularity and Irregularity in Infinite-State CTMC’s 50 10.1 Instantaneous Transitions, Explosions and the Minimal Construction . 51 10.2 Conditions for Regularity and Recurrence . .......... 52 11 More on Reversible CTMC’s and Birth-and-Death Processes 53 11.1 Spectral Representation in Reversible Markov Chains . .............. 53 11.2 FittingBDProcessestoData . ..... 55 11.3 ComparingBDprocesses.
    [Show full text]
  • Queuing Networks in Healthcare Systems
    Queuing Networks in Healthcare Systems Maartje E. Zonderland and Richard J. Boucherie Abstract Over the last decades, the concept of patient flow has received an increased amount of attention. Healthcare professionals have become aware that in order to analyze the performance of a single healthcare facility, its relationship with other healthcare facilities should also be taken into account. A natural choice for analysis of networks of healthcare facilities is queuing theory. With a queuing network a fast and flexible analysis is provided that discovers bottlenecks and allows for the evaluation of alternative set-ups of the network. In this chapter we describe how queuing theory, and networks of queues in particular, can be invoked to model, study, analyze and solve healthcare problems. We describe important theoretical queuing results, give a review of the literature on the topic, discuss in detail two examples of how a healthcare problem is analyzed using a queuing network, and suggest directions for future research. 1 Introduction With an aging population, the rising cost of new medical technologies, and a soci- ety wanting higher quality care, the demand for healthcare is increasing annually. In European countries, such as the Netherlands, healthcare expenditures consume around 10% of the GDP. In the United States this percentage is even bigger at 16% [45] (2008 data). Since the supply of healthcare is finite, policy makers have to ra- tion care and make choices on how to distribute physical, human, and monetary resources. Such choices also have to be made at the hospital level (e.g., which pa- Maartje E. Zonderland · Richard J.
    [Show full text]