Bibliographic Notes

Rather than providing a complete bibliography, our intention is to guide the reader into parts of the literature that we have found relevant for supplementary reading. For well-established results, we usually refer to textbooks or research monographs, whereas original sources are cited for specialized and more recent results. We apolo- gize for the inevitable (and entirely unintentional) omission of important references.

Chapter 1

The infinitesimal approach to the Poisson process can be found in C¸inlar[69]or Karlin and Taylor [118]. For an extensive treatment of the Poisson process, also in higher dimensions, we refer to Kingman [120]. Our main references to discrete state-space Markov chains are Asmussen [18], C¸inlar[69], Karlin and Taylor [118], and Bremaud [59]. The material on Markov jump processes is mainly taken from Asmussen [18]andC¸inlar[69]. For a gentle introduction to Markov chains and processes (without measure theory), we suggest Hoel, Port, and Stone [104]. Discrete phase-type distributions were introduced in Neuts [150] and also treated in Neuts [153]. Distributions of jumps and sojourn times in Markov jump processes are treated in Bladt et al. [47]. The embedding problem is treated in general in Kingman [121] and was characterized in three dimensions in Johansen [110]. Uni- formization goes back at least to Jensen [109]. The coupling proof of the conver- gence of transition for both Markov chains and jump processes is taken from Lindvall [131] and Asmussen [18]. See also Thorisson [188] for further prop- erties of coupling methods and their relationship to regeneration. Properties of Kronecker sums and products can be found in Graham [93].

© Springer Science+Business Media LLC 2017 703 M. Bladt, B.F. Nielsen, Matrix-Exponential Distributions in Applied , and Stochastic Modelling 81, DOI 10.1007/978-1-4939-7049-0 704 Bibliographic Notes Chapter 2

The main references on martingales in discrete time are Neveu [156] and Doob [77]. For continuous-time martingales, we refer to Revuz [173]. The central Martingale of Section 2.2.4 is adapted from Hall and Heyde [97]. The martingale relation to Example 2.2.8 can be found in Li [129] and Pozdnyakov and Kulldorff [166]. For further reading on general Markov processes, we suggest Kallenberg [114], Ethier and Kurtz [81], Revuz [172], Dynkin [79], and Loeve` [133, 134]. For , see, e.g., Kallianpur [115], Schilling [178], or Karatzas and Shreve [117].

Chapter 3

Erlang [65] is usually accredited for the method of stages. However, as was pointed out in [72], the idea occurred earlier, in works by Ellis [80]. The idea of extending the method of stages to absorption times of Markov jump processes is due to Jensen [108]. Neuts [150] started research on phase-type distributions and their application in queueing models; Neuts [153]. See also Latouche and Ramaswami [126], As- mussen [18], Breuer and Baum [63], Asmussen and Albrecher [20], He [99], and Nelson [149]. The denseness of phase-type distributions is due to Schassberger [177]. The proof of Theorem 3.2.9, p. 153, is taken from Johnson [111]. The minimal closure char- acterization (Theorem 3.2.10, p. 153) is from Maier and O’Cinneide [139]. Strassen’s theorem is a deep result in probability theory and was first proved byStrassenin[185] and later also by Dellacherie and Meyer in [75]. Both these proofs are based on functional analysis. Hoffman–Jørgensen proved the result in the preliminaries of [105] using an entirely measure-theoretic approach. For the case of R, a simple proof based on stop-loss transforms has more recently become available in Stoyan and M¨uller [147]. The proof of Theorem 3.3.8, p. 162, is adapted from Asmussen [19] (private communication). The minimality of the coefficient of variation for the distributions is originally due to Aldous and Shepp [7], who employed a martingale approach. The result was later generalized and proved using Strassen’s theorem and convex ordering in O’Cinneide [163]. For discrete phase-type distributions, a similar result was proved by Telek in [186]. For a more extensive treatment of stochastic ordering, see, e.g., Shaked and Shantikumar [181]. Functional calculus is treated in Haase [96] for general sectorial operators. A nice condensed paper on functions of matrices, defined in terms of the Cauchy integral, is Doolittle [78]. For all other methods of dealing with functions of matrices, we refer to Higham [103]. The application of functional calculus to phase-type distributions in Section 3.4, p. 169, is from Bladt, Campillo, and Nielsen [44]. The material on infinite-dimensional phase-type distributions (NPH) in Sec- tion 3.5, p. 180, is from Bladt, Nielsen, and Samorodnitsky [52]. For results on heavy-tailed distribution, regular variation, and Breiman’s theorem, we refer to Bibliographic Notes 705

Bingham et al. [41]. While the distributions of NPH may be genuinely heavy-tailed, other papers have dealt with the approximation of heavy-tailed distributions using phase-type distributions and Markovian arrival processes; see, e.g., Feldmann and Whitt [88] for the former and Andersen and Nielsen [9, 13, 14] for the latter.

Chapter 4

Distributions with rational Laplace transforms were discussed in Cox [71]. It is standard in linear systems theory (Farina and Rinaldi [87], Astr˚ om¨ [33]) that matrix- exponential functions are exactly those having a rational Laplace transform. The proof that the dominant eigenvalue is real is taken from Fackrell [84]. A companion matrix approach was given in Asmussen and Bladt [23]. The idea of Lemma 4.2.5, p. 212, is from O’Cinneide [161]. Theorem 4.2.16, p. 216, for matrix-exponential distributions is from Asmussen and Bladt [23]. The ideas were applied to RAPs in Buchholz and Telek [66]. Details on the poles of rational Laplace transforms has been treated partially in O’Cinneide [162], where also the characterization in terms of a finite-dimensional vector space spanned by the residual life operator is given. General algebraic prop- erties and related linear vector spaces has been taken from Asmussen and Bladt [23]. The material on flow interpretation is based on Bladt and Neuts [48] and Bean and Nielsen [38]. The expression of matrix-exponential distributions in terms of their moments using Hankel matrices, and the resulting order determination is based on Bladt and Nielsen [49] and has previously been noted in the unpublished technical report Van de Liefvoort [130].SeealsoHe[101] and Bodrog, Horvath,´ and Telek [58]. The method using permanents is taken from the survey paper by Balakrishnan [36] on order for i.n.i.d., i.e., independent nonidentically distributed, ran- dom variables. Abdelkader [2] obtains Laplace transforms of order statistics for i.n.i.d. phase-type distributed random variables. The distribution of the spread for a phase-type renewal process was discussed in Kao [116] but without providing a specific representation. The latter was derived in Bladt and Nielsen [51], and with a proof based on functional calculus in Bladt, Campillo, and Nielsen [44]. The characterization of phase-type distributions in terms of their densities is orig- inally due to O’Cinneide [162]. The proof leans on convex analysis, for which we refer to Rockafellar [174] and on semigroup theory (see, e.g., Davis [73]). The con- cept of invariant polytopes and the residual life operator were introduced to show that the sufficient conditions ensured that the distribution could be written as a con- vex combination of phase-type distributions and was thus of phase type. A proof based on automata theory was given in Maier [138]. An algebraic proof was given in Horvarth´ and Telek [106]. The latter is somewhat related to unicyclic representa- tions, which can be found in Mocanu and Commault [144] based on the terminology of O’Cinneide [162]. See also Fackrell et al. [83] for related work. 706 Bibliographic Notes Chapter 5

For general we refer to Asmussen [18], C¸inlar[69], and Lindvall [131]. Renewal reward processes of Section 5.2.4, p. 320, are from Sigman and Wolff [183]andWolff[192]. Anscombe’s theorem (Theorem 5.2.20, 321) is from Gut [95]. Phase-type renewal theory is now well established and can be found, e.g., in Neuts [153] and Latouche and Ramaswami [126]. Moment distributions of phase- type (Section 5.5, p. 330) are from Bladt and Nielsen [51]. Renewal theory for matrix-exponential distributions is from Asmussen and Bladt [23] and Bladt and Neuts [48], while renewal theory for phase-type distributions is presented in Neuts [151]. Reversible phase-type representations seem to have appeared first in Ramaswami [168]. The concept was used by Andersen, Neuts, and Nielsen [11, 12].

Chapter 6

The main references to random walks are Feller [89] and Asmussen [18]. Random walks with a phase-type component (Section 6.2.1, p. 368) are from Asmussen [15]. For a matrix-exponential component, we refer to Asmussen and Bladt [23]. The occupation measures (6.6) and (6.7), both p. 365, are from Pitman [165]. For application of phase-type and matrix-exponential distributions in , we refer to the monographs by Neuts [153, 154], Asmussen [18], [132], Nelson [149], Latouche and Ramaswami [126], Breuer [63], Bini, Latouche and Meini [42], and He [99]. For a classical introduction to queueing theory with an emphasis on rational transform methods, we refer to Cohen [70]. Some important papers and introductions to queues using matrix-analytic methods are Lucantoni [135, 136], Latouche and Ramaswami [127], Gail, Hantler, and Taylor [91], Bright and Taylor [64], Bean, Pollett, and Taylor [37]. Concerning applications of random walks and phase-type distributions in risk theory, we refer to Asmussen [30, 27]. A classical reference on Wiener–Hopf factorizations for random walks and ad- ditive processes is Greenwood [94], while Asmussen [17] provides a probabilistic view.

Chapter 7

Regenerative processes in the classical sense go back at least to Smith [184]. Wide- sense regeneration can be found, e.g., in Thorisson [188], Kalashnikov [113], and Asmussen [18]. Limit theorems for regenerative processes are from Asmussen [18] andSigmanandWolff[183]. The general coupling inequality (Section 7.2, p. 396) can be found, e.g., in Lindvall [131] and Thorisson [188]. The construction of Harris Bibliographic Notes 707 chains in terms of a geometric trial argument on a regeneration set is due to Num- melin [158], which may also be found in his book [159], and is generally referred to as Nummelin splitting. Monte Carlo methods (Section 7.3.2, p. 404) are usually defined in terms of general Markov chains, in particular for Harris re- current Markov chains, where the convergence is guaranteed. See Asmussen and Glynn [25] for a simplified proof of the convergence of MCMC methods. Poisson’s equation has been studied in several places, see, e.g., Nummelin [159], but for the application to waiting time averages in the M/G/1 queue, we refer to Glynn [92]. For extension of this model to the PH/PH/1 queue we refer to Bladt [43], and for the MAP/G/1 setting, to Asmussen and Bladt [22]. The calculation of time averages of data raised to any fractional power is new.

Chapter 8

The first formulation of multivariate phase-type distributions, the MPH class of Sec- tion 8.1.5, was presented in Assaf et al. [32]. This class was generalized to the MPH∗ class of Section 8.1 by Kulkarni [124]. Both these classes are contained in the MVME class of Bladt and Nielsen [49], which in turn characterizes multivariate nonnegative distributions with rational Laplace transforms. For multivariate and bivariate exponential distributions we refer to Balakrish- nan and Basu [35] and and Kotz, Balakrishnan, and Johnson [123]. Regarding Sec- tion 8.3, p. 459, we refer to Bladt and Nielsen [50] and He, Zhang, and Vera [102]. Univariate bilateral phase-type distributions go back to Shantikumar [182]and were later treated by Ahn and Ramaswami [5]. The multivariate bilateral distribu- tions (Section 8.5, p. 469) with a rational Laplace transform were introduced and characterized in Bladt and Nielsen [45].

Chapter 9

The foundation of Markov additive processes was treated extensively in the two papers by C¸inlar[67, 68]. Sections 9.4, 9.5, and 9.6 on fluid flow processes are from Asmussen [16]. A more compact version of Theorem 9.4.2 is possible by applying a uniformization argument (see Asmussen [16] and Problem 9.6.6, p. 514), but it is our experience that the speed of convergence of the uniformized procedure is substantially slower. Markov additive processes have received considerable attention recently, in par- ticular regarding Levy´ processes and their applications to insurance and finance (Mordecki [145, 146], Breuer [61], Asmussen [21], all of whom consider Levy´ pro- cesses with phase-type or matrix-exponentially distributed jumps in one or both directions). A special case of the mixed fluid model of Section 9.6.2 is equivalent to Levy´ processes discarding the small jumps. 708 Bibliographic Notes

Fluid flow models have also been investigated by, e.g., Ramaswami [169]and Ahn and Ramaswami [4] using matrix analytic methods. These two papers were a major inspiration for several other papers on fluid modeling.

Chapter 10

The Markovian arrival process (MAP) goes at least back to Rudemo [175]. It was later published as a versatile Markovian in Neuts [152] as an extension of the PH renewal process (Neuts [151]). The modern parametrization was given in Lucantoni [137]. Properties of MAPs, and their extension to marked MAPs, batch MAP or BMAP, can be found in the monographs Neuts [153], Neuts [154], and Latouche and Ramaswami [126]. Interpretation of BMAPs as marked MAPs can be found in He and Neuts [100]. Formulas for the case of the Markov modulated Poisson process are collected in Fisher and Meier-Hellstern [90]. MAPs are dense in the class of point processes on the real line, a fact that has not been proved in this book. A proof can be found in Asmussen and Koole [27]. The material on the MAP/G/1 queue is from Asmussen and Bladt [22]. The rational arrival process (RAP) was first introduced in Asmussen and Bladt [24] using the characterization of Section 10.5, while Mitchell [143], seemingly unaware of [24], used Theorem 10.5.2 (a) to define a sequence of correlated matrix- exponential random variables. Similarity transformations of MAPs were discussed in Andersen, Barker, and Nielsen [10] and in Telek and Horvath´ [187]. The RAP has been extended to a batch RAP and was applied to queues in Bean and Nielsen [38]. The staircase algorithm for dimension reduction in RAPs is described in Buch- holz and Telek [66]. The description in Neuts [154] is based on the versatile Marko- vian arrival process parametrization, while Latouche and Ramaswami [126]havea condensed description of MAPs. Moments are discussed in Narayana and Neuts [148]andNielsenetal.[157]. The calculation of counting probabilities in a MAP is described in Li and Neuts [155] using uniformization. Transient MAPs are introduced and analyzed in Latouche, Remiche, and Taylor [128].

Chapter 11

Risk processes with reserve-dependent premiums and phase-type claims are taken from Asmussen and Bladt [31]. The calculations, however, are performed in a slightly different manner, with more emphasis on providing exact conditions on the premium function, which determines the development of the risk reserve pro- cess between claims through an autonomous differential equation. The model with Bibliographic Notes 709 matrix-exponential claims is from Bladt and Neuts [48]. A duality result to storage models can be found in Asmussen and Schock Petersen [29]. The reserve dependent risk model with MAP-modulated arrivals is a slight extension of the model from Asmussen and Bladt [31]. Modeling with infinite- dimensional phase-type distributions is taken from Bladt, Nielsen, and Samorod- nitsky [52]. Risk models with Brownian components is an application of Asmussen [16]. For the latter, we used a fluid interpretation in order to calculate ruin proba- bilities and severity in terms of some explicit formulas. One can, however, also see these models with a Brownian component as Levy´ processes, on which there exists a rapidly growing literature covering this approach. We refer, e.g., to Kyprianou [125] and references therein or Asmussen and Albrecher [20]. For risk processes with matrix-exponential claims, where the focus is on the rational form of the Laplace transform and their roots, we refer to Mordecki [146], Jacobsen [107], and Breuer [61, 62].

Chapter 12

The main references to inference for Markov chains and jump processes is the sur- vey article [40] and the monograph [39], both by Billingsley, in which also the rela- tionship between transition data and contingency tables is treated. The methods for binomial and multinomial distributions is taken from Andersen [8]. For a reference in English, see, e.g., Agresti and Kateri [3]. The EM algorithm is due to Dempster, Laird, and Rubin [76], while the extraction of the information matrix is from Oakes [160]. Inference for Poisson processes can be found in numerous places, including Albrecht [6].

Chapter 13

The EM algorithm for uncensored data is due to Asmussen, Nerman, and Olsson [28]. In our treatment of the theory, we have changed the use of a Riemann approx- imation to a shorter transform argument, and the computation of certain “unsolved” integrals in [28] can be explicitly computed by applying a method from Van Loan [190]. Results involving censoring can be found in Olsson [164]. Software relating to the EM algorithm is available as a C program for free download from http:// home.math.au.dk/asmus/pspapers.html. Bobbio et al. [57] performed estimation of acyclic phase-type distributions. The MCMC method for estimation is from Bladt, Gonzalez, and Lauritzen [46]. This method has been implemented in R by Louis Assletunderthe name “PhaseType” (s e e http://www.louisaslett.com\discretionary-/PhaseType/). Phase-type estimation of parameters in phase-type distributions is complicated due to the problems concerning identifiability. If we restrict attention to certain 710 Bibliographic Notes subclasses of phase-type distributions and ordering states according to increasing or decreasing intensities, these subclasses may produce unique representations. In Ausin et al. [34], a reversible jump Markov chain Monte Carlo was produced, where also the order of the phase-type distribution may be estimated, for the class of hyper- Erlang distributions. See also Buchholz and Telek [189] for estimation on the hyper- Erlang. Phase-type distributions have been fitted to real data in a number of applications; see e.g., Marshal et al. [140, 141] and Faddy [85, 86]. Aalen [1] surveys the poten- tial use of phase-type distributions more generally in survival analysis, where the phases have a physical interpretation. As mentioned in the chapter, the estimation of discretely observed phase-type distributions closely follows this idea. Importance sampling for Markovian bridges of jump processes (Asmussen and Hobolth [26]) may improve on the simple rejection-based simulation scheme pro- posed in this book. Moment-based methods for fitting phase-type distributions can be found in Johnson [112], a mixed Erlang case in Schmickler [179], and a mixed exponential case in Harris and Sykes [98]. Bobbio and Telek [55, 56] treated the case with an upper triangular subintensity matrix. For further discussion and bibli- ographic notes on earlier estimation methods, see Asmussen, Nerman, and Olsson [28]. Dependent phase-type distributions have been estimated in terms of point pro- cesses in Meier-Hellstern [142], Ryden´ [176], and Breuer [60] (respectively Markov- modulated Poisson processes and batch MAP). Klemm, Lindemann, and Lohman [122] used an EM algorithm with uniformization to estimate parameters in BMAPs. Inference for discretely observed Markov jump processes appeared in Bladt and Sørensen [53] with further properties and tests in Bladt and Sørensen [54]. A procedure for maximum likelihood fitting of matrix-exponential distributions (which are not necessarily of phase type) is given in Fackrell [82]and[119]foran EM algorithm using Pade approximations.

Appendix

A good source for formulas involving matrix inversion is Seber [180]. A classic on Kronecker products and sums is Graham [93]. The results on matrix-exponentials is taken from Van Loan [190] and Doolittle [78]. Appendix A

A.1 Matrix Inversion

Lemma A.1.1. Let A,B,C and D be matrices, with A and D nonsingular square matrices. Then     −1 −1 + −1 ( − −1 )−1 −1 − −1 ( − −1 )−1 AABB = A A B D CCAA B CCAA A B D CCAA B . CCDD −(D −CCAA−1B)−1CCAA−1 (D −CCAA−1B)−1

In particular,

(A − BBDD−1C)−1 = A−1 + A−1B(D −CCAA−1B)−1CCAA−1 whenever the terms make sense.

A.2 The Matrix Exponential and Its Properties

The exponential of a finite-dimensional square matrix A is defined as the Picard series ∞ An eA = ∑ , n=0 n! with A0 = I. It is clear that the matrix exponential satisfies d eAAx = AAeAAx = eAAxA. (A.1) dx In particular, if A is nonsingular, we have that  − eAAxdx = A 1eAAx. (A.2)

© Springer Science+Business Media LLC 2017 711 M. Bladt, B.F. Nielsen, Matrix-Exponential Distributions in , Probability Theory and Stochastic Modelling 81, DOI 10.1007/978-1-4939-7049-0 712 A Appendix

If A and B are square matrices of the same dimension, then

e(A+B)x = eAAxeBBx if A and B commute. Consequently, the inverse of eAAx is e−AAx. Let P be a nonsingular matrix and consider the similarity transformation B = PPAAAPP−1.Then ∞ xn eBBx = ∑ Bn n=0 n! ∞ n  n x = ∑ PPAAAPP−1 n! n=0  ∞ xn = P ∑ An P−1 n=0 n! = PPeAAxP−1. (A.3)

The matrix exponential of a diagonal matrix is a diagonal matrix of the correspond- ing exponential of the diagonal entries. This follows immediately from the fact that powers of diagonal matrices are again diagonal. More generally, the matrix ex- ponential of a block-diagonal matrix is again a block-diagonal matrix, where the blocks are the matrix exponentials of the corresponding blocks. An important issue is how to calculate the matrix exponential in practice. Here we essentially distinguish between symbolic and numerical methods, where the former are used for calculating matrix exponentials in situations in which the matrix is not numerical and may contain unknown constants or variables, while numerical methods refer to situations in which the matrix is entirely numerical and where speed and precision are often an issue, in particular if numerous evaluations are to take place as part of a larger program or implementation. A common method for symbolic calculation of the matrix exponential is via a diagonalization argument. If the n × n matrix A has n linearly independent eigen- vectors v1,...,vn with distinct eigenvalues λ1,...,λn,then ⎛ ⎞ λ1 0 ... 0 ⎜ λ ... ⎟ ⎜ 0 2 0 ⎟ − A = P ⎜ . . . . ⎟P 1, ⎝ . . . . ⎠ 00... λn where P is the matrix whose columns are the eigenvectors vi. Hence ⎛ ⎞ λ e 1x 0 ... 0 ⎜ λ ⎟ 0 e 2x ... 0 AAx ⎜ ⎟ −1 e = P ⎜ . . . . ⎟P . (A.4) ⎝ . . . . ⎠ λ 00... e nx A.2 The Matrix Exponential and Its Properties 713

More generally, if A is not diagonalizable, we may apply a Jordan decomposi- tion in the following way. If λ1,...,λk denote the k distinct eigenvalues of the n × n matrix, then for some P, A = PPJJJPP,where ⎛ ⎞ J1 0000 ... 0 ⎜ ⎟ ⎜ 0 J2 0 ... 0 ⎟ ⎜ ... ⎟ J = ⎜ 0000 J 3 0 ⎟ ⎜ . . . . ⎟ ⎝ ...... ⎠ 0000000 ... J n for some n,and ⎛ ⎞ λi 10... 0 ⎜ ⎟ ⎜ 0 λi 1 ... 0 ⎟ ⎜ 00λ ... 0 ⎟ Ji = ⎜ i ⎟. ⎜ . . . . ⎟ ⎝ ...... ⎠ 000... λi Then ⎛ ⎞ eJ1x 0 ... 0 ⎜ 0 eJ2x ... 0 ⎟ A ⎜ ⎟ − eAAx = P ⎜ . . . . ⎟P 1, (A.5) ⎝ . . . . ⎠ 0000 ... eJnx where ⎛ ⎞ λ λ 2 λ n λ e ix xe ix x e ix ... x i e ix ⎜ 2 ni! ⎟ λ λ n −1 λ ⎜ 0 e ix xe ix ... x i e ix ⎟ ⎜ (ni−1)! ⎟ − ⎜ λ x xni 2 λ x ⎟ exp(Jix)=⎜ 00e i .... e i ⎟. (A.6) ⎜ (ni−2)! ⎟ ⎜ . . . . ⎟ ⎝ ...... ⎠ λ 00 0... e ix Next we consider the matrix exponentials of certain block-partioned matrices. Theorem A.2.1 (Van Loan [190]). Let p,q ≥ 1 be integers. Consider a block- partitioned matrix   Λ = AABB , 0 C where AAisap× pmatrix,CCCaq×qmatrix,BBBap×q matrix, and 0 the q× pmatrix of zeros. Then   x  AAx A(x−z) CCz Λ x e e BBe dz e = 0 . (A.7) 0 eCCx 714 A Appendix

Proof. We have that    1 x+h x eA(x+h−z)BBeCCzdz− eA(x−z)BBeCCzdz h 0 0    x 1 = eA((x+h−z)) − eA(x−z) BBeCCzdz h 0  1 x+h + eA(x+h−z)BBeCCzdz  h x x → AAeA(x−z)BBeCCzdz+ eA·0BBeCCx 0 x = AAeA(x−z)BBeCCzdz+ BBeCCx, 0 where the convergence of the second integral is due to the mean value theorem. Let Λ (x) denote the right-hand side of (A.7). Then   x  d AAeAAx A eA(x−z)BBeCCzdz+ BBeCCx Λ (x)= 0 dx 0 CCeCCx    x  AAx A(x−z) CCz = AABB e e BBe dz C 0 0 C 0 eCCx   AABB = Λ (x). 0 C

Since Λ (0)=I, we obtain that Λ (x)=exp(Λ x) is indeed of the desired form.  There is a discrete version of the previous result as well. Theorem A.2.2. Let p,q ≥ 1 be integers and consider the block-partitioned matrix   Λ = AABB , 0 C where AAisap× pmatrix,CCCaq× qmatrix,BBBap× q matrix, and 0 the q × pzero matrix. Then for every n ∈ N, we have that

⎛ n ⎞ An+1 ∑ An−kBBCCk Λ n+1 = ⎝ ⎠. k=0 + 0 Cn 1

Proof. Follows by a simple induction argument. 

Definition A.2.3. Let U (x)={uij(x)} be a matrix that depends on some real (or complex) variable x. Then we define   ∂ ∂ U (x)= u (x) . ∂x ∂x ij

That is, matrix differentiation is performed entrywise. A.2 The Matrix Exponential and Its Properties 715

We have the following product rule. Theorem A.2.4. Let U (x) and V (x) denote two matrices of compatible orders such that U(x)V (x) is well defined. Then

∂ ∂U (x) ∂V (x) (U (x)V (x)) = V (x)+U(x) . ∂x ∂x ∂x Proof.    ∂ ∂ (U (x)V (x)) = ∑u (x)v (x) ∂x ∂x ik kj  k  = (  ( ) ( )+ ( )  ( )) ∑ uik x vkj x uik x vkj x  k    =  ( ) ( ) + ( )  ( ) ∑uik x vkj x ∑uik x vkj x k k ∂U (x) ∂V (x) = V (x)+U (x) . ∂x ∂x 

Let E ij = {δij} denote the matrix that is zero everywhere except for the element ij, which is 1. Then we have the following important special case.

Corollary A.2.5. Let X = {xij} be a square matrix. Then

∂ n−1 X n = X kE X n−1−k. ∂ X ∑ X E ijX xij k=0 This can also be written as     ∂   XXEE n 0 X n = I 0 ij , ∂xij 0 X I where I and 0 are the identity and zero matrices of the same order as XX.

Proof. Follows by induction and successive use of Theorem A.2.4. The second part follows directly from Theorem A.2.2. 

Theorem A.2.6. Let X = {xij} be a square matrix. Then    ∂   XXEE 0 eX = I 0 exp ij , ∂xij 0 X I where I and 0 are the identity and zero matrices of the same order as XX.

Proof. Follows directly from differentiating under the expansion of eX .  716 A Appendix A.3 Solving AAXX − XXBB = C

Theorem A.3.1. Let A and B be square matrices such that their spectra (sets of eigenvalues) are disjoint. Let γ be a path (contour) enclosing the eigenvalues for B such that no eigenvalue for A lies within or on this contour. Then  1 X = − (zII − A)−1C(zII − B)−1dz 2πi γ is the unique solution to AAXX − XXBB = C. Proof. First we prove that the proposed X is indeed a solution. Since A commutes with (zII − A)−1 and B with (zII − B)−1, we get that  1 AAXX − XXBB = − (zII − A)−1(AACC −CCBB)(zII − B)−1dz 2πi γ  1 = (zII − A)−1(zCC − AACC +CCBB − zCC)(zII − B)−1dz 2πi γ  1 = (C(zII − B)−1 − (zII − A)−1C)dz 2πi γ = C, since γ encloses the spectrum for B only, implying that the second integral vanishes (due to the Cauchy integral theorem). To prove that it is the only solution, we proceed as follows. First we notice that for an arbitrary X ,  1 (zII − A)−1AAXX(zII − B)−1dz 2πi γ  1 = (zII − A)−1(A − zII + zII)X (zII − B)−1dz 2πi γ   1 1 = − X (zII − B)−1dz+ z(zII − A)−1X (zII − B)−1dz 2πi γ 2πi γ  1 = −X + z(zII − A)−1X (zII − B)−1dz. 2πi γ

Similarly,  1 (zII − A)−1XXBB(zII − B)−1dz 2πi γ  1 = (zII − A)−1X (B − zII + zII)(zII − B)−1dz 2πi γ   1 1 = − (zII − A)−1XXdz+ z(zII − A)−1X (zII − B)−1dz 2πi γ 2πi γ  1 = 0 + z(zII − A)−1X (zII − B)−1dz. 2πi γ A.4 Kronecker Notation 717

From subtracting the two expressions, we get that  1 X = − (zII − A)−1(AAXX − XXBB)(zII − B)−1dz. 2πi γ

Therefore, if X 1 and X 2 are solutions,

AAXX 1 − X 1B = C = AAXX 2 − X 2B, then  1 −1 −1 X 1 = − (zII − A) (AAXX 1 − X 1B)(zII − B) dz 2πi γ  1 −1 −1 = − (zII − A) (AAXX 2 − X 2B)(zII − B) dz 2πi γ = X 2. 

A.4 Kronecker Notation

Definition A.4.1. Let A = {aij} be an m × n matrix and B = {bij} an r × s matrix. Then the Kronecker product A ⊗ B is defined as the mr × ns matrix given by ⎛ ⎞ a11BBa12B ··· a1nB ⎜ ⎟ ⎜ a21BBa22B ··· a2nB ⎟ A ⊗ B = ⎜ ...... ⎟. ⎝ ...... ⎠ am1BBam2B ··· amnB

In the following, we state the most important properties of the Kronecker prod- uct. It is assumed that the dimensions of the matrices involved are such that the operations make sense (e.g., the resulting matrices in a sum have the same dimen- sion). It is easy to see that the Kronecker product satisfies the properties

(A + B) ⊗C = A ⊗C + B ⊗C, A ⊗ (B +C)=A ⊗ B + A ⊗C, (A ⊗ B) ⊗C = A ⊗ (B ⊗C).

The Kronecker product is in general not commutative. 718 A Appendix

Theorem A.4.2 (Mixed product rule).

(A ⊗ B)(C ⊗ D)=(AACC ⊗ BBDD)

Proof. Follows directly from ⎡ ⎤ c1 jD   ⎢ ⎥ ⎢ c2 jD ⎥ [a BBa B ··· a B]⎢ ⎥ = ∑a c BBDD i1 i2 in ⎣ . ⎦ ik kj . k cnjD =( ) . AACC ijBBDD 

It follows that if A and B are invertible matrices, then

− (A ⊗ B)=A 1 ⊗ B−1. (A.8)

Another immediate consequence is the following.

Theorem A.4.3. Let A and B be matrices with eigenvalues λi,i= 1,...,m, and μ j, j = 1,..,r, respectively and corresponding eigenvectors xi and yi. Then the eigen- values of A ⊗ BBareλiμ j,i= 1,...,m, j = 1,...,r, with corresponding eigenvectors xi ⊗y j.

Proof. Follows from

(A ⊗ B)(xi ⊗y j)=(AAxxi) ⊗ (BByy j)

=(λixi) ⊗ (μ jy j)

= λiμ j(xi ⊗y j). 

Since the determinant of a matrix equals the product of its eigenvalues, we conclude that

det(A ⊗ B)=(λ1μ1)(λ1μ2)···(λmμr−1)(λmμr) = λ mλ m ···λ mμrμr ···μr 1 2 m 1 2 r = det(A)rdet(B)m.

Concerning the trace, the following holds. Theorem A.4.4. tr(A ⊗ B)=tr(A)tr(B). A.4 Kronecker Notation 719

Proof.

tr(A ⊗ B)=tr(a11B)+tr(a22B)+···+ tr(ammB)

=(a11 + a22 + ···amm)tr(B) = tr(A)tr(B). 

Definition A.4.5. Let A be an m × m matrix and B an r × r matrix. Then the Kro- necker sum A ⊕ B of the two matrices is defined as

A ⊕ B = A ⊗ Ir + Im ⊗ B, where I n denotes the identity matrix of dimension n × n.

Theorem A.4.6. Let AAbeanm× m matrix with eigenvalues λi and corresponding eigenvectors xi and let BBbeanr× r matrix with eigenvalues μ j and corresponding eigenvectors y j.ThenAA ⊕ B has eigenvalues λi + μ j,i= 1,...,m, j = 1,...,r, with corresponding eigenvectors xi ⊗y j.

Proof.

(A ⊕ B)(xi ⊗y j)=(A ⊗ Ir + Im ⊗ B)(xi ⊗y j)

=(A ⊗ Ir)(xi ⊗y j)+(Im ⊗ B)(xi ⊗y j)

=(AAxxi ⊗y j)+(xi ⊗ BByy j)

= λi(xi ⊗y j)+μ j(xi ⊗y j)

=(λi + μ j)(xi ⊗y j). 

The following property plays a major role in the development of many formulas in this book. Theorem A.4.7. exp(A ⊕ B)=exp(A) ⊗ exp(B).

Proof. First of all, A ⊗ I r and Im ⊗ B commute, since by the mixed product rule, their product in either order equals A ⊗ B. Hence

exp(A ⊕ B)=exp(A ⊗ Ir)exp(I m ⊗ B).

The result will then follow from the mixed product rule if we can prove that exp(A ⊗ Ir)=exp(A) ⊗ I r and exp(I m ⊗ B)=I r ⊗ exp(B). The latter is immediate, since Im ⊗ B is a block diagonal matrix, whose exponential is simply the block diagonal matrix of the exponentials. To prove the former, we expand the exponential into the series by which it is defined and use the mixed product rule. Indeed, by induction n n we see that (A ⊗ Ir) = A ⊗ Ir, and hence 720 A Appendix ∞ 1 n exp(A ⊗ Ir)=∑ (A ⊗ Ir) n=0 n! ∞ 1 = ∑ (An ⊗ I ) n! r n=0  ∞ 1 n = ∑ A ⊗ Ir n=0 n!

= exp(A) ⊗ Ir. 

Now using the mixed product rule together with Theorem A.4.7,wethengetthe useful result  u − αeSSxsπeTTxttdx =(α ⊗β )(S ⊕ T ) 1(exp((S ⊕ T )u) − I)(s ⊗t), (A.9) 0 whenever S ⊕ T is invertible. References

1. Aalen, O.O.: Phase type distributions in survival analysis. Scandinavian Jour- nal of Statistics 22(4), 447–463 (1995) 2. Abdelkader, Y.H.: A Laplace transform method for order statistics from non- identical random variables and its application in Phase-type distribution. Statistics & Probability Letters 81(8), 1143–1149 (2011) 3. Agresti, A., Kateri, M.: Categorical data analysis. Springer (2011) 4. Ahn, S., Ramaswami, V.: Fluid flow models and queues – a connection by stochastic coupling. Stochastic Models 19(3), 325–348 (2003) 5. Ahn, S., Ramaswami, V.: Bilateral Phase Type Distributions. Stochastic Models 21, 239–259 (2005) 6. Albrecht, P.: Statistische Analyse des homogenen und des inhomogenen Pois- sonprozesses. Blatter¨ der DGVFM 14(4), 585–610 (1980) 7. Aldous, D., Shepp, L.: The least variable phase type distribution is Erlang. Commun. in Statist. - Stochastic Models 3(3), 467–473 (1987) 8. Andersen, A.: Statistik 1: Forelæsningsnoter. Aarhus Universitet, Matematisk Institut, Afdeling for Teoretisk Statistik (1984) 9. Andersen, A., Nielsen, B.F.: On the statistical implications of certain random permutations of inter-arrival times or counts in a point process. In: Sixth IFIP Workshop on Performance Modelling and Evaluation (ATM ’98). IFIP, IFIP, Ilkley (1998) 10. Andersen, A.T., Barker, V.A., Nielsen, B.F.: Similarity Transformations of MAPs. Mathematical Problems in Engineering 5, 349–359 (1999) 11. Andersen, A.T., Neuts, M.F., Nielsen, B.F.: PH-Distributions Arising through Conditioning. Communications in Statistics Stochastic Models 16(1), 179–188 (2000) 12. Andersen, A.T., Neuts, M.F., Nielsen, B.F.: On the time reversal of Marko- vian Arrival Processes. Stochastic Models 20(3), 237–260 (2004) 13. Andersen, A.T., Nielsen, B.F.: A Markovian Approach for Modeling Packet Traffic with Long Range Dependence. IEEE JSAC 16(5), 719–732 (1998)

© Springer Science+Business Media LLC 2017 721 M. Bladt, B.F. Nielsen, Matrix-Exponential Distributions in Applied Probability, Probability Theory and Stochastic Modelling 81, DOI 10.1007/978-1-4939-7049-0 722 References

14. Andersen, A.T., Nielsen, B.F.: On the statistical implications of certain ran- dom permutations in Markovian Arrival Processes (MAP)s and second order self-similar processes. Performance Evaluation 41, 67–82 (2000) 15. Asmussen, S.: Phase-type representations in and queueing problems. The Annals of Probability 20(2), 772–789 (1992) 16. Asmussen, S.: Stationary Distributions for Fluid Flow Models with or with- out Brownian Noise. Commun. Statist.-Stochastic Models 11(1), 21–49 (1995) 17. Asmussen, S.: A probabilistic look at the Wiener–Hopf equation. SIAM Rev. 40(2), 189–201 (1998) 18. Asmussen, S.: Applied Probability and Queues. Stochastic Modelling and Applied Probability. Springer (2003) 19. Asmussen, S.: Risk (incomplete draft) (2014). URL http://home. math.au.dk/asmus/ 20. Asmussen, S., Albrecher, H.: Ruin Probabilities, 2 edn. World scientific (2011) 21. Asmussen, S., Avram, F., Pistorius, M.R.: Russian and American put options under exponential phase–type Levy´ models. preprint pp. 1–37 (2003) 22. Asmussen, S., Bladt, M.: Poisson’s equation for queues driven by a Marko- vian marked point process. Queueing Systems 17(1), 235–274 (1994) 23. Asmussen, S., Bladt, M.: Renewal Theory and Queueing Algorithms for Matrix-Exponential Distributions. In: A. Alfa, S.R. Chakravarthy (eds.) MAM1, pp. 313–341 (1995) 24. Asmussen, S., Bladt, M.: Point processes with finite-dimensional probabili- ties. Stochastic Processes and their Applications 82(1), 127–142 (1999) 25. Asmussen, S., Glynn, P.: A new proof of convergence of MCMC via the ergodic theorem. Statistics and Probability letters 81, 1482–1485 (2011) 26. Asmussen, S., Hobolth, A.: Markov Bridges, Bisection and Variance Reduc- tion. Monte Carlo and Quasi-Monte Carlo Methods 2010 pp. 3–22 (2012) 27. Asmussen, S., Koole, G.: Marked point processes as limits of Markovian arrival streams. J. Appl. Prob. 30, 365–372 (1993) 28. Asmussen, S., Nerman, O., Olsson, M.: Fitting phase-type distributions via the em algorithm. Scandinavian Journal of Statistics 23, 419–441 (1996) 29. Asmussen, S., Petersen, S.S.: Ruin probabilities expressed in terms of storage processes. Advances in Applied Probability pp. 913–916 (1988) 30. Asmussen, S., Rolski, T.: Computational methods in risk theory: A matrix- algorithmic approach. Insurance: Mathematics and Economics 10, 259–274 (1991) 31. Asmussen, S., Teugels, J.L.: Convergence Rates for M/G/1 Queues and Ruin Problems with Heavy Tails. J. Appl. Prob. 33, 1181–1190 (1996) 32. Assaf, D., Langberg, N.A., Savits, T.H., Shaked, M.: Multivariate phase-type distributions. Operations Research 32(3), 688–702 (1984) 33. Astr˚ om,¨ K.J., Murray, R.M.: Feedback systems : an introduction for scientists and engineers. Princeton university press, Princeton, Oxford (2008) References 723

34. Ausin, M., Wiper, M.P., Lillo, R.E.: Bayesian estimation for the m/g/1 queue using a phase-type approximation. Journal of Statistical Planning and Infer- ence 118, 83–101 (2004) 35. Balakrishnan, K.: Exponential Distribution: Theory, Methods and Applica- tions. Taylor & Francis (1996) 36. Balakrishnan, N.: Permanents, order statistics, outliers, and robustness. Revista Matematica´ Complutense 20(1), 7–107 (2007) 37. Bean, N., Pollett, P.K., Taylor, P.G.: Quasistationary distributions for level- dependent quasi-birth-death processes. Stoch. Mod. 16(5), 511–541 (2000). Appear 38. Bean, N.G., Nielsen, B.F.: Quasi-birth-and-death processes with rational ar- rival process components. Stochastic Models 26(3), 309–334 (2010) 39. Billingsley, P.: Statistical Inference for Markov Processes. Statistical Re- search Monographs. University of Chicago Press (1961) 40. Billingsley, P.: Statistical methods in Markov chains. The Annals of Mathe- matical Statistics pp. 12–40 (1961) 41. Bingham, N., Goldie, C., Teugels, J.: Regular Variation. No. 1 in Encyclope- dia of Mathematics and its Applications. Cambridge University Press (1989) 42. Bini, D.A., Latouche, G., Meini, B.: Numerical Methods for Structured Markov Chains (Numerical Mathematics and Scientific Computation). Ox- ford University Press, Inc., New York, NY, USA (2005) 43. Bladt, M.: The Variance Constant for the Actual Waiting Time of the ph/ph/1 Queue. The Annals of Applied Probability 6(3), 766–777 (1996) 44. Bladt, M., Campillo-Navarro, A., Nielsen, B.F.: On the use of functional cal- culus for phase-type and related distributions. Stochastic Models 32(1), 1–19 (2015) 45. Bladt, M., Esparza, L.J.R., Nielsen, B.F.: Bilateral matrix-exponential dis- tributions. In: G. Latouche, V. Ramaswami, J. Sethuraman, K. Sigman, M. Squillante, D. Yao (eds.) Matrix-Analytic Methods in Stochastic Mod- els, vol. 27, pp. 41–56. Springer Proceedings in Mathematics and Statistics (2012) 46. Bladt, M., Gonzalez, A., Lauritzen, S.L.: The estimation of phase-type related functionals using Markov chain Monte Carlo methods. Scand. Actuarial J. 4, 280–300 (2003) 47. Bladt, M., Meini, B., Neuts, M., Sericola, B.: Distributions of reward func- tions on continuous-time Markov chains. In: 4th International Conference on Matrix Analytic Methods in Stochastic Models (MAM4) (2002) 48. Bladt, M., Neuts, M.F.: Matrix-exponential distributions: Calculus and inter- pretations via flows. Stochastic Models 19(1), 113–124 (2003) 49. Bladt, M., Nielsen, B.F.: Multivariate matrix-exponential distributions. Stochastic Models 26(1), 1–26 (2010) 50. Bladt, M., Nielsen, B.F.: On the construction of bivariate exponential distri- butions with an arbitrary correlation coefficient. Stochastic Models 26(2), 295–308 (2010) 724 References

51. Bladt, M., Nielsen, B.F.: Moment distributions of phase-type. Stochastic Models 27(4), 651–663 (2011) 52. Bladt, M., Nielsen, B.F., Samorodnitsky, G.: Calculation of ruin probabilities for a dense class of heavy tailed distributions. Scandinavian Actuarial Journal 2015(7), 573–591 (2015) 53. Bladt, M., Sørensen, M.: Statistical inference for discretely observed Markov jump processes. Journal of the Royal statistical society, series B 67(3), 395–410 (2005) 54. Bladt, M., Sorensen, M.: Efficient estimation of transition rates between credit ratings from observations at discrete time points. Quantitative finance 9(2), 147–160 (2009) 55. Bobbio, A., Telek, M.: Parameter estimation of phase type distributions. Technical Report R.T. 423, Istituto Elettrotecnico Nazionale Galileo Ferraris (1992). 20-th European Meeting of Statisticians, Bath, UK; September 1418, 1992 56. Bobbio, A., Telek, M.: A benchmark for PH estimation algorithms: results for Acyclic-PH. Communications in Statistics Stochastic Models 10(3), 661–677 (1994) 57. Bobbio, A., th, A.H., Scarpa, M., Telek, M.: Acyclic discrete phase type dis- tributions: properties and a parameter estimation algorithm. Performance Evaluation 54, 1–32 (2003) 58. Bodrog, L., Horvath,´ A., Telek, M.: Moment characterization of matrix ex- ponential and Markovian arrival processes. Annals of Operations Research 160, 69–82 (2008) 59. Bremaud, P.: Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues. Texts in Applied Mathematics. Springer (1998) 60. Breuer, L.: Parameter Estimation for a Class of BMAPs. In: G. Latouche, P.G. Taylor (eds.) MAM3, pp. 87–97. Notable Publications, Leuven (2000) 61. Breuer, L.: A quintuple law for Markov additive processes with phase-type jumps. J. Appl. Probab. 47(2), 441–458 (2010) 62. Breuer, L., Badescu, A.: A generalised Gerber-Shiu measure for Markov- additive risk processes with phase-type claims and capital injections. Scan- dinavian Actuarial Journal 2014(2), 93–115 (2014). URL http://kar. kent.ac.uk/31240/ 63. Breuer, L., Baum, D.: An introduction to queueing theory: and matrix- analytic methods. Springer (2005) 64. Bright, L., Taylor, P.G.: Calculating the equilibrium distribution in level de- pendent quasi-birth-and-death processes. Stochastic Models 11(3), 497–525 (1995) 65. Brockmeyer, E., Halstrøm, H., Jensen, A.: The life and works of AK Erlang. The Academy of Danish Sciences (1948) 66. Buchholz, P., Telek, M.: On minimal representations of rational arrival pro- cesses. Ann Oper Res 202, 35–58 (2013) 67. Cinlar, E.: Markov additive processes. I. Zeitschrift f¨ur Wahrscheinlichkeits- theorie und Verwandte Gebiete 24(2), 85–93 (1972) References 725

68. Cinlar, E.: Markov additive processes. II. Zeitschrift f¨ur Wahrscheinlichkeit- stheorie und Verwandte Gebiete 24(2), 95–121 (1972) 69. Cinlar, E.: Introduction to Stochastic Processes. Prentice-Hall, Inc. (1975) 70. Cohen, J.W.: The single server queue. North-Holland (1969) 71. Cox, D.R.: A use of complex probabilities in the theory of stochastic pro- cesses. Proc. Camb. Phil. Soc. 51, 313–319 (1955) 72. Daley, D.J., Vere-Jones, D.: An Introduction to the Theory of Point Processes. Volume I: Elementary Theory and Methods. Springer Science & Business Media (2003) 73. Davies, E.: One-parameter semigroups. L.M.S. monographs. Academic Press (1980) 74. Davis, M.H.: Markov Models & Optimization, vol. 49. CRC Press (1993) 75. Dellacherie, C., Meyer, P.A.: Probabilities and Potential, C: Potential Theory for Discrete and Continuous Semigroups. Elsevier Science Publishers (1988) 76. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incom- plete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological) pp. 1–38 (1977) 77. Doob, J.L.: Stochastic processes. Wiley-Interscience (1990) 78. Doolittle, E.: Analytic functions of matrices (1998). "http:// citeseerx.ist.psu.edu\discretionary-/viewdoc/ summary?doi=10.1.1.51.2968" 79. Dynkin, E.: Markov Processes. Springer-Verlag (1965) 80. Ellis, R.L.: On a question in the theory of probabilities. Cambridge Math. J. 4(21), 127–133 (1844). Reprinted in [191], pages 173–179. 81. Ethier, S.N., Kurtz, T.G.: Markov Processes. Characterization and Conver- gence. Wiley (2009) 82. Fackrell, M.: Fitting with Matrix-exponential Distributions. Stochastic Mod- els 21, 377–400 (2004) 83. Fackrell, M., He, Q.M., Taylor, P., Zhang, H.: The algebraic degree of phase- type distributions. J. Appl. Prob 47, 611–629 (2010) 84. Fackrell, M.W.: Characterization of Matrix-exponential Distributions . Ph.D. thesis pp. 1–211 (2005) 85. Faddy, M.J.: Examples of fitting structured phase-type distributions. Applied stochastic models and data analysis 10, 247–255 (1994) 86. Faddy, M.J., McClean, S.I.: Analysing data on lengths of stay of hospital pa- tients using phase-type distributions. Applied Stochastic Models in Business and Industry 15(4), 311–317 (1999) 87. Farina, L., Rinaldi, S.: Positive Linear Systems: Theory and Applications. A Wiley-Interscience publication. Wiley (2000) 88. Feldmann, A., Whitt, W.: Fitting mixtures of exponentials to long-tail distri- butions to analyze network performance models. Performance Evaluation 31, 245–279 (1998) 89. Feller, W.: An Introduction to Probability Theory and its Applications, vol. II. John Wiley & Sons (1971) 90. Fischer, W., Meier-Hellstern, K.: The Markov-modulated Poisson process (MMPP) cookbook. Performance Evaluation 18, 149–171 (1992) 726 References

91. Gail, H.R., Hantler, S.L., Taylor, B.A.: Spectral analysis of M/G/1and G/M/1 type Markov chains. Adv. Appl. Prob. 28, 114–165 (1996) 92. Glynn, P.: Poisson’s equation for the recurrent M/G/1 queue. Advances in Applied Probability. pp. 1044–1062 (1994) 93. Graham, A.: Kronecker Products and Matrix Calculus: With Applications. Wiley (1982) 94. Greenwood, P.: Wiener-Hopf decomposition of random walks and Levy pro- cesses. Zeitschrift f¨ur Wahrscheinlichkeitstheorie und Verwandte Gebiete 34(3), 193–198 (1976) 95. Gut, A.: Stopped Random Walks: Limit Theorems and Applications. Opera- tions Research and Financial Engineering. Springer (2009) 96. Haase, M.: The Functional Calculus for Sectorial Operators. Springer (2006) 97. Hall, P., Heyde, C.: 3 - the central limit theorem. In: P. Hall, C. Heyde (eds.) Martingale Limit Theory and its Application, Probability and Mathemati- cal Statistics: A Series of Monographs and Textbooks, pp. 51–96. Academic Press (1980) 98. Harris, C.M., Sykes, E.A.: Likelihood estimation for generalized mixed expo- nential distributions. Naval Research Logistics Quarterly 34, 251–279 (1987) 99. He, Q.M.: Fundamentals of Matrix-Analytic Methods. Springer (2013) 100. He, Q.M., Neuts, M.: Markov chains with marked transitions. Stochastic Processes and their Applications 74, 37–52 (1998) 101. He, Q.M., Zhang, H.: On matrix exponential distributions. Adv. in Appl. Probab. 39(1), 271–292 (2007). DOI 10.1239/aap/1175266478 102. He, Q.M., Zhang, H., Vera, J.C.: On some properties of bivariate exponential distributions. Stochastic Models 28(2), 187–206 (2012) 103. Higham, N.J.: Functions of matrices. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA (2008) 104. Hoel, P.G., Port, S.C., Stone, C.J.: Introduction to stochastic processes. Houghton Mifflin Boston (1972) 105. Hoffmann-Jørgensen, J.: Probability in Banach space. Springer (1977) 106. Horvath,´ I., Telek, M.: A constructive proof of the phase-type characteriza- tion theorem. Stochastic Models 31(2), 316–350 (2015) 107. Jacobsen, M.: The time to ruin in some additive risk models with random premium rates. Journal of Applied Probability 49(4), 915–938 (2012) 108. Jensen, A.: Distribution patterns composed of a limited number of ex- ponential distributions. In: Den 11. skandinaviske matematikerkongres, pp. 209–215. Trondheim (1949) 109. Jensen, A.: Markoff Chains as an Aid in the Study of Markoff Processes. Skandinavisk Aktuarietidskrift 36, 87–91 (1953) 110. Johansen, S.: Some results on the imbedding problem for finite Markov chains. Journal of the London Mathematical Society 2(2), 345 (1974) 111. Johnson, M.A., Taaffe, M.R.: The denseness of phase distributions. Research Memorandum 88-20, School of Industrial Engineering, Purdue University West Lafayette, IN 47907 (1988) References 727

112. Johnson, M.A., Taaffe, M.R.: Matching moments to phase type distributions: Nonlinear programming approaches. Commun. Statist.-Stochastic Models 6(2), 259–281 (1990) 113. Kalashnikov, V.: Regeneration and general Markov chains. Journal of Ap- plied Mathematics and Stochastic Analysis 7(3), 357–371 (1994) 114. Kallenberg, O.: Foundations of modern probability. Springer Verlag (2002) 115. Kallianpur, G., Sundar, P.: Stochastic Analysis and Diffusion Processes. Ox- ford Graduate Texts in Mathematics. Oxford University Press (2014) 116. Kao, E.P.C., Smith, M.S.: On excess-, current-, and total-life distributions of phase-type renewal processes. Naval Research Logistics 39(6), 789–799 (1992) 117. Karatzas, I., Shreve, S.: Brownian Motion and . Graduate Texts in Mathematics. Springer (1991) 118. Karlin, S., Taylor, H.M.: A first course in stochastic processes, second edn. Academic Press [A subsidiary of Harcourt Brace Jovanovich, Publishers], New York-London (1975) 119. Kim, K., Thomas, N.: A fitting method with generalized Erlang distributions. Simulation Modelling Practice and Theory 19(7), 1507–1517 (2011) 120. Kingman, J.: Poisson Processes. Oxford Studies in Probability. Clarendon Press (1992) 121. Kingman, J.F.C.: The imbedding problem for finite Markov chains. Proba- bility Theory and Related Fields 1, 14–24 (1962) 122. Klemm, A., Lindemann, C., Lohmann, M.: Modeling ip traffic using the batch Markovian arrival process. Performance Evaluation 54, 149–173 (2003) 123. Kotz, S., Balakrishnan, N., Johnson, N.L.: Continuous Multivariate Distribu- tions. John Wiley and Sons (2000) 124. Kulkarni, V.G.: A new class of multivariate phase type distributions. Opera- tions Research 37(1), 151–158 (1989) 125. Kyprianou, A.: Gerber–Shiu Risk Theory. EAA Series. Springer International Publishing (2013) 126. Latouche, G., Ramaswami, V.: Introduction to Matrix Analytic Methods in Stochastic Modeling. SIAM Publications (1999) 127. Latouche, G., Ramaswami, V.: A logarithmic reduction algorithm for quasi- birth-death processes. J. Appl. Prob. 30, 650–674 (93) 128. Latouche, G., Remiche, M.A., Taylor, P.: Transient Markov arrival processes. The Annals of Applied Probability 13(2), 628–640 (2003) 129. Li, S.Y.R.: A martingale approach to the study of occurrence of sequence patterns in repeated experiments. Annals of Probability 8(6), 1171–1176 (1980) 130. de Liefvoort, A.V.: The moment problem for continuous distributions. Com- puter science telecommunications program, University of Missouri, Kansas City, 5100 Rockhill Road, Kansas City, Missouri 64110 (2006) 131. Lindvall, T.: Lectures on the coupling method. Dover Publications (2002) 132. Lipsky, L.: Queueing Theory: A Linear Algebraic Approach, 2nd edn. Springer (2008) 728 References

133. Loeve, M.: Probability Theory I. Springer Verlag (1978) 134. Loeve, M.: Probability Theory II. Springer (1978) 135. Lucantoni, D.M.: New results on the single server queue with a batch Marko- vian arrival process. Commun. Statist.-Stochastic Models 7(1), 1–46 (1991) 136. Lucantoni, D.M.: The BMAP/G/1 queue: A Tutorial. In: L. Dantiello, R. Nel- son (eds.) Performance Evaluation of Computer and Communication Sys- tems: Joint Tutorial Papers of Performance ’93 and Sigmetrics ’93, pp. 330– 358. Springer-Verlag (1993) 137. Lucantoni, D.M., Meier-Hellstern, K.S., Neuts, M.F.: A single-server queue with server vacations and a class of non-renewal arrival processes. Adv. Appl. Prob. 22, 676–705 (1990) 138. Maier, R.S.: The algebraic construction of phase-type distributions. Com- mun. Statist.-Stochastic Models 7(4), 573–602 (1991) 139. Maier, R.S., O’Cinneide, C.A.: A Closure Characterization of Phase-Type Distributions. Journal of Applied Probability 29(1), 92–103 (1992) 140. Marshall, A.H., McClean, S.I.: Conditional phase-type distributions for mod- elling patient length of stay in hospital. International Transactions in Opera- tional Research 10(6), 565–576 (2003) 141. Marshall, A.H., McClean, S.I.: Using Coxian phase-type distributions to identify patient characteristics for duration of stay in hospital. Health Care Management Science 7(4), 285–289 (2004) 142. Meier-Hellstern, K.S.: A fitting algorithm for Markov-modulated Poisson processes having two arrival rates. European Journal of Operational Research 29, 370–377 (1987) 143. Mitchell, K.: Constructing a correlated sequence of matrix exponentials with invariant first-order properties. Oper. Res. Let. 28(1), 27–34 (2001) 144. Mocanu, S., Commault, C.: Sparse representations of phase-type distribu- tions. Commun.-Statist.-Stochastic Models 15(4), 759–768 (1999) 145. Mordecki, E.: The distribution of the maximum of a Levy´ process with posi- tive jumps of phase-type. Th. Stoch. Proc. (8), 309–316 (2002) 146. Mordecki, E.: Ruin probabilities for Levy´ processes with mixed-exponential negative jumps. Rossi˘ıskaya Akademiya Nauk. Teoriya Veroyatnoste˘ıiee Primeneniya 48(1), 188–194 (2003) 147. M¨uller, A., Stoyan, D.: Comparison methods for stochastic models and risks. Wiley series in probability and statistics. Wiley, Chichester 148. Narayana, S., Neuts, M.F.: The first two moment matrices of the counts for the Markovian arrival process. Commun. Statist.-Stochastic Models 8(3), 459–477 (1992) 149. Nelson, R.: Probability, Stochastic Processes, and Queueing Theory: The Mathematics of Computer Performance Modeling. Springer, New York, NY, USA (1995) 150. Neuts, M.F.: Probability distributions of phase type. In: Liber Amicorum Pro- fessor Emeritus H. Florin, pp. 173–206. Department of Mathematics, Univer- sity of Louvian, Belgium (1975) References 729

151. Neuts, M.F.: Renewal processes of phase type. Naval Research Logistics Quarterly 25, 445–454 (1978) 152. Neuts, M.F.: A versatile Markovian point process. J. Appl. Prob. 16, 764–779 (1979) 153. Neuts, M.F.: Matrix-Geometric Solutions in Stochastic Models an Algorith- mic Approach, John Hopkins Series in Mathematical Sciences,vol.2.The John Hopkins University Press (1981) 154. Neuts, M.F.: Structured Stochastic Matrices of M/G/1 Type and Their Ap- plications, A Series of Textbook and Reference Books, vol. 5. Marcel Dekker, Inc. (1989) 155. Neuts, M.F.: Algorithmic Probability, 1 edn. Stochastic Modeling Series. Chapman & Hall (1995) 156. Neveu, J.: Discrete-Parameter Martingales. North Holland (1975) 157. Nielsen, B.F., Thygesen, U.H., Nilsson, L.A.F., Beyer, J.E.: Higher order mo- ments and conditional asymptotics of the batch Markovian arrival process. Stochastic Models 23(1), 1–26 (2007) 158. Nummelin, E.: A splitting technique for Harris recurrent Markov chains. Zeitschrift f¨ur Wahrscheinlichkeitstheorie und Verwandte Gebiete 43(4), 309–318 (1978) 159. Nummelin, E.: General Irreducible Markov Chains and Non-negative Opera- tors. Cambridge University Press (1984) 160. Oakes, D.: Direct calculation of the information matrix via the EM algorithm. J.R. Statist. Soc. B 61(Part 2), 479–482 (1999) 161. O’Cinneide, C.A.: On non-uniqueness of representations of phase-type dis- tributions. Commun. Statist.-Stochastic Models 5(2), 247–259 (1989) 162. O’Cinneide, C.A.: Characterization of phase-type distributions. Commun. Statist.-Stochastic Models 6(1), 1–57 (1990) 163. O’Cinneide, C.A.: Phase-type distributions and majorization. The Annals of Applied Probability 1(2), 219–227 (1991) 164. Olsson, M.: Estimation of phase-type distributions from censored data. Scan- dinavian Journal of Statistics pp. 443–460 (1996) 165. Pitman, J.W.: Occupation measures for Markov chains. Advances in Applied Probability 9(1), 69–86 (1977) 166. Pozdnyakov, V., Kulldorff, M.: Waiting times for patterns and a method of gambling teams. The American Mathematical Monthly pp. 134–143 (2006) 167. Raheem, M.A., Yahya, W.B., Obisesan, K.O.: A Markov approach on pattern of rainfall distribution. Journal of Environmental Statistics 7(1), 1–13 (2015). URL http://jes.stat.ucla.edu/v07/i01 168. Ramaswami, V.: A duality theorem for the matrix paradigms in queueing theory. Commun. Statist. - Stochastic Models 6(1), 151–161 (1990) 169. Ramaswami, V.: Matrix analytic methods for stochastic fluid flows. Teletraf- fic science and engineering pp. 1019–1030 (1999) 170. Ramsay, C.M.: A solution to the ruin problem for Pareto distributions. Insur- ance: Mathematics and Economics 33, 109–116 (2003) 730 References

171. Ramsay, C.M.: Exact waiting time and queue size distributions for equilib- rium M/G/1 queues with Pareto service. Queueing Systems 57(4), 147–155 (2007) 172. Revuz, D.: Markov Chains. North-Holland (1975) 173. Revuz, D., Yor, M.: Continuous Martingales and Brownian Motion. Springer (1999) 174. Rockafellar: Convex Analysis pp. 1–468 (2011) 175. Rudemo, M.: Point processes generated by transitions of Markov chains. Ad- vances in Applied Probability pp. 262–286 (1973) 176. Ryden, T.: Parameter estimation for Markov modulated Poisson processes. Commun. Statist.-Stochastic Models 10(4), 795–829 (1994) 177. Schassberger, R.: Warteschlangen. Springer-Verlag (1973) 178. Schilling, R., Partzsch, L.: Brownian Motion: An Introduction to Stochastic Processes. De Gruyter graduate. De Gruyter (2012) 179. Schmickler, L.: Meda: Mixed Erlang distributions as phase-type representa- tions of empirical distribution functions. Commun. Statist.-Stochastic Mod- els 8(1), 131–156 (1992) 180. Seber, G.A.: Multivariate observations, vol. 252. Wiley. com (2009) 181. Shaked, M., Shanthikumar, J.G.: Stochastic Orders. Springer (2007) 182. Shantikumar, J.G.: Bilateral phase-type distributions. Naval Research Logis- tics Quarterly 32, 119–136 (1985) 183. Sigman, K., Wolff, R.W.: A review of regenerative processes. SIAM Review 35(2), 269–288 (1993) 184. Smith, W.: Regenerative stochastic processes. Proceedings of the Royal So- ciety of London. Series A. Mathematical and Physical Sciences 232(1188), 6–31 (1955) 185. Strassen, V.: The existence of probability measures with given marginals. The Annals of pp. 423–439 (1965) 186. Telek, M.: The minimal coefficient of variation of discrete phase type distri- butions. In: Third international conference on Matrix Analytic Methods in stochastic Models, MAM3, pp. 391–400. Notable Publications Inc. (2000) 187. Telek, M., Horvath,´ G.: A minimal representation of Markov arrival processes and a moments matching method. Performance Evaluation 64, 1153–1168 (2007) 188. Thorisson, H.: Coupling, stationarity, and regeneration. Probability and its applications. Springer, New York (2000) 189. Th¨ummler, A., Buchholz, P., Telek, M.: A novel approach for phase-type fitting with the em algorithm. IEEE Transactions on Dependable and Secure Computing 3(3), 245–258 (2006) 190. Van Loan, C.: Computing integrals involving the matrix exponential. IEEE Transactions on Automatic Control 23(3), 395–404 (1978) 191. Walton, W.E.: The mathematical and other writings of Robert Leslie Ellis. Deighton Bell, Cambridge (1863) 192. Wolff, R.: Stochastic Modeling and the Theory of Queues. Prentice-Hall in- ternational series in industrial and systems engineering. Prentice Hall (1989) Index

α +, 369, 371, 373 functional calculus, 169–180 actual waiting time, 381 function of matrix, 170 affine, 266 product rule, 171 Anscombe’s theorem, 321 g+(x)=β B¯(x), 379 Bayesian methods, see Markov chain Monte geometric distribution, 29 Carlo Gini index, 249 Breiman’s theorem, 188 Green matrix Brownian motion, 110–122 discrete phase-type, 31 construction, 110 law of the iterated logarithm, 121 Harris chain reflection principle, 117 definition, 398 strong , 116 ergodic, 402 stationary distribution, 402 companion matrix, see matrix-exponential Nummelin splitting, 398–399 distribution Poisson’s equation, see Poisson’s equation convex risk ordering, 163 positive recurrent, 402 coupling, 20 recurrence set, 398 coupling inequality, 20 regeneration set, 398 general coupling inequality, 396 stationarity, 400, 401 Cramer–Lundberg´ model, 380 time average asymptotics, 407 time average properties, 403 direct Riemann integrability, 314 heavy tail necessary conditions, 316 ruin probability, see ruin probability sufficient conditions, 317 hyperexponential distribution definition, 130 Erlang distribution definition, 128 Laplace transform, 205 infinitesimal generator, see intensity matrix Erlangization, 620, 623 intensity matrix, 40 Richardson extrapolation, 620, 623 invariant polytope lemma, 270

filtration, 77 Karamata’s theorem, 185 fluid flow process, see Markov Krylov space, 214

© Springer Science+Business Media LLC 2017 731 M. Bladt, B.F. Nielsen, Matrix-Exponential Distributions in Applied Probability, Probability Theory and Stochastic Modelling 81, DOI 10.1007/978-1-4939-7049-0 732 Index ladder heights, 361 Brownian component, 505–510 distributions of, 362 busy period, 500, 503 Laplace transform, 204 duality, 499, 500 lexicographical ordering, 27 idle period, 500 likelihood ladder process, 492, 497 EM algorithm, 659 limiting distribution, 502 calculation of information matrix, 662 maximum, 499 Fisher information matrix, 630 mixed fluid and Brownian, 510–513 general theory, 627–631 negative drift condition, 491 likelihood function, 627 reflected fluid flow, 498 Markov chain, 641 time reversal, 499 Billingsley–Meyer’s theorem, 644 Markov random walk, 482, 488–490 equivalence to multinomial, 644 Chapman–Kolmogorov, 488 nested testing, 650 classification, 490 Whittle’s theorem, 644 regenerative structure, 488 Markov Markov renewal process, 482–488 discretely observed, see Markov jump Markov renewal function, 486 process semi-Markov kernel, 483 likelihood function, 652 semi-Markov process, 483 maximum likelihood estimator, 653 strong Markov property, 481 testing for equal intensities, 654 Markov chain, 6–36 maximum likelihood estimator, 628 absorbing state, 13 multinomial, 632 aperiodic, 19 likelihood ratio, 633 Chapman–Kolmogorov, 9 maximum likelihood estimator, 632 communication of states, 12 nested testing, 638 coupling, 20 observed information matrix, 634 definition, 6 −2logQ, 634, 638 ergodic, 20 several multinomial observations, 639–641 geometric convergence, 24 observed information, 628 irreducible, 13 phase-type estimation, see phase-type multidimensional, 27 estimation null recurrence, 17 Poisson process periodicity, 19 confidence interval, 657 positive recurrence, 17 likelihood function, 655 recurrence, 11 score function, 628 reversible, 27 Lindley processes, 380 stationarity, 13 definition, 380 stationary measure, 13 Lorenz curve, 248 , 10 strong Markov property, 10 MAP/G/1 queue time reversal, 24 ladder process, 535 time-homogeneous, 6 matrix occupation measure, 538 transience, 11 occupation measure, 537 transition matrix, 6 Pollaczek–Khinchin, 535 Markov chain Monte Carlo, 404 stability condition, 530 conjugate prior, 664 stationary distribution, 533 credibility interval, 663 time reversion, 531 independence sampler, 406 time-average properties, 542–557 Markov jump process virtual waiting time, 529 discretely observed, see Markov jump MAP, see Markovian arrival process process Markov additive process Metropolis–Hastings, 405 definition, 481 phase-type estimation, see phase-type fluid flow, 482, 490–513 estimation Index 733

posterior, 663 Doob’s inequalities, 99 prior, 662 optional stopping, 97, 99 Markov jump process, 36–68 regularization, 97 Chapman–Kolmogorov, 36 discrete time, 77 coupling, 50 central limit theorem, 101 definition, 36 Doob decomposition, 83 discretely observed, 695–699 Doob’s inequalities, 93 EM algorithm, 696 martingale convergence theorem, 87 embedding, 696 optional stopping, 78, 79, 84 identifiability, 696 stopped martingale, 78 information matrix, 697 submartingale, 77 Markov chain Monte Carlo, 698 supermartingale, 77 maximum likelihood estimation, 695 uniformly integrable, 88 distribution of jumps, 57–66 upcrossing inequality, 86 distribution of occupation times, 57–66 matrix-exponential distribution embedding, 66–68 atom at zero, 199 ergodic, 49 characterization with residual life operator, explosion, 46 257 intensity matrix, 40 companion matrix, 206 irreducible, 43 eigenvalues, 209 Kolmogorov differential equation, 41 compound sum, 229 matrix exponential, 43 convolution, 228 multidimensional, 55 definition, 199 Reuter’s criterion, 47 degree, 211 stationarity, 43 density stationary measure, 43 expanded form, 201 stopping time, 37 general form, 199 strong Markov property, 37 ME that is not PH, 205 time reversal, 54 dimension, 211 transition matrix, 43 distribution function, 210 uniformization, 51–53 flow arguments, 346–348 Markov processes, 107–110 flow interpretation, 263–265 Chapman–Kolmogorov, 108 Hankel matrix, 221–223 Markov kernel, 109 degree, 222 probability kernel, 109 higher order representation, 227 transition density, 108 invariant polytope lemma, 270 transition probability, 107 Krylov space, 214 Markov random walk, see Markov additive Lp, L∞, Le, 214 process Laplace transform, 205 Markov renewal process, see Markov additive of Erlang distribution, 205 process poles, 209 Markovian arrival process, 523–528 (fractional) moments, 210 construction, 524 minimal representation definition, 525 characterization, 216 distribution of arrival times, 526 definition, 211 distribution of interarrival times, 526 order statistics, 231–242 event-stationarity, 527 Gini index, 249 MAP/G/1 queue, see MAP/G/1 queue Lorenz curve, 248 representation, 525 representation, 243 time-stationarity, 527 property (L), 215 martingale property (R), 215 continuous time Rp, R∞, Re, 214 cadl` ag,` 97 random walk, 368 definition, 95 reduced moments, 221–223 734 Index

renewal theory, see renewal theory dimension, 138 residual life operator, 250 distribution function, 132 characterization of ME, 257 eigenvalues, 134 matrix-exponential form, 255 (generalized) Erlang, 126 spectral properties, 336 general expected value, 174 matrix-geometric distribution Green matrix, 134 convolution, 228 hyperexponential, 128 definition, 224 irreducible representation, 132, 328 probability generating function, 225 Laplace transform, 136 MCMC, see Markov chain Monte Carlo Mellin transform, 175 moment distribution minimal characterization, 153 definition, 242 minimal variability of Erlang, 169 monotone class argument, 8 mixture, 140 Multivariate Matrix-Exponential moment distribution, 332, 333, 335 MPH Class, 446 moments, 135 multivariate matrix-exponential order, 138 MME∗ class, 439 order statistics, 143 by joint order statistics, 459–467 random convolution, 141 by moment distributions, 467–468 relation to discrete phase-type, 137 degree, 443 representation, 126 MPH∗ class, 437 rewards transformation, 147 Cheriyan and Ramabhadran’s multivariate simulation, 664–668 gamma, 451 sine transform, 174 marginal distributions, 441 time-reversed representation, 329 Marshall Olkin’s bivariate exponential, discrete, 28–36 452 convolution, 32 McKays bivariate gamma, 450 definition, 29 moment-generating function, 438 density, 30 moments, 440 distribution function, 30 multivariate exponential and gamma, eigenvalues, 32 448–459 factorial moments, 35 Prekopa´ and Szantai’s´ multivariate generating function, 34 gamma, 452 geometric, 29 Raftery’s bivariate Exponential, 454 Green matrix, 31 survival function, 442 mixtures, 33 MV ME class representation, 29 characterization, 473 estimation, see phase-type estimation definition, 471 infinite-dimensional, 181–193 randomly observed Brownian motion, calibration to Pareto, 192 475–478 NPH class, 181 lower bound on its order, 220 O(h),o(h) functions, 2 multivariate, see mltivariate matrix- order statistics exponential437 density, 232 random walk, 368 joint density, 233 renewal theory, see renewal theory phase-type estimation permanent, 231 completely observed phase-type distribution likelihood, 672 continuous maximum likelihood estimator, 674 convolution, 139 EM algorithm cosine transform, 174 censoring, 685–690 definition, 126 continuous phase-type, 678–681 denseness, 153 discrete observed phase-type, 691 density, 131 discrete phase-type, 675–678 Index 735

discretely observed phase-type, 695 definition, 559 fitting a theoretical distribution, 681–685 event-stationarity, 577 Kullback–Leibler, 684 interarrivals Markov chain Monte Carlo, 690–691 joint Laplace transform, 573 point process joint moment-generating function, 573 definition, 517 moments and cross moments, 574 event-stationary, 518 Palm distribution, 576 Palm distribution, 519 superposition, 575 Palm’s inversion formula, 521 time-stationarity, 576 time-stationary, 518 rational function, 204 Poisson process, 1–6 recurrence set, 398 characterisation, 3 regeneration definition, 2 central limit theorem, 394 independent increments, 2 classical, 387 stationary increments, 2 limiting distribution, 388 Poisson’s equation time average limits, 391 actual waiting time, 415–425 wide-sense, 388 ME/ME/1 example), 425 regular variation, 183 solution kernel), 420 renewal process, 356 discrete time, 408 renewal reward process, 320 central limit theorem, 415 renewal reward theorem, 320 solution kernel, 408 renewal theory, 297 uniqueness, 411 age process, 318 Potter bounds, 188 Anscombe’s theorem, 321 pre-τ+ occupation measure, 365 Blackwell’s theorem, 311 direct Riemann integrability, 314 queueing–risk duality, 383 discrete phase-type, 354–356 discrete time, 348–353 random walk key renewal theorem, 314 classification, 366 matrix-exponential renewal, 342 definition, 361 renewal density, 342 duality, 364 renewal measure, 342 ladder height density, 379 residual life time, 343 ladder heights, 361 stationarity, 344 matrix-exponential distribution, 368 terminating, 344 phase-type distribution, 368 matrix-geometric, 354–356 Pollaczek–Khinchin, 366 NPH interarrivals, 607 time reversal, 363 lifetime of terminal, 608 Wiener–Hopf factorization, 367 renewal density, 607 RAP, see rational arrival process residual lifetime, 608 rational arrival process phase-type renewal process, 325 characterization, 558 delayed renewal density, 326 coordinate process life time for terminating, 328 definition, 561 renewal density, 325 jump behavior, 567 residual life time, 326 martingale, 562 stationarity, 327 off jump behavior, 565, 566 renewal argument, 301 sample path behavior, 567 renewal equation, 301 sample paths, 563 renewal function, 298 strong Markov property, 567 renewal process, 297 counting process delayed, 297 joint generating function, 572 pure, 297 moment-generating function, 570 renewal reward process, 320 probability generating function, 572 renewal reward theorem, 320 736 Index

residual life time, 318 stopping time spread, 318 general definition, 73 stationarity, 307 Markov chain, 10 terminating, 308 Markov jump process, 37 residual life operator, 250 Strassen’s theorem, 162 Richardson extrapolation, 620, 623 strong Markov property ruin probability Markov chain, 10 Brownian off claim increments, 614–616 Markov jump processes, 37 Cramer–Lundberg,´ 614–615 subtransition matrix, 29 Sparre–Andersen, 615–616 Cramer–Lundberg´ model, 380 time averages finite time, 616–624 continuous time, 426 matrix-exponential claims, 377 central limit theorem, 434 NPH, 605–614 equivalent martingale, 426 g+(x), 610 solution kernel, 427, 431 Pareto example, 611 uniqueness, 431 ruin function, 611 time reversal, 24 phase-type claims, 377 Markov chain, 24 random walk, 383 Markov jump process, 54 reserve dependent premiums, 581–605 total variation MAP modulation, 601–605 Markov chain, 21 matrix-exponential claims, 595–601 transition matrix, 6 νi(u), 585, 589 subtransition matrix, 29 premium function, 582, 583 Runge–Kutta, 593 uniformization severity of ruin, 592 Markov jump process, 51–53 Richardson extrapolation, 620, 623 severity of ruin, 377 Wald’s identity Sparre–Andersen model, 376 first identity, 80 second identity, 80 Sparre–Andersen model, 376 Wiener–Hopf, see random walk standard argument, 8 stop-loss transform, 155 zero-modified distribution, 328