Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Birth-death processes

Jorge Julvez´ University of Zaragoza

1 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Outline

1 Birth Processes

2 Birth-Death Processes

3 Relationship to Markov Chains

4 Linear Birth-Death Processes

5 Examples

2 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Outline

1 Birth Processes

2 Birth-Death Processes

3 Relationship to Markov Chains

4 Linear Birth-Death Processes

5 Examples

3 / 47 What is the time evolution of the system?

Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Pure6.1 BirthPure ProcessBirth Process (Yule-Furry(Yule-Furry Process) Process) Example. Consider cells which reproduce according to the following Example:rules: Consider cells which reproduce according to the following rules: i. A cell present at time t has probability λh + o(h) of splitting in two • A cell present at time t has probability λh + o(h) of splitting in the interval (t, t + h) in two in the interval (t, t + h) •ii.ThisThis probabilityprobability is isindependent independentof age. of age •iii.EventsEvents betweeenbetween different differentcells cellsare independent are independent

> Time

4 / 47 231 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

> Time What is the time evolution of the system?

4 / 47 231 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Pure Birth Process (Yule-Furry Process)

Non-Probabilistic Analysis • Let n(t) =number of cells at time t • Let λ be the birth rate per single cell Thus ≈ λn(t)∆(t) births occur in (t, t + ∆t) Then: n(t + ∆t) = n(t) + n(t)λ∆t n(t + ∆t) − n(t) dn = n(t)λ → = n0(t) = n(t)λ ∆t dt

• The solution of this differential equation is: n(t) = Keλt

• If n(0) = n0 then λt n(t) = n0e

5 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Pure Birth Process (Yule-Furry Process)

Probabilistic Analysis Notation: • N(t) = number of cells at time t

• P{N(t) = n} = Pn(t)

Assumptions: • A cell present at time t has probability λh + o(h) of splitting in two in the interval (t, t + h) • The probability of more than one birth occurring in time interval (t, t + h) is o(h)

All states are transient

6 / 47 Pn(t + h) − Pn(t) = −nλhPn(t) + Pn−1(t)(n − 1)λh + f (h), with f (h) ∈ o(h) P (t + h) − P (t) f (h) n n = −nλP (t) + P (t)(n − 1)λ + h n n−1 h Let h → 0, 0 Pn(t) = −nλPn(t) + (n − 1)λPn−1(t)

Initial condition Pn0 (0) = P{N(0) = n0} = 1

Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Pure Birth Process (Yule-Furry Process)

Assumptions: • Probability of splitting in (t, t + h): λh + o(h) • Probability of more than one split in (t, t + h): o(h) The probability of birth in (t, t + h) if N(t) = n is nλh + o(h). Then,

Pn(t + h) = Pn(t)(1 − nλh − o(h)) + Pn−1(t)((n − 1)λh + o(h))

7 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Pure Birth Process (Yule-Furry Process)

Assumptions: • Probability of splitting in (t, t + h): λh + o(h) • Probability of more than one split in (t, t + h): o(h) The probability of birth in (t, t + h) if N(t) = n is nλh + o(h). Then,

Pn(t + h) = Pn(t)(1 − nλh − o(h)) + Pn−1(t)((n − 1)λh + o(h))

Pn(t + h) − Pn(t) = −nλhPn(t) + Pn−1(t)(n − 1)λh + f (h), with f (h) ∈ o(h) P (t + h) − P (t) f (h) n n = −nλP (t) + P (t)(n − 1)λ + h n n−1 h Let h → 0, 0 Pn(t) = −nλPn(t) + (n − 1)λPn−1(t)

Initial condition Pn0 (0) = P{N(0) = n0} = 1 7 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Pure Birth Process (Yule-Furry Process)

Probabilities are given by a set of ordinary differential equations.

0 Pn(t) = −nλPn(t) + (n − 1)λPn−1(t)

Pn0 (0) = P{N(0) = n0} = 1

Solution

n − 1 −λn0t −λt n−n0 Pn(t) = e (1 − e ) n = n0, n0 + 1,... n − n0 n n! where = . k k!(n − k)!

8 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Pure Birth Process (Yule-Furry Process)

Solution

n − 1 −λn0t −λt n−n0 Pn(t) = e (1 − e ) n = n0, n0 + 1,... n − n0

Observation: The solution can be seen as a negative binomial distribution, i.e., probability of obtaining n0 successes in n trials. Suppose p =prob. of success and q = 1 − p =prob. of failure. Then, the probability that the ﬁrst (n − 1) trials result in (n0 − 1) th successes and (n − n0) failures followed by success on the n trial is: n − 1 n − 1 n0−1 n−n0 n0 n−n0 p q p = p q ; n = n0, n0 + 1,... n − n0 n − n0 If p = e−λt and q = 1 − e−λt , both equations are the same. 9 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Pure Birth Process (Yule-Furry Process)

• Yule studied this process in connection with the theory of evolution, i.e., population consists of the species within a genus and creation of a new element is due to mutations. • This approach neglects the probability of species dying out and size of species. • Furry used the same model for radioactive transmutations.

10 / 47 Birth ProcessesJ. Virtamo Birth-Death Processes Relationship to Markov 38.3143 Chains Queueing Linear Theory Birth-Death / ProcessesBirth-death Examples processes 6

Example 2. Pure birth process (Poisson process) Pure Birth Processes. Generalization

 λi = λ  1 i =0  i =0, 1, 2,... πi(0) =  • In aµ Yule-Furryi =0 process, for N(t) = n the probability0 ofi> a 0 change during (t, t + h) depends on n.  birth probability per time unit is initially the population size is 0 • In a Poisson process, the probability of a change during (tconstant, t + h) isλ independent of N(t).

l l l l l l 0 1 2 ... i-1 i All states are transient

Generalizationd  dt πi(t)=−λπi(t)+λπi−1(t) i>0 • Assume that for N(t) = n the probability of a new change  to nd + 1 in (t, t−+ h) is λ h + o(h). ⇒ −λt  dt π0(t)= λπ0(t) n π0(t)=e  • The probability of more than one change is o(h).

d λt λt −λt t # λt# # (e π (t)) = λπ (t)e ⇒ π (t)=e λ π (11t /) 47e dt dt i i−1 i %0 i−1 t # # π (t)=e−λtλ e−λt eλt dt# = e−λt(λt) 1 %0 1 & '( ) (λt)i Recursively π (t)= e−λt Number of births in interval (0,t) ∼ Poisson(λt) i i! Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Pure Birth Processes. Generalization

Generalization • Assume that for N(t) = n the probability of a new change to n + 1 in (t, t + h) is λnh + o(h). • The probability of more than one change is o(h).

Then,

Pn(t + h) = Pn(t)(1 − λnh) + Pn−1(t)λn−1h + o(h), n 6= 0

P0(t + h) = P0(t)(1 − λ0h) + o(h) 0 ⇒Pn(t) = −λnPn(t) + λn−1Pn−1(t) 0 P0(t) = −λ0Pn(t)

−λ t Equations can be solved recursively with P0(t) = P0(0)e 0

12 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Pure Birth Process. Generalization

Let the initial condition be Pn0 (0) = 1. The resulting equations are:

0 Pn(t) = −λnPn(t) + λn−1Pn−1(t), n > n0 P0 (t) = − P (t) n0 λn0 n0

Yule-Furry processes assumed λn = nλ

13 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Outline

1 Birth Processes

2 Birth-Death Processes

3 Relationship to Markov Chains

4 Linear Birth-Death Processes

5 Examples

14 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Birth-Death Processes

Notation • Pure Birth process: If n transitions take place during (0, t), we may refer to the process as being in state En. • Changes in the pure birth process: En → En+1 → En+2 → ... J. Virtamo 38.3143 Queueing Theory / Birth-death processes 4 • Birth-Death Processes consider transitions En → En−1 as The time-dependentwell as En → En solution+1 if n ≥ of1. a If BDn = process0, only E (continued)0 → E1 is allowed.

l0 l1 l2 l i-1 l i l i+1 0 1 2 ... i i+1

m1 m2 m3 m i m i+1 m i+2

The equations component wise 15 / 47 dπi(t)   = −(λi + µi)πi(t) + λi−1πi−1(t)+µi+1πi+1(t) i =1, 2,...   dt   % flows&' out ( % flows&' in (   dπ0(t)   = −λ0π0(t) + µ1π1(t)   dt   %flow&' out( %flow&' in(  Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Birth-Death Processes

16 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Birth-Death Processes

Assumptions

If the process at time t is in En, then during (t, t + h):

• Transition En → En+1 has probability λnh + o(h)

• Transition En → En−1 has probability µnh + o(h) • Probability that more than 1 change occurs = o(h).

Pn(t + h) = Pn(t)(1 − λnh − µnh)

+ Pn−1(t)(λn−1h) + Pn+1(t)(µn+1h) + o(h)

Time evolution of the probabilities 0 ⇒ Pn(t) = −(λn + µn)Pn(t) + λn−1Pn−1(t) + µn+1Pn+1(t)

17 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Birth-Death Processes

For n = 0

P0(t + h) = P0(t)(1 − λ0h) + P1(t)µ1h + o(h)

0 ⇒ P0(t) = −λ0P0(t) + µ1P1(t)

• If λ0 = 0, then E0 → E1 is impossible and E0 is an absorbing state. 0 • If λ0 = 0, then P0(t) = µ1P1(t) ≥ 0 and hence P0(t) increases monotonically.

Note:

limt→∞ P0(t) = P0(∞) = Probability of being absorbed.

18 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Steady-state distribution

0 P0(t) = −λ0P0(t) + µ1P1(t) 0 Pn(t) = −(λn + µn)Pn(t) + λn−1Pn−1(t) + µn+1Pn+1(t)

As t → ∞, Pn(t) → Pn(limit). 0 0 Hence, P0(t) → 0 and Pn(t) → 0. Therefore,

0 = −λ0P0 + µ1P1 λ0 ⇒P1 = P0 µ1 0 = −(λ1 + µ1)P1 + λ0P0 + µ2P2 λ0λ1 ⇒P2 = P0 µ1µ2 λ0λ1λ2 ⇒P3 = P0 etc µ1µ2µ2 19 / 47 ... Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Steady-state distribution

λ0 λ0λ1 λ0λ1λ2 P1 = P0; P2 = P0; P3 = P0; P4 = ... µ1 µ1µ2 µ1µ2µ2

The dependence on the initial conditions has disappeared. ∞ P After normalizing, i.e., Pn = 1: n=1

n−1 Q λi 1 µi+1 P = ; P = i=0 , n ≥ 1 0 ∞ n−1 n ∞ n−1 1 + P Q λi 1 + P Q λi µi+1 µi+1 n=1 i=0 n=1 i=0

20 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Steady-state distribution

n−1 Q λi 1 µi+1 P = ; P = i=0 , n ≥ 1 0 ∞ n−1 n ∞ n−1 1 + P Q λi 1 + P Q λi µi+1 µi+1 n=1 i=0 n=1 i=0

Ergodicity condition

Pn > 0, for all n ≥ 0, i.e.,:

∞ n−1 X Y λ i < ∞ µi+1 n=1 i=0

21 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Example.J. Virtamo A single server 38.3143 system Queueing Theory / Birth-death processes 7

Example 3. A single server system

l 0 1 - constant• constant arrival rate arrivalλ (Poisson rate arrivals)λ (Poisson m - stoppingarrivals) rate of the service µ (exponential distribution) The states• stopping of the system rate of service µ 0 server free  (exponential distribution) 1  1 server busy 0  • states of the system: 0 (server

ì í î

í ì î ~ Exp(m ) ~ Exp(l ) free), 1 (server busy)

d π (t)=− λπ (t)+µπ (t)  dt 0 0 0 1 −λλ  P (t) = −λP (t) + µP (t)   0 0 Q1=  d µ −µ π1(t)= λπ0(t0) − µπ1(t)    dt P (t) = λP (t) − µP (t)  1 0 1 BY adding both sides of the equations d dt(π0(t)+π1(t)) = 0 ⇒ π0(t)+π1(t) = constant = 1 ⇒ π1(t) = 1 − π0(t)

d d (λ+µ)t (λ+µ)t 22 / 47 dtπ0(t) + (λ + µ)π0(t)=µ ⇒ dt(e π0(t)) = µe

µ µ −(λ+µ)t π0(t)=λ+µ +(π0(0) − λ+µ)e λ λ −(λ+µ)t π1(t)=λ+µ +(π1(0) − λ+µ)e

equilibrium deviation from decays expo- distribution) *+ , ) the equilibrium*+ , nentially) *+ , Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

J. Virtamo 38.3143 Queueing Theory / Birth-death processes 7 Example. A single server system Example 3. A single server system

l 0 1 - constant arrival rate λ (Poisson arrivals) m - stopping rate of the service µ (exponential distribution) The states of the system0 P0(t) = −λP0(t) + µP1(t) 0 server free  0 1  1 server busyP1(t) = λP0(t) − µP1(t) 0 

ì í î

í ì î ~ Exp(m ) ~ Exp(l ) 0 Givend that:π (t)=P0−(tλπ) +(t)+P1µ(πt)(t =) 1, P0(t) + (λ + µ)P0(t) = µ.  dt 0 0 1 −λλ     Q =  d µ −µ π1(t)= λπ0(t) −µµπ1(t)  µ −(λ+µ)t  dt  P0(t) = + P0(0) − e BY adding both sides ofλ the+ equationsµ λ + µ d dt(π0(t)+π1(t)) = 0 ⇒λ π0(t)+π1(t) = constantλ = 1 ⇒−(λπ+1(µt)t = 1 − π0(t) P1(t) = + P1(0) − e d λ + µ d (λ+µ)t λ + µ(λ+µ)t dtπ0(t) + (λ + µ)π0(t)=µ ⇒ dt(e π0(t)) = µe

π (t)= µ +(π (0) − µ )e−(λ+µ)t Solution0 =λ Equilibrum+µ 0 λ+ distributionµ + Deviation from the λ λ −(λ+µ)t equilibriumπ1(t)=λ+ withµ +(π1 exponential(0) − λ+µ)e decay.

equilibrium deviation from decays expo- 23 / 47 distribution) *+ , ) the equilibrium*+ , nentially) *+ , J. Virtamo 38.3143 Queueing Theory / Birth-death processes 6 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples Example 2. Pure birth process (Poisson process) Poisson Process. Probabilities

 λi = λ  1 i =0  i =0, 1, 2,... πi(0) =  µi =0 0 i>0 Poisson Process  • Birthbirth probability probability per per time time unit unit is constant is initiallyλ the population size is 0 • Theconstant populationλ size is initially 0

l l l l l l 0 1 2 ... i-1 i All states are transient

All states are transient d Equations dt πi(t)=−λπi(t)+λπi−1(t) i>0   0 d Pi (t−) = −λPi (t) + λPi−1(t), i⇒> 0 −λt  dt π0(t)= λπ0(t) π0(t)=e  0  P0(t) = −λP0(t)

d λt λt −λt t # λt# # dt(e πi(t)) = λπi−1(t)e ⇒ πi(t)=e λ % πi−1(t )e dt 0 24 / 47 t # # π (t)=e−λtλ e−λt eλt dt# = e−λt(λt) 1 %0 1 & '( ) (λt)i Recursively π (t)= e−λt Number of births in interval (0,t) ∼ Poisson(λt) i i! Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Poisson Process. Probabilities

Equations 0 Pi (t) = −λPi (t) + λPi−1(t), i > 0 0 P0(t) = −λP0(t)

−λt ⇒ P0(t) = e Z t d λt λt −λt 0 λt0 0 [e Pi (t)] = λPi−1(t)e ⇒ Pi (t) = e λ Pi−1(t )e dt dt 0 Z t −λt −λt0 λt0 0 −λt P1(t) = e λ e e dt = e (λt) 0

(λt)i Recursively: P (t) = e−λt i i! Number of births in interval (0, t) ∼ Poisson(λt).

25 / 47 J. Virtamo 38.3143 Queueing Theory / Birth-death processes 5

Birth ProcessesExample Birth-Death Processes 1. Pure Relationship death to process Markov Chains Linear Birth-Death Processes Examples

λ =0 1 i = n Pure Death Process.i Probabilities   i =0, 1, 2,... πi(0) =  µi = iµ 0 i =! n   Pure Deathall Process individuals have the same the system starts from state n • All themortality individuals rate haveµ the same mortality rate µ • The population size is initially n State 0 is an absorbing state, 0 1 2 ... n-1 n m 2m 3m (n-1) m n m other states are transient State 0 is an absorbing state. The rest are transient.

Equations d − ⇒ −nµt  dt πn(t)= nµπn(t) πn(t)=e 0 Pn(t) = −nµPn(t)  d 0( ) =πi (t)+ =) (i +( 1)) −µπi+1((t)) −=iµπi(t) − i =0, 1, . . . , n − 1 Pi tdt i 1 µPi+1 t iµPi t , i 0,..., n 1  t $ d iµt iµt ⇒ −iµt $ iµt $ dt(e πi(t)) = (i + 1)µπi+1(t)e πi(t)26 = / 47 (i + 1)e µ %0 πi+1(t )e dt t $ $ −(n−1)µt −nµt (n−1)µt $ −(n−1)µt − −µt πn−1(t)=ne µ %0 e e dt = ne (1 e ) −µt$ & e '( ) Binomial distribution: the survival n −µt i −µt n−i −µt Recursively πi(t)= (e ) (1 − e ) probability at time t is e inde-  i    pendent of others Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Pure Death Process. Probabilities

Equations 0 Pn(t) = −nµPn(t) 0 Pi (t) = (i + 1)µPi+1(t) − iµPi (t), i = 0,..., n − 1

−nµt ⇒ Pn(t) = e Z t d iµt iµt −iµt 0 iµt0 0 [e Pi (t)] = (i+1)µPi+1(t)e ⇒ Pi (t) = (i+1)e µ Pi+1(t )e dt dt 0 Z t −(n−1)µt −nµt0 (n−1)µt0 0 −(n−1)µt −µt Pn−1(t) = ne µ e e dt = ne (1 − e ) 0

n Recursively: P (t) = (e−µt )i (1 − e−µt )n−i i i Binomial distribution: The survival probability at time t is e−µt independent of others. 27 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Outline

1 Birth Processes

2 Birth-Death Processes

3 Relationship to Markov Chains

4 Linear Birth-Death Processes

5 Examples

28 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Relation to CTMC

Inﬁnitesimal generator matrix:

  −λ0 λ0 0 ......  µ −(λ + µ ) λ 0 ......   1 1 1 1   0 µ −(λ + µ ) λ 0 ......  Q =  2 2 2 2  J. Virtamo . 38.3143 Queueing Theory / Birth-death processes  4  . 0 µ3 −(λ3 + µ3) λ3 0 ...  ......  The time-dependent. . solution of. a BD process. (continued). . .

l0 l1 l2 l i-1 l i l i+1 0 1 2 ... i i+1

m1 m2 m3 m i m i+1 m i+2

The equations component wise 29 / 47 dπi(t)   = −(λi + µi)πi(t) + λi−1πi−1(t)+µi+1πi+1(t) i =1, 2,...   dt   % flows&' out ( % flows&' in (   dπ0(t)   = −λ0π0(t) + µ1π1(t)   dt   %flow&' out( %flow&' in(  Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Relation to DTMC

Embedded Markov chain of the process. For t → ∞, deﬁne:

P(En+1|En) = Prob. of transition En → En+1

= Prob. of going to En+1conditional on being inEn

Deﬁne P(En−1|En) similarly. Then

P(En+1|En) v λn, P(En−1|En) v µn

λn µn P(En+1|En) = , P(En−1|En) = λn + µn λn + µn The same conditional probabilities hold if it is given that a transition will take place in (t, t + h) conditional on being in En.

30 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Outline

1 Birth Processes

2 Birth-Death Processes

3 Relationship to Markov Chains

4 Linear Birth-Death Processes

5 Examples

31 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Linear Birth-Death Processes

Linear Birth-Death Process

• λn = nλ

• µn = nµ

0 ⇒P0(t) = µP1(t) 0 Pn(t) = −(λ + µ)nPn(t) + λ(n − 1)Pn−1(t) + µ(n + 1)Pn+1(t)

Steady state behavior is characterized by:

0 lim P0(t) = 0 ⇒ P1(∞) = 0 t→∞

0 Similarly as t → ∞ Pn(∞) = 0

32 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Linear Birth-Death Processes

Steady state behavior is characterized by:

0 lim P0(t) = 0 ⇒ P1(∞) = 0 t→∞

0 Similarly as t → ∞ Pn(∞) = 0 Two cases can happen:

• If P0(∞) = 1 ⇒ the probability of ultimate extinction is 1.

• If P0(∞) = P0 < 1, the relations P1 = P2 = P3 ... = 0 imply with probability 1 − P0 that the population can increase without bounds.

The population must either die out or increase indeﬁnitely.

33 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Mean of a Linear Birth-Death Process

0 Pn(t) = −(λ + µ)nPn(t) + λ(n − 1)Pn−1(t) + µ(n + 1)Pn+1(t) ∞ P Deﬁne Mean by M(t) = nPn(t) n=1 ∞ 0 P 0 and consider M (t) = nPn(t), then: n=1

∞ ∞ 0 X 2 X M (t) = −(λ + µ) n Pn(t) + λ (n − 1)nPn−1(t) n=1 n=1 ∞ X + µ (n + 1)nPn+1(t) n=1

Write (n − 1)n = (n − 1)2 + (n − 1), (n + 1)n = (n + 1)2 − (n + 1)

34 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Mean of a Linear Birth-Death Process

∞ 0 X 2 M (t) = −(λ + µ) n Pn(t) n=1 ∞ ∞ ! X 2 X 2 + λ (n − 1) Pn−1(t) + µ (n + 1) Pn+1(t) + P1(t) n=1 n=1 ∞ ∞ ! X X + λ (n − 1)Pn−1(t) − µ (n + 1)Pn+1(t) + P1(t) n=1 n=1 ∞ ∞ 0 X X ⇒ M (t) = λ nPn(t) − µ nPn(t) = (λ − µ)M(t) n=1 n=1

(λ−µ)t M(t) = n0e if Pn0 (0) = 1

35 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Mean of a Linear Birth-Death Process

(λ−µ)t M(t) = n0e

• If λ > µ then M(t) → ∞ • If λ < µ then M(t) → 0 ∞ P 2 Similarly if M2(t) = n Pn(t) one can show that: n=1

0 M2(t) = 2(λ − µ)M2(t) + (λ + µ)M(t)

and when λ > µ, the variance is: λ + µ n e2(λ−µ)t 1 − e(µ−λ)t 0 λ − µ

36 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Outline

1 Birth Processes

2 Birth-Death Processes

3 Relationship to Markov Chains

4 Linear Birth-Death Processes

5 Examples

37 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Linear Birth-Death Process. Example

Let X(t) be the number of bacteria in a colony at instant t. Evolution of the population is described by: • the time that each of the individuals takes for division in two (binary ﬁssion), independently of the other bacteria • the life time of each bacterium (also independent) Assume that: • Time for division is exponentially dist. (rate λ) • Life time is also exponentially dist. (rate µ)

(λ−µ)t M(t) = n0e • If λ > µ then the population tends to inﬁnity • If λ < µ then the population tends to 0

38 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

A queueing system

s servers µ • s servers

K places µ • K waiting places λ • λ arrival rate (Poisson) µ • µ Exp(µ) holding time

µ (expectation 1/µ)

Is it a birth-death process?

39 / 47 J. Virtamo 38.3143 Queueing Theory / Birth-death processes 10

Example 1. A queueing system

Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples s palvelinta

m s servers A queueing system  K odotuspaikkaa   m  K waiting places l s servers   µ λ arrival rate (Poisson) • s servers m   µ Exp(µ) holding time (expectation 1/µ)  K places µ • K waiting places λ ì K=5 m í • λ arrival rate (Poisson) î s=4 µ • µ Exp(µ) holding time The number of customers inµ system N is(expectation an appropriate 1/µ state) variable Let “N =number of customers in the system” be the state variable. - uniquely• N determines determines the uniquely number the of customersnumber of in customers service and inin service waiting room and waiting room. - after• eachAfter arrival each andarrival departure and departure the remaining the remaining service tim servicees of thetimes customers of the in customers service are Exp( in serviceµ) distributed are Exp( (memoryless)µ) distributed (memoryless).

0lllllllll1 2 3 4 5 6 7 8 9

m 2m 3m 4m 4m 4m 4m 4m 4m 40 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Call blocking in an ATM network

An ATM network offers calls of two different types.   R = 1Mbps R = 2Mbps  1  2 λ1 = arrival rate λ2 = arrival rate   µ1 = mean holding time µ2 = mean holding time

Assume that the capacity of the link is inﬁnite:

Is it a birth-death process?

41 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Call blockingJ. Virtamo in an ATM 38.3143 network Queueing Theory / Birth-death processes 11 Example 2. Call blocking in an ATM network

An ATMA virtual network path (VP) offers of an calls ATM of network two different is oﬀered calls types. of two diﬀerent types.

  R1 = 1Mbps  R2 = 2Mbps R1 =1Mbps  R2 = 2Mbps   λ = arrival rate  λ = arrival rate   1  2  =arrivalµ1 = mean rate holding time  µ2 = mean= holdingarrival time rate λ1   λ2       µ1a)= Themean capacity holding of the link time is large (inﬁnite) µ2 = mean holding time Assume that the capacity of the link is inﬁnite: n2

l 2 (n22 +1)m l1 l1

n11m (n11 +1)m l2 n22m

The stateThe state variable variable isof the the Markov pair process(N , N in) thiswhere exampleN is thdeﬁnese pair (N1 the,N2), number where Ni deﬁnes of the number of class-i connections in1 progress.2 i class-i connections in progress. 42 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Call blocking in an ATM network

An ATM network offers calls of two different types.   R = 1Mbps R = 2Mbps  1  2 λ1 = arrival rate λ2 = arrival rate   µ1 = mean holding time µ2 = mean holding time

Assume that the capacity of the link is limited to 4.5 Mbps

Is it a birth-death process?

43 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Call blocking in an ATM network

An ATM network offers calls of two different types.  J. Virtamo 38.3143 Queueing Theory / Birth-death processes 12 R = 1Mbps R = 2Mbps  1  2 λ1 = arrivalCall rate blocking in an ATMλ network2 = arrival (continued) rate   µ1 = mean holding time µ2 = mean holding time Assume that the capacity of the link is limited to 4.5 Mbps b) The capacity of the link is 4.5 Mbps

n 1 44 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Exercise 1

Process deﬁnition • There are two transatlantic cables each of which handle one telegraph message at a time. • The time-to-breakdown for each has the same exponential random distribution with parameter λ. • The time to repair for each cable has the same exponential random distribution with parameter µ.

Tasks: • Draw the corresponding birth-death process. • Write its inﬁnitesimal generator. • Write differential equations for the probabilities. • Compute the steady state distribution

45 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Exercise 2

Birth-disaster process

Consider that Xt is a continuous-time Markov process deﬁned as follows: • Each individual gives a birth after an exponential random time of parameter λ, independent of each other. • A disaster occurs randomly at exponential random time of parameter δ. • Once a disaster occurs, it wipes out all the entire population.

Tasks: • What is the inﬁnitesimal generator matrix of the process? • What is the time evolution of M(t) = E[Xt ]?

46 / 47 Birth Processes Birth-Death Processes Relationship to Markov Chains Linear Birth-Death Processes Examples

Acknowledgments

Much of the material in the course is based on the following courses: • Queueing Theory / Birth-death processes. J. Vitano • Birth and Death Processes http://www.bibalex.org/supercourse/ • Performance modelling and evaluation. Birth-death processes. J. Campos • Discrete State Stochastic Processes J. Baik

47 / 47