Lecture 7: Transform Methods

Home , Generating function

TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected]

Text…& Sekretariat HFT6 Patrycja Chudzik

Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] Lecture 7: Transform Methods

www.mk.tu-berlin.de

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin Recall the deﬁnition of expectation: Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected] •

Text…& Sekretariat HFT6 Patrycja Chudzik

Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] E[X]= xpX(x),Xdiscrete with pmf pX x X and E[X]= xfX(x)dx, X continuous with pdf fX Z

This can be compactly written as www.mk.tu-berlin.de •

E[X]= xdFX(x) Z where

dFX(x)=FX(x) lim FX(x h) or dFX(x)=fX(x)dx h 0 #

Copyright G. Caire (Sample Lectures) 181 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected]

Text…& SekretariatThis HFT6 suggests a new notation that uniﬁes the above two (and therefore Patrycja Chudzik

• Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] applies to mixed-type RVs):

E[X]= xdFX. R The notation extends to all cases we have seen so far. For example: •

[g(X)] = g(x)dF , [g(X) Y ]= g(x)dF E X E X Y www.mk.tu-berlin.de | | Z Z

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587F Berlin (x) X yields a probability measure on the Borel -ﬁeld on deﬁned by R Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 • [email protected] B

Text…& Sekretariat HFT6 Patrycja Chudzik

Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] µ((a, b]) = F (b) F (a),µ(B)= dF B X X X 8 2 B ZB This is particularly simple to understand if you consider that B is a (countable) union of intervals. As already noticed, (R, ,µ) forms a B probability space itself. For any -measurable function g : R R, we write • B !

www.mk.tu-berlin.de E[g(X)] = g(x)dFX Z

Let B , and deﬁne g(x)=h(x)I x B , then • 2 B { 2 }

E[g(X)] = g(x)dFX = h(x)dFX Z ZB

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected]

X Text…& Consider a sequence of RVs , deﬁned on a common probability space Sekretariat HFT6 n Patrycja Chudzik

• Telefon +49 (0)30 314-28459 { }

Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] (⌦, , P), such that for all ! ⌦ we have limn Xn(!)=X(!), where X F 2 !1 is a well-deﬁned RV deﬁned on the same probability space.

We study the continuity of the operation E[ ]: • · a) Monotone convergence. If 0 X X for all n, !, then  n  n+1 limn E[Xn]=E[X]. !1

b) Dominated convergence. If X Y for all n, ! and [Y ] < , then www.mk.tu-berlin.de n E | |  1 limn E[Xn]=E[X]. !1 c) Bounded convergence. If Xn c for all n, !, then limn E[Xn]=E[X]. | |  !1 Since null events do not contribute to the expectation, the conditions of the • above theorems can be relaxed from “for all !” to almost everywhere (also indicated as almost surely). This means that they must hold up to sets of probability measure zero.

Copyright G. Caire (Sample Lectures) 184 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected]

Example: sums of non-negative RVs. Consider Z such that Z 0 a.s. Text…& i i Sekretariat HFT6 Patrycja Chudzik n • Telefon +49 (0)30 314-28459 { }

Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] X = Z X (almost surely), and let . Clearly, is a non-decreasing n i=1 i { n} sequence (a.s.). Deﬁne X = i1=1 Zi, by the monotone convergence theorem we have P P 1 E[X]= E[Zi] i=1 even though the sum may not converge.X

n www.mk.tu-berlin.de Example: suppose that E[ i=1 Xi ] < . Then, E[ i1=1 Xi]= i1=1 E[Xi]. • | | 1 n Proof: Let Y = 1 Xi , and notice that, for any n and ! ⌦, Xi Y i=1 | | P P 2 Pi=1  and that, by assumption, E[Y ] < . Hence, by dominated convergence: P 1 P n n n 1 1 Xi = lim Xi =lim Xi =lim [Xi]= [Xi] E E n n E n E E "i=1 # " !1 i=1 # !1 "i=1 # !1 i=1 i=1 X X X X X

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 A a =inf A a x x A Subject: TelefaxFor +49 (0)30 314-28320a set R, is the largest value such that , for all , [email protected]

• ⇢ { }  2 Text…& Sekretariat HFT6 Patrycja Chudzik b =sup A b x x A and is the smallest value such that for all . Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] { } 2 Let A denote a sequence of sets (not necessarily nested). We deﬁne • { n}

1 lim inf An =lim Am = Am n n 8 9 !1 !1 m n n=1 m n < \ = [ \

www.mk.tu-berlin.de and : ; 1 lim sup An =lim Am = Am n n 8 9 !1 !1 m n n=1 m n < [ = \ [ : ; Notice that the new sequence Bn = m n Am is nested, in fact, Bn • ✓ Bn+1 Bn+2 . Also, Bn = m n Am is nested, in fact, Bn Bn+1 ✓ ··· T ◆ ◆ Bn+2 . Hence, the limits are well-deﬁned. ··· S

Copyright G. Caire (Sample Lectures) 186 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected]

Text…& SekretariatFor HFT6 a sequence of numbers an , we deﬁne Patrycja Chudzik

• Telefon +49 (0)30 314-28459 { } Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected]

lim inf an =liminf am : m n n n !1 !1 { } and lim sup an =limsup am : m n n n { } !1 !1

www.mk.tu-berlin.de a has a limit if and only if • { n}

lim inf an =limsupan =liman n n n !1 !1 !1

Copyright G. Caire (Sample Lectures) 187 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected]

Text…& SekretariatFatou’s HFT6 Lemma: Let Xn be a sequence of RVs such that Xn Y a.s., Patrycja Chudzik

• Telefon +49 (0)30 314-28459 { } Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] where Y is a RV with well-deﬁned expectation (i.e., such that E[ Y ] < ), | | 1 then E lim inf Xn lim inf E[Xn] n  n In particular, if X 0 a.s.,h the!1 inequalityi !1 can be applied with Y =0. n

www.mk.tu-berlin.de

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: DeﬁnitionTelefax +49 (0)30 314-28320 30. The moment generating function of a RV X is deﬁned as [email protected]

Text…& Sekretariat HFT6 Patrycja Chudzik

Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] tX tx MX(t)=E e = e dFX Z ⇥ ⇤ MX(t) is a function R R+. ! ⌃

MGFs are related to the Laplace transform of the pdf (for continuous RVs), in

•

www.mk.tu-berlin.de fact, tX tx E e = e fX(x)dx Z ⇥ ⇤

Copyright G. Caire (Sample Lectures) 189 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected]

Text…& SekretariatIf HFT6M X(t) < for t in an open interval containing the origin, then Patrycja Chudzik

• Telefon +49 (0)30 314-28459 1 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] k dk (k) 1. E[X ]= k MX(t) = MX (0); dt t=0 2. The function MX can be expanded in Taylor series as 1 [Xk] M (t)= E tk X k! k=0

www.mk.tu-berlin.de within its convergence domain. 3. Convolution – product property: if X and Y are independent, then

MX+Y (t)=MX(t)MY (t)

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: DeﬁnitionTelefax +49 (0)30 314-28320 31. The characteristic function of a RV X is deﬁned as [email protected]

Text…& Sekretariat HFT6 Patrycja Chudzik

Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] |uX |ux X(u)=E e = e dFX Z ⇥ ⇤ where | = p 1. ⌃

Ch.Fs are related to the Fourier transform of the pdf (for continuous RVs), in

•

www.mk.tu-berlin.de fact, |uX |ux E e = e fX(x)dx Z ⇥ ⇤

Copyright G. Caire (Sample Lectures) 191 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: TheoremTelefax +49 (0)30 314-28320 15. The characteristic function X satisﬁes: [email protected]

Text…& Sekretariat HFT6 Patrycja Chudzik

Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected]

1. X(0) = 1, X(u) 1 for all u R. | |  2

2. X is uniformly continuous on R.

3. X is non-negative deﬁnite, which is to say that

www.mk.tu-berlin.de (u u )z z⇤ 0 X i k i k Xi,k

for all reals u1,u2,...,un and complex coefﬁcients z1,z2,...,zn.

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: TheoremTelefax +49 (0)30 314-28320 16. a) If the k-th derivative of X at u =0exists, then [email protected]

Text…& Sekretariat HFT6 Patrycja Chudzik

Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] E[ X k] < ,keven | |k 1 1 E[ X ] < ,kodd ⇢ | | 1

b) If E[ X k] < , then | | 1

k i www.mk.tu-berlin.de [X ] (u)= E (|u)i + o(uk) X i! i=0 X

k k dk and also E[X ]=| k X . du u=0 Proof. It follows from Taylor’s series theorem for a function of complex variable.

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: TheoremTelefax +49 (0)30 314-28320 17. Let MX(t) and X(u) denote the MGF and the Ch.F of X. For [email protected]

Text…& Sekretariat HFT6 Patrycja Chudzik a>0 anyTelefon +49 (0)30 314-28459 the following statements are equivalent:

Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected]

1. M(t) < for all t

2. X(s), with s C, is analytic on the strip Im s

1/k 1 www.mk.tu-berlin.de lim supk mk /k! a . !1{| | } 

If the above conditions hold, then MX(t) may be extended analytically to the strip Im s

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: TheoremTelefax +49 (0)30 314-28320 18. If X and Y are independent, then [email protected]

Text…& Sekretariat HFT6 Patrycja Chudzik

Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected]

X+Y (u)=X(u)Y (u)

Theorem 19. For a, b R and Y = aX + b, then 2

|bu

(u)=e (au) Y X www.mk.tu-berlin.de

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Deﬁnition 32. Let X be an n-dimensional random vector with joint cdf F (x). Einsteinufer 25 X 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected] Then, its joint characteristic function is deﬁned as

Text…& Sekretariat HFT6 Patrycja Chudzik

Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] T n |u X | i=1 uixi X(u)=E e = n e dFX P h i ZR ⌃

In particular, for a pair of jointly distributed RVs X and Y , we have •

|(uX+vY ) www.mk.tu-berlin.de X,Y (u, v)=E e h i

If X and Y are independent, then •

X,Y (u, v)=X(u)Y (v)

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected]

Text…& X f SekretariatFrom HFT6 Fourier transform theory: if is continuous with density X, then Patrycja Chudzik

• Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected]

1 1 |ux f (x)= (u)e du X 2⇡ X Z1

for all x R where fX is differentiable. 2

If the above integral is not absolutely convergent, we interpret is as the

www.mk.tu-berlin.de • Cauchy principal value integral.

Sufﬁcient (but not necessary) condition such that is the Ch.F of a • X continuous RV is that 1 (u) du < | X | 1 Z1

Copyright G. Caire (Sample Lectures) 197 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected]

The general case is more complicated and it is stated as follows: Text…& Sekretariat HFT6 Patrycja Chudzik

• Telefon +49 (0)30 314-28459

Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] theorem. Let X have cdf F and Ch.F. . Deﬁne F as X X X

1 F X(x)= FX(x)+limFX(x h) 2 h 0  # Then, T |au |bu e e

F X(b) F X(a)= lim X(u) du

T www.mk.tu-berlin.de T 2⇡|u !1 Z

Notice: analogous inversion formulas hold of the multidimensional (random vector) case, from the theory of multidimensional Fourier transform.

Corollary 5. X and Y have the same Ch.Fs if and only if they have the same cdfs.

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 n Einsteinufer 25 2 10587 Berlin

X (0, 1) Y = X Y LetTelefon +49 (0)30 314i-29668 be i.i.d. RVs and i . The RV is said to be Subject: Telefax +49 (0)30 314-28320 i=1 [email protected] ⇠ N 2

Text…& Sekretariat HFT6 “centralPatrycja Chudzik chi-squared” with n degrees of freedom (we say Y (n). For

Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] P ⇠ n =2 m (even integer) ﬁnd the pdf and the cdf of Y .

Solution:

First, we ﬁnd the Ch.F of Z = X2. We have

|uX2 1 1 (1 |u)x2 Z(u)= [e ]= e 2 dx E

p 2⇡ www.mk.tu-berlin.de Z1 Notice that this integral is absolutely convergent for all u R, and in particular, 2 replacing 1 |u with a complex variable s, we notice that 2

1 1 sx2 gZ(s)= e dx p2⇡ Z1 converges for all Re s > 0 (right half-plane). { }

Copyright G. Caire (Sample Lectures) 199 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 It followsEinsteinufer 25 that we can use analytic continuation, compute g (s) for real s in an 10587 Berlin Z

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected]

interval [a, b] containing the point 1/2, and then extend the result to the vertical Text…& Sekretariat HFT6 Patrycja Chudzik

Telefon +49 (0)30 314-28459 1

Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected]<-berlin.de Re s

www.mk.tu-berlin.de 1 (u)=g (1/2 |u)= Z Z p1 |2u 2m 2 Now, consider Y = i=1 Xi . By the Sum-of-RVs theorem we have that P 1 (u)= (u)2m = Y Z (1 |2u)m

Copyright G. Caire (Sample Lectures) 200 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: TheTelefax +49 (0)30 pdf 314-28320 of Y is found by using the inversion formula (similar to the inverse [email protected]

Text…& Sekretariat HFT6 Patrycja Chudzik Fourier transform) applied to Y (u). To this purpose, we notice that the Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] Laplace transform of the pdf of Y is obtained as follows:

sY 1 Y (s)=E[e ]=Y (u) = L |u=|s (1 + 2s)m

This is analytic in the right half-plane Re s > 1/2, and it has a pole of { }

multiplicity m at s0 = 1/2. Since the ROC of Y (s) contains the imaginary

www.mk.tu-berlin.de L axis, it follows that the inverse transform of Y (u) and the inverse Laplace transform of (s) coincide. The latter can be easily calculated using contour LY integration and the Cauchy residue theorem.

Copyright G. Caire (Sample Lectures) 201 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: In particular,Telefax +49 (0)30 314-28320 for a suitable integration path c + ju with c> 1/2 and [email protected]

Text…& Sekretariat HFT6 Patrycja Chudzik u ( , ), we have: Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] 2 1 1

1 1 f (y)= Y L (1 + 2s)m ⇢ c+j sy 1 1 e = mds 2⇡| c j (1 + 2s) Z 1

m sy www.mk.tu-berlin.de 2 e = , Res 1 m poles on the left half of the complex plane (2 + s) ! X 2 mesy 1 = , Res 1 m (2 + s) 2 !

The residue at the pole with multiplicity m is computed as follows:

Copyright G. Caire (Sample Lectures) 202 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory g(s)

Prof. Dr. Giuseppe Caire Let f(s)= be a function with a pole at s with multiplicity m (assume Berlin, 1. Month 2014 m 0 Einsteinufer 25 (s s ) 10587 Berlin 0

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected] g(s ) is analytic at s ), then 0

Text…& Sekretariat HFT6 Patrycja Chudzik

Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] m 1 1 d g(s) Res f(s),s = { 0} (m 1)! dsm 1 s=s0 Applying the above formula to our case, we obtain

m sy m 1 2 e 1 1 d m sy Res , = 2 e 1 m 1 m ( + s) 2 (m 1)! ds s= 1/2 www.mk.tu-berlin.de 2 ! m 2 m 1 sy = y e (m 1)! s= 1/2 m 2 m 1 y/2 = y e (m 1)! This holds for y 0. Clearly, for y<0 the pdf of Y is zero since Y 0 with probability 1 (it is a sum of squares).

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: DeﬁnitionTelefax +49 (0)30 314-28320 33. We say that the sequence of cdfs F1,F2,... converges to the [email protected]

Text…& Sekretariat HFT6 Patrycja Chudzik F F F cdfTelefon +49 (0)30 314-28459(we write n ) if

Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] !

lim Fn(x)=F (x) n !1 for all x R where F is continuous. 2 ⌃

Theorem 20. Let F1,F2,... be a sequence of cdfs with Ch.Fs 1, 2,...,

www.mk.tu-berlin.de respectively.

a) If F F , where F is a cdf with Ch.F , then n !

lim n(u)=(u), u n R !1 8 2

b) Conversely, if (u)=limn n(u) exists and is continuous at u =0, then !1 is a valid characteristic function of some cdf F such that F F . n !

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: DeﬁnitionTelefax +49 (0)30 314-28320 34. (Convergence in distribution). A sequence of RVs X1,X2,... [email protected]

Text…& Sekretariat HFT6 Patrycja Chudzik with cdfs F ,F ,... is said to converge in distribution to a RV X with cdf F , if Telefon +49 (0)30 314-28459 1 2

Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected]

Fn F as n . ! !1 D In this case, we write X X. n ! ⌃

Theorem 21. Law of Large Numbers. Let X1,X2,... be a sequence of i.i.d. n RVs with ﬁnite mean µ = E[X1] and deﬁne their partial sum Sn = i=1 Xi. Then, 1 D P

S µ n www.mk.tu-berlin.de n !

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected]

Text…& Sekretariat HFT6 Patrycja Chudzik

Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] X ,X ,... Theorem 22. Central Limit Theorem. Let 1 2 be a sequence of i.i.d. 2 RVs with ﬁnite mean µ = E[X1] and variance =var(X1), and deﬁne their n partial sum Sn = i=1 Xi. Then, P S nµ D n (0, 1) pn2 ! N

www.mk.tu-berlin.de Notice the abuse of notation ...here (0, 1) indicates a Gaussian(0,1) variable. N

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: TheoremTelefax +49 (0)30 314-28320 23. Local CLT. Let X1,X2,... be a sequence of i.i.d. RVs with ﬁnite [email protected]

Text…& Sekretariat HFT6 Patrycja Chudzik [X ]=0 (X )=1 meanTelefon +49 (0)30 314-28459 and variance var . Suppose that their common Ch.F. E 1 1 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] satisﬁes (u) rdu < | | 1 Z n for some integer r 1. The pdf f of Y = 1 X exists for all n r and n n pn i=1 i furthermore 1 y2P/2 fY (y) e n ! p2⇡

www.mk.tu-berlin.de uniformly for all y R. 2

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 n Subject: Telefax +49 (0)30 314-28320 [email protected] S = X n nµ The LLN states that n i, for large , is typically close to . i=1 Text…& Sekretariat HFT6 Patrycja Chudzik •

Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] P The CLT states that with high probability we may ﬁnd S between the values • n nµ pn2 and nµ + pn2 (e.g., the probability is larger than 0.98 if 3. Notice that the size of this high probability interval is O(pn), which is “much • less than n”.

www.mk.tu-berlin.de A “large deviation” from this typical behavior is when S is found far apart • n from its expected value nµ.

We are interested in studying the probability •

P(Sn >na), for a>µ

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: TelefaxThe +49 (0)30 314-28320 log-moment generating function is deﬁned as [email protected]

Text…& • Sekretariat HFT6 Patrycja Chudzik

Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] tX mX(t) = log MX(t) = log E[e ]

Properties: • 1. mX(0) = log MX(0) = 0.

M (0) X0 www.mk.tu-berlin.de 2. m0 (0) = = µ. X MX(0) 3. mX(t) is convex in its domain (where it is ﬁnite), in fact,

2 2 tX tX tX 2 MX00 (t)MX(t) (MX0 (t)) E[X e ]E[e ] E[Xe ] mX00 (t)= 2 = 2 0 MX(t) MX(t) where the latter follows from Cauchy-Schwarz, applied to the RVs Y = XetX/2 and Z = etX/2.

Copyright G. Caire (Sample Lectures) 209 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: DeﬁnitionTelefax +49 (0)30 314-28320 35. Fenchel-Legendre Transform. The F-L transform of mX(t) is [email protected]

Text…& Sekretariat HFT6 Patrycja Chudzik

theTelefon +49 function (0)30 314-28459

Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] mX⇤ (a)=sup at mX(t) t { } 2R ⌃

Theorem 24. Large deviations. Let X1,X2,...be a sequence of i.i.d. RVs with mean µ and suppose that their common MGF, M(t), exists and it is ﬁnite in a neighborhood of the origin (t =0). Consider a>µsuch that (X1 >a) > 0 and P

www.mk.tu-berlin.de let m(t) = log M(t). Then, m⇤(a) > 0 and

1 n log P Xi >na m⇤(a), as n n ! !1 i=1 ! X

Copyright G. Caire (Sample Lectures) 210 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 n 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Notice:Telefax +49 (0)30 314-28320 in practice, this means that the probability ( Xi >na) vanishes [email protected] P i=1

Text…& Sekretariat HFT6 Patrycja Chudzik exponentially as Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] n P . nm⇤(a) P Xi >na = e i=1 ! The exponential rate (or simplyX “exponent”) at which the large deviation probability vanishes is given by the F-L transform m⇤(a) of the log-MGF m(t).

www.mk.tu-berlin.de

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected] Let ai : i =0, 1, 2,... denote a sequence of real numbers. Its associated

Text…& Sekretariat HFT6 Patrycja Chudzik • { } Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] function is deﬁned as

1 i Ga(s)= ais ,sC 2 i=0 X

G (s) a is meaningful inside its region of convergence (ROC), deﬁned as the www.mk.tu-berlin.de • subset of C for which the sum is absolutely convergent.

The intersection of the ROC with the real line R is called interval of • convergence.

The elements of the sequence a can be recovered from Ga(s) by setting • (i) ai = Ga (0)/i!.

Copyright G. Caire (Sample Lectures) 212 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected]

Text…& SekretariatGenerating HFT6 functions are often easier to deal with than directly handling the Patrycja Chudzik

• Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] associated sequences.

Convolution property: Given a = a : i 0 and b = b : i 0 , the • { i } { i } convolution sequence c = a b is deﬁned as ⇤ i

ci = ajbi j = a0bi + a1bi 1 + a2bi 2 + + aib0

··· www.mk.tu-berlin.de j=0 X Then, the associated generating functions satisfy

Gc(s)=Ga(s)Gb(s)

Copyright G. Caire (Sample Lectures) 213 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 2 [email protected] n n 2n

Text…& SekretariatExample: HFT6 show the combinatorial identity = . Patrycja Chudzik i=0 i n

• Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] P

Example: Let X and Y be independent Poisson RVs with parameters and • µ, respectively. Find the distribution of Z = X + Y .

www.mk.tu-berlin.de

i Convergence: power series of the type G (s)= 1 a s (i.e., involving only • a i=0 i non-negative powers of s) converge in a circle of given radius R. That is: the sum is absolutely convergent for all s R. | | | | Furthermore, for all R0

Copyright G. Caire (Sample Lectures) 214 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected]

Text…& SekretariatDifferentiation: HFT6 Ga(s) can be differentiated (any number of times) or Patrycja Chudzik

• Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] integrated term by term for all s

Uniqueness: if Ga(s)=Gb(s) for all s such that s

Abel’s theorem: if a 0 for all i =0, 1, 2,... and G (s) < for all s < 1, • i | a | 1 | |

then www.mk.tu-berlin.de 1 lim Ga(s)= ai s 1 " i=0 where the sum in the left-hand side may beX ﬁnite or inﬁnite.

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin Deﬁnition 36. Let X denote a discrete RV taking on values in 0, 1, 2,... , with Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected] { }

Text…& pmfSekretariat HFT6p X[0],pX[1],pX[2],.... The probability generating function of X is given by Patrycja Chudzik

Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected]

1 i X GX(s)= pX[i]s = E[s ] i=0 X ⌃

t t |u Notice: by letting s = e , we have that GX(e )=MX(t). Also, by letting s = e

|u www.mk.tu-berlin.de we have that GX(e )=X(u).

Directly from the deﬁnition we have that G (0) = p [0], and that G (1) = 1. • X X X Directly from the convolution property we have that if X and Y are • independent, then GX+Y (s)=GX(s)GY (s)

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected] X c

Text…& G (s)= [s ]=s SekretariatConstant HFT6 RV. X E . Patrycja Chudzik

• Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected]

Bernoulli RV. G (s)=(1 p)s0 + ps =1 p + ps. • X

i 1 i ps 1 Geometric RV. GX(s)= i=1 p(1 p) s = 1 s(1 p). • P isi (s 1) 1 Poisson RV. GX(s)= i=0 i! e = e .

•

www.mk.tu-berlin.de P Binomial RV. Recalling that X can be written as n X where the X are • i=0 i i Bernoulli i.i.d., we obtain P n G (s)= G (s)=G (s)n =(1 p + ps)n X Xi X1 i=1 Y

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 X G (s) Subject: TheoremTelefax +49 (0)30 314-28320 25. Let have probability generating function X . Then, [email protected]

Text…& Sekretariat HFT6 Patrycja Chudzik [X]=G (1) a) 0 . TelefonE +49 (0)30 314-28459 X

Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] (k) b) E[X(X 1)(X 2) (X k + 1)] = G (1). ··· X

Notice: The quantity E[X(X 1)(X 2) (X k + 1)] is known as the “k-th (k) ··· factorial moment” of X. Also, GX (1) should be intended as the limit for s 1

www.mk.tu-berlin.de when the ROC of G is s < 1. X | |

Example: in order to calculate the variance of X, we write

2 2 2 2 var(X)=E[X ] E[X] = E[X(X 1) + X] E[X] = GX00 (1) + GX0 (1) G0(1)

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: TheoremTelefax +49 (0)30 314-28320 26. Let X1,X2,... denote a sequence of i.i.d. discrete RVs with [email protected]

Text…& Sekretariat HFT6 Patrycja Chudzik G (s) N commonTelefon +49 (0)30 314-28459 generating function X , and let denote a RV taking on values

Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected]

in 0, 1, 2,..., independent of the Xi’s, with generating function GN (s). Then, the N new random variable S = i=1 Xi has generating function P GS(s)=GN (GX(s))

www.mk.tu-berlin.de n Notice: if N = n is a constant, then GN (s)=s and we get back the n well-known convolution theorem, where GN (GX(s)) = GX(s) .

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 N N Subject: A henTelefax +49 (0)30 314-28320 lays eggs, where is Poisson with parameter . Each egg hatches [email protected]

Text…& Sekretariat HFT6 Patrycja Chudzik p K independently with probability . Let denote the number of chicks. We want Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] toﬁnd the distribution of K. Clearly, N

K = Xi i=1 X where the Xi are i.i.d. Bernoulli. Hence, using the compounding theorem, we

have

www.mk.tu-berlin.de

(GX(s) 1) (1 p+sp 1) (s 1)p GK(s)=GN (GX(s)) = e = e = e

Therefore we conclude that K is Poisson, with parameter p.

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected]

Text…& X Y SekretariatThe HFT6 joint probability generating function of and is given by Patrycja Chudzik

• Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected]

X Y 1 1 i j GX,Y (s, r)=E[s r ]= pX,Y [i, j]s r i=0 j=0 X X

X Y G (s, r)=G (s)G (r) s, r and are independent if and only if X,Y X Y for all .

• www.mk.tu-berlin.de Defective RVs. It might happen that certain RVs take on the value + with • 1 positive probability. For such RVs, we have that G (s) converges for s < 1 X | | and that lim GX(s)= pX[i]=1 P(X =+ ) s 1 1 " i The moments of X are all equalX to + . 1

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected]

We have a collection A1,A2,...,An of events. Let X be the number of such Text…& Sekretariat HFT6 Patrycja Chudzik • { } Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] that occur in a given random experiment. For example, with n letters

and n envelopes picked at random, let Ai denote the event that letter i is put into its correct envelope, and we are interested in the probability P(X = k) for some 0 k n, that is, the probability that exactly k letters are put into   their correct envelopes.

The problem consists of expressing the pmf of X in terms of probabilities of

• www.mk.tu-berlin.de the form P(Ai1,Ai2,...,Aim) for m =1, 2,...,n.

Let us introduce the notation: •

Sm = P(Ai1,Ai2,...,Aim) i

with the convention that S0 =1.

Copyright G. Caire (Sample Lectures) 222 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected]

Text…& SekretariatWe HFT6 have Patrycja Chudzik

• Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected]

Sm = E[IAi IAi IAi ] 1 2 ··· m i

= E IAi IAi IAi 2 1 2 ··· m3 i

4 5 www.mk.tu-berlin.de

= E E IAi IAi IAi X 2 2 1 2 ··· m 33 i1

Copyright G. Caire (Sample Lectures) 223 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

TelHence,efon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected] n

• Text…& Sekretariat HFT6 i Patrycja Chudzik

Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Sm = (X = i) Prof. Dr. Giuseppe Caire [email protected] P m i=0 X ✓ ◆ We introduce the generating functions: • n n m i GS(s)= Sms ,GX(s)= s P(X = i) m=0 i=0

X X www.mk.tu-berlin.de

Multiplying by sm and summing over m, we obtain • n n i n G (s)= (X = i) sm = (1 + s)i (X = i)=G (s + 1) S P m P X i=0 m=0 i=0 X X ✓ ◆ X from which we get G (s)=G (s 1). X S

Copyright G. Caire (Sample Lectures) 224 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected] i

Text…& SekretariatEquating HFT6 the coefﬁcients of s , we ﬁnd: Patrycja Chudzik

• Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected]

n j i j P(X = i)= ( 1) Sj, 0 i n i   j=i X ✓ ◆ This is the well-known Waring theorem formula, that we have already obtained in a Homework in a fairly more complicated way.

www.mk.tu-berlin.de

Copyright G. Caire (Sample Lectures) 225 Branching processes

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected]

Text…& SekretariatProcesses HFT6 that describe in probabilistic terms the evolution of populations. Patrycja Chudzik

• Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected]

Useful to describe cells, atoms, etc ... • At each time, each member of the current generation gives birth to a random • number of children. The random variable that describes the number of generated children from a given parent is called family size.

www.mk.tu-berlin.de Assumption: the family sizes form a collection of i.i.d. RVs, with common pmf • p and probability generating function G.

Copyright G. Caire (Sample Lectures) 226 Generation distribution

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected]

Text…& SekretariatLet HFT6 Zn denote the number of elements in generation n, where we assume Patrycja Chudzik

• Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] Z0 =1with probability 1.

Zn The probabiliy generating function of Zn is denoted by Gn(s)=E[s ]. •

Theorem 27. Gn+m(s)=Gm(Gn(s)) = Gn(Gm(s)), therefore,

G (s)=G(G(G( G(s))) ) n

··· ··· www.mk.tu-berlin.de n times | {z }

Copyright G. Caire (Sample Lectures) 227 Moments of the generation distribution

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: LemmaTelefax +49 (0)30 314-28320 26. Let Z1 denote the family size RV, with moments [Z1]=µ and [email protected] E

Text…& Sekretariat HFT6 2 Patrycja Chudzik var (Z )= . Then, Telefon +49 (0)30 3141-28459

Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected]

n2 if µ =1 n 2 n n 1 E[Zn]=µ , var(Zn)= (µ 1)µ ( µ 1 if µ =1 6

www.mk.tu-berlin.de

Copyright G. Caire (Sample Lectures) 228 Example: geometric branching process

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected] Let Z be geometric, such that 1

Text…& Sekretariat HFT6 • Patrycja Chudzik

Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] p [k]=qpk, with q =1 p Z1

We have • 1 q G(s)= qpksk =

1 sp www.mk.tu-berlin.de kX=0

We can show using induction that • n (n 1)s n+1 ns if p = q =1/2 n n n 1 n 1 Gn(s)= q(p q ps(p q ) ( pn+1 qn+1 ps(pn qn) if p = q 6

Copyright G. Caire (Sample Lectures) 229 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: TelefaxA +49 (0)30 typical 314-28320 question that we ask is the probability of ultimate extinction: notice [email protected]

Text…& • Sekretariat HFT6 Patrycja Chudzik that if at a certain n we have Z =0, then the population is extinct, it will Telefon +49 (0)30 314-28459 n

Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] remain equal to zero for ever. The probability of ultimate extinction is deﬁned as •

P(ultimate extinction)=P Zn =0 { } n ! [

www.mk.tu-berlin.de We have that Z =0 Z =0. Hence, the sequence of events • { n } ✓ { n+1 } Z =0 is nested (increasing). { n } Using continuity of the probability measure, we have •

P Zn =0 = P lim Zn =0 =limP(Zn = 0) { } n { } n n ! !1 !1 [ ⇣ ⌘

Copyright G. Caire (Sample Lectures) 230 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected] For the geometric branch process, we have explicitly G (s), hence, n Text…& Sekretariat HFT6 Patrycja Chudzik • Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] n n+1 if p = q P(Zn = 0) = Gn(0) = q(pn qn) ( pn+1 qn+1 if p = q 6

Hence, it follows that P(ultimate extinction)=1if p q, and that • 

www.mk.tu-berlin.de q (ultimate extinction)= , if p>q P p

This result is intuitive: notice that E[Z1]=p/q, then, if E[Z1] 1 the process •  will sooner or later die, almost surely. On the contrary, if E[Z1] > 1, the process will last forever with positive probability 1 q/p.

Copyright G. Caire (Sample Lectures) 231 Probability of ultimate extinction: the general case

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected]

Text…& Sekretariat HFT6 Patrycja Chudzik

Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected]

Theorem 28. For a branching process with family size distribution with mean E[Z1]=µ, the probability of ultimate extinction

(ultimate extinction)= lim (Zn = 0) P n P !1

is equal to the value ⌘ of the smallest non-negative solution of the equation www.mk.tu-berlin.de s = G(s). Also, ⌘ =1if µ<1 and ⌘ < 1 if µ>1. If µ =1, then ⌘ =1if Z1 has strictly positive variance.

Copyright G. Caire (Sample Lectures) 232 TU Berlin | Sekr. HFT 6 | Einsteinufer 25 | 10587 Berlin Faculty of Electrical Engineering and Computer Systems Department of Telecommunication Firma xy Systems Herrn Mustermann Beispielstraße 11 12345 Musterstadt Information and Communication Theory

Prof. Dr. Giuseppe Caire

Berlin, 1. Month 2014 Einsteinufer 25 10587 Berlin

Telefon +49 (0)30 314-29668 Subject: Telefax +49 (0)30 314-28320 [email protected]

Text…& Sekretariat HFT6 Patrycja Chudzik

Telefon +49 (0)30 314-28459 Telefax +49 (0)30 314-28320 Prof. Dr. Giuseppe Caire [email protected] End of Lecture 7

www.mk.tu-berlin.de