Lecture 12: Central Limit Theorem and Cdfs Raw Moment: 0 N Μn = E(X )

Home , Central moment, Standardized moment

CLT Moments of Distributions Moments

Lecture 12: Central Limit Theorem and CDFs Raw moment: 0 n µn = E(X )

Statistics 104 Central moment: 2 Colin Rundel µn = E[(X − µ) ]

February 27, 2012 Normalized / Standardized moment:

µn σn

Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 1 / 22

CLT Moments of Distributions CLT Moments of Distributions Moment Generating Function Moment Generating Function - Properties

The moment generating function of a random variable X is deﬁned for all If X and Y are independent random variables then the moment generating real values of t by function for the distribution of X + Y is (P tx tX x e P(X = x) If X is discrete MX (t) = E[e ] = R tx t(X +Y ) tX tY tX tY x e P(X = x)dx If X is continuous MX +Y (t) = E[e ] = E[e e ] = E[e ]E[e ] = MX (t)MY (t) This is called the moment generating function because we can obtain the Similarly, the moment generating function for S , the sum of iid random raw moments of X by successively diﬀerentiating MX (t) and evaluating at n t = 0. variables X1, X2,..., Xn is 0 0 MX (0) = E[e ] = 1 = µ0 n MS (t) = [MX (t)] d d n i M0 (t) = E[etX ] = E etX = E[XetX ] X dt dt 0 0 0 MX (0) = E[Xe ] = E[X ] = µ1

d d d M00(t) = M0 (t) = E[XetX ] = E (XetX ) = E[X 2etX ] X dt X dt dt 00 2 0 2 0 MX (0) = E[X e ] = E[X ] = µ2

Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 2 / 22 Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 3 / 22 CLT Moments of Distributions CLT Moments of Distributions Moment Generating Function - Unit Normal Moment Generating Function - Unit Normal, cont.

Let Z ∼ N (0, 1) then 0 d t2/2 t2/2 Z ∞ MZ (t) = e = te tZ tx 1 −x2/2 dt MZ (t) = E[e ] = e √ e dx 0 0 −∞ 2π µ1 = MZ (0) = 0 Z ∞ 2 1 − x −tx = √ e 2 dx 2 2 2π −∞ 00 d t /2 2 t /2 MZ (t) = te = (1 + t )e ∞ 2 dt 1 Z (x−t) t2 − 2 + 2 0 00 = √ e dx µ2 = MZ (0) = 1 2π −∞ Z ∞ 2 t2/2 1 − (x−t) = e √ e 2 dx d 2 2 M000(t) = (1 + t2)et /2 = (3t + t3)et /2 −∞ 2π Z dt t2/2 0 000 = e µ3 = MZ (0) = 0

d 2 2 M000(t) = (3t + t3)et /2 = et /2(3 + 6t2 + t4) Z dt 0 0000 µ4 = MZ (0) = 3

Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 4 / 22 Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 5 / 22

CLT Moments of Distributions CLT Moments of Distributions Sketch of Proof

Central Limit Theorem Proposition Let X , X ,... be a sequence of independent and identically distributed 1 2 Let X1, X2,... be a sequence of independent and identically distributed random 2 random variables each having mean µ and variance σ . Then the distribution variables and Sn = X1 +···+Xn. The distribution of Sn is given by the distribution

of function fSn which has a moment generating function MSn with n ≥ 1. X + ··· + X − nµ 1 √ n Let Z being a random variable with distribution function fZ and moment generat- σ n ing function MZ . tends to the unit normal as n → ∞. If MSn (t) → MZ (t) for all t, then fSn (t) → fZ (t) for all t at which fZ (t) is continuous. That is, for −∞ < a < ∞,

t2/2 We can prove the CLT by letting Z ∼ N (0, 1), MZ (t) = e and then Z a t2/2 X1 + ··· + Xn − nµ 1 −x2/2 showing for any S that M √ → e as n → ∞. P √ ≤ a → √ e dx = Φ(a) as n → ∞ n Sn/ n σ n 2π −∞

Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 6 / 22 Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 7 / 22 CLT Moments of Distributions CLT Moments of Distributions Proof of the CLT Proof of the CLT, cont. √ Some simplifying assumptions and notation: The moment generating function of Xi / n is given by E(X ) = 0 i tX t M √ (t) = E exp √ i = M √ Var(Xi ) = 1 Xi / n n Xi n

MXi (t) exists and is ﬁnite √ P n √ L(t) = log M(t) and this the moment generating function of Sn/ n = i = 1 Xi / n is L’Hospital’s Rule: given by f (x) f 0(x) lim = lim t n x→∞ g(x) x→∞ g 0(x) M √ (t) = M √ Sn/ n Xi n

√ Therefore in order to show MSn/ n → MZ (t) we need to show n t 2 M √ → et /2 Xi n

Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 8 / 22 Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 9 / 22

CLT Moments of Distributions CLT Moments of Distributions Proof of the CLT, cont. Proof of the CLT, cont.

2 LXi (t) = log MXi (t) √ n t /2 [MX (t/ n)] → e L (0) = log M (0) = log 1 = 0 i Xi Xi √ 2 nLXi (t/ n) → t /2

0 d M (t) 0 Xi √ √ 1 L (t) = log MX (t) = 0 −3/2 Xi i L(t/ n) L (t/ n)(− 2 tn ) dt MXi (t) lim = lim by L’Hospital’s rule n→∞ −1 n→∞ −2 0 n −n MX (0) µ √ L0 (0) = i = = 0 L0(t/ n)t Xi M (0) 1 = lim Xi n→∞ 2n−1/2 √ L00(t/ n)t(− 1 tn−3/2) = lim 2 by L’Hospital’s rule 0 00 0 2 n→∞ −3/2 d MX (t) MXi (t)MX (t) − [MX (t)] −n L00 (t) = i = i i Xi dt M (t) [M (t)]2 √ t2 Xi Xi = lim L00(t/ n) 00 0 2 n→∞ 2 MXi (0)MX (0) − [MX (0)] L00 (0) = i i 2 Xi 2 t [MX (0)] = i 2 0 2 2 E(X )E(X ) − E(Xi ) = i i = E(X 2) − E(X )2 = σ2 = 1 0 2 i i E(Xi )

Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 10 / 22 Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 11 / 22 CLT Moments of Distributions CLT Continuous Random Variables Proof of the CLT, Final Comments Cumulative Distribution Function

The preceding proof assumes that E(Xi ) = 0 and Var(Xi ) = 1. We have already seen a variety of problems where we ﬁnd P(X <= x) or P(X > x) etc. The former is given a special name - the cumulative We can generalize this result to any collection of random variables Yi by distribution function. considering the standardized form Y ∗ = (Y − µ)/σ. i i If X is discrete with probability mass function f (x) then

x X Y + ··· + Y − nµ Y − µ Y − µ √ P(X ≤ x) = F (x) = f (z) 1 √ n = 1 + ··· + n n σ n σ σ z=−∞ ∗ ∗ √ = (Y1 + ··· + Yn ) / n If X is continuous with probability density function f (x) then Z x P(X ≤ x) = F (x) = f (z)dz ∗ E(Yi ) = 0 −∞ ∗ Var(Yi ) = 1 CDF is deﬁned for for all −∞ < x < ∞ and follows the following rules:

lim F (x) = 0 lim F (x) = 1 x < y ⇒ F (x) < x→−∞ x→∞ F (y)

Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 12 / 22 Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 13 / 22

CLT Continuous Random Variables CLT Continuous Random Variables Binomial CDF Uniform CDF

Let X ∼ Binom(n, p) then Let X ∼ Unif(a, b) then

Probability Mass Function Cumulative Density Function Probability Mass Function Cumulative Density Function (  1 0 for x ≤ a ! b−a for x ∈ [a, b]  n bxc ! f (x) =  k n−k X n k n−k F (x) = x−a P(X = k) = f (k) = p (1 − p) P(X ≤ x) = F (x) = p (1 − p) 0 otherwise b−a for x ∈ [a, b] k k  k=0 1 for x ≥ b

Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 14 / 22 Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 15 / 22 CLT Continuous Random Variables CLT Continuous Random Variables Normal CDF Exponential Distribution

Let X ∼ N (µ, σ2) then In general terms, the Exponential distribution describes the time between events which occur continuously with a given rate λ (the expected number Probability Mass Function Cumulative Density Function of events in a given unit of time).

2 1 − (x−µ) f (x) = φ(x) = √ e 2σ2 F (x) = Φ(x) Let X ∼ Exp(λ), we deﬁne one unit of time as 1/λ which we can 2πσ sub-divide into n sub-intervals. The probability that an event occurs during a particular sub-interval is approximately λ/n.

The probability that we must wait b or fewer units of time between events is the same as the probability that an event does occur in one of the b · nth sub-intervals. Therefore, if we let Y ∼ Geo(λ/n) then

bn−1 bn−1 k X X λ λ P(X ≤ b) ≈ P(Y ≤ nb) = P(Y = k) = 1 − n n k=0 k=0

Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 16 / 22 Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 17 / 22

CLT Continuous Random Variables CLT Continuous Random Variables Exponential Distribution, cont. Exponential Distribution, cont.

From your calculus class remember that: F (b) = P(X ≤ b) = lim P(Y ≤ b) m m+1 n→∞ X k 1 − a a = λ nb 1 − a = lim 1 − 1 − k=0 n→∞ n Therefore λ nb = 1 − lim 1 − n→∞ n bn−1 k bn−1 k b X λ λ λ X λ h −λi −λb 1 − = 1 − = 1 − e = 1 − e n n n n k=0 k=0 In this case we have the CDF but not the PDF, how do we get the PDF? λ 1 − (1 − λ )nb = n n λ 1 − (1 − n ) λnb = 1 − 1 − n

Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 18 / 22 Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 19 / 22 CLT Continuous Random Variables CLT Continuous Random Variables Exponential Distribution, cont. Exponential Distribution - Memoryless Property

Let X be a random variable that reﬂects the time between events which Let X ∼ Exp(λ) (assume λ has units of events/min) then if we have occur continuously with a given rate λ, X ∼ Exp(λ) waited s minutes without observing an event what is the probability that an event occurs in the next t minutes? f (x|λ) = λe−λx P(X > s + t, X > s) P(X > s + t) −λx P(X > s + t|X > s) = = P(X ≤ x) = F (x|λ) = 1 − e P(X > s) P(X > s) 1 − P(X ≤ s + t) 1 − F (s + t) = = t −1 1 − P(X ≤ s) 1 − F (s) M (t) = 1 − X λ 1 − (1 − e−λ(s+t)) = 1 − (1 − e−λs ) E(X ) = λ−1 = e−λ(s+t)/e−λs = e−λt −λt Var(X ) = λ−2 = 1 − (1 − e ) = 1 − F (t) log 2 = P(X > t) Median(X ) = λ

Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 20 / 22 Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 21 / 22

CLT Continuous Random Variables Exponential Distribution - Example

Strontium 90 is a radioactive component of fallout from nuclear explosions. The halﬂife of Strontium 90 is 28 years and the decay of an individual atom can be modeled by an exponential random variable.

(a) What is the decay rate λ? (b) What is the average lifetime of a Strontium 90? (c) What is the probability that a Strontium 90 survives at least 50 years? (d) What is the probability that a Strontium 90 survives at least 75 years given it has survived at least 25 years?

Statistics 104 (Colin Rundel) Lecture 12 February 27, 2012 22 / 22