sign in sign up
<<
Home , Normal distribution

Distributions Derived from Normal Random Variables

Distributions Derived From the

MIT 18.443

Dr. Kempthorne

Spring 2015

MIT 18.443 Distributions Derived From the Normal Distribution 1 χ2 , t, and F Distributions Distributions Derived from Normal Random Variables from Normal Samples Outline

1 Distributions Derived from Normal Random Variables χ2 , t, and F Distributions Statistics from Normal Samples

MIT 18.443 Distributions Derived From the Normal Distribution 2 χ2 , t, and F Distributions Distributions Derived from Normal Random Variables Statistics from Normal Samples Normal Distribution

Definition. A Normal / Gaussian random X ∼ N(µ, σ2) has density : 1 2 2 f (x) = √ e −(x−µ) /2σ , −∞ < x < +∞. σ 2π with mean and variance parameters: J +∞ µ = E [X ] = −∞ xf (x)dx 2 2 J +∞ 2 σ = E [(X − µ) ] = −∞ (x − µ) f (x)dx Note: −∞ < µ < +∞, and σ2 > 0. Properties: Density function is symmetric about x = µ. f (µ + x ∗) = f (µ − x ∗). f (x) is a maximum at x = µ. f ""(x) = 0 at x = µ + σ and x = µ − σ (inflection points of bell curve) generating function: tX µt+σ2t2/2 MX (t) = E [e ] = e MIT 18.443 Distributions Derived From the Normal Distribution 3 χ2 , t, and F Distributions Distributions Derived from Normal Random Variables Statistics from Normal Samples Chi-Square Distributions

Definition. If Z ∼ N(0, 1) (Standard Normal .v.) then 2 2 U = Z ∼ χ1, has a Chi-Squared distribution with 1 degree of freedom.

Properties: The density function of U is: −1/2 f (u) = u√ e−u/2, 0 < u < ∞ U 2π Recall the density of a Gamma(α, λ) distribution: λα α−1 −λx g(x) = Γ(α) x e , x > 0, So U is Gamma(α, λ) with α = 1/2 and λ = 1/2. Moment generating function tU −α −1/2 MU (t) = E [e ] = [1 − t/λ] = (1 − 2t)

MIT 18.443 Distributions Derived From the Normal Distribution 4 χ2 , t, and F Distributions Distributions Derived from Normal Random Variables Statistics from Normal Samples Chi-Square Distributions

Definition. If Z1, Z2,..., Zn are i.i.d. N(0, 1) random variables 2 2 2 V = Z1 + Z2 + ... Z n has a Chi-Squared distribution with n degrees of freedom. Properties (continued) The Chi-Square r.v. V can be expressed as: V = U1 + U2 + ··· + Un 2 where U1,..., Un are i.i.d χ1 r.v. Moment generating function tV t(U1+U2+···+Un) MV (t) = E [e ] = E [e ] = E[etU1 ] ··· E [etUn ] = (1 − 2t)−n/2 Because Ui are i.i.d. Gamma(α = 1/2, λ = 1/2) r.v.,s V ∼ Gamma(α = n/2, λ = 1/2). 1 Density function: f (v) = v(n/2)−1 e −v /2, v > 0. 2n/2Γ(n/2) (α is the shape and λ is the )

MIT 18.443 Distributions Derived From the Normal Distribution 5 χ2 , t, and F Distributions Distributions Derived from Normal Random Variables Statistics from Normal Samples Student’s t Distribution

Definition. For independent r.v.’s Z and U where Z ∼ N(0, 1) 2 U ∼ χr the distribution of T = Z / U/r is the t distribution with r degrees of freedom.

Properties The density function of T is Γ[(r + 1)/2] t2 −(r+1)/2 f (t) = √ 1 + , −∞ < t < +∞ rπΓ(r/2) r For what powers k does E [T k ] converge/diverge? Does the moment generating function for T exist?

MIT 18.443 Distributions Derived From the Normal Distribution 6 χ2 , t, and F Distributions Distributions Derived from Normal Random Variables Statistics from Normal Samples F Distribution

Definition. For independent r.v.’s U and V where 2 U ∼ χm 2 V ∼ χn U/m the distribution of F = is the V /n F distribution with m and n degrees of freedom. (notation F ∼ Fm,n) Properties The density function of F is Γ[(m + n)/2] m n/2 m −(m+n)/2 f (w) = w m/2−1 1 + w , Γ(m/2)Γ(n/2) n n with domain w > 0. 1 n E [F ] = E [U/m] × E [n/V ] = 1 × n × n−2 = n−2 (for n > 2). 2 If T ∼ tr , then T ∼ F1,r .

MIT 18.443 Distributions Derived From the Normal Distribution 7 χ2 , t, and F Distributions Distributions Derived from Normal Random Variables Statistics from Normal Samples Outline

1 Distributions Derived from Normal Random Variables χ2 , t, and F Distributions Statistics from Normal Samples

MIT 18.443 Distributions Derived From the Normal Distribution 8 χ2 , t, and F Distributions Distributions Derived from Normal Random Variables Statistics from Normal Samples Statistics from Normal Samples

Sample of size n from a Normal Distribution 2 X1,..., Xn iid N(µ, σ ). n 1 n Sample : X = X n i i=1 n 1 n Sample : S2 = (X − X )2 n − 1 i i=1 Properties of X The moment generating function of X is t n tX n 1 Xi MX (t) = E[e ] = E [e ] 2 n n σ 2 µ(t/n)+ 2 (t/n) = 1 MXi (t/n) =i=1 [e ] 2 µt+ σ /n t2 = e 2 i.e., X ∼ N(µ, σ2/n).

MIT 18.443 Distributions Derived From the Normal Distribution 9 χ2 , t, and F Distributions Distributions Derived from Normal Random Variables Statistics from Normal Samples Independence of X and S 2 .

Theorem 6.3.A The X and the vector of random variables (X1 − X , X2 − X ,..., Xn − X ) are independent. Proof:

Note that X and each of Xi − X are linear combinations of X1,..., Xn, i.e., Ln T X = 1 ai Xi = a X = U and Ln (k) (k) T Xk − X = 1 bi Xi = (b ) X = Vk where X = (X1,..., Xn) a = (a1,..., an) = (1/n,..., 1/n) (k) (k)(k) b = (b1 ,..., bn ) . 1 (k) 1 − n , for i = k with bi = 1 − n , for i = k U and V1,..., Vn are jointly normal r.v.s U is uncorrelated (independent) of each Vk

MIT 18.443 Distributions Derived From the Normal Distribution 10 χ2 , t, and F Distributions Distributions Derived from Normal Random Variables Statistics from Normal Samples Independence of U = aT X and V = bT X where X = (X1,..., Xn) a = (a1,..., an) = (1/n,..., 1/n) b = (b1,..., bn) . 1 (k) 1 − n , for i = k with bi = bi = 1 (so V = Vk ) − n , for i =6 k Proof: Joint MGF of (U, V ) factors into MU (s) × MV (t) sU+tV saT X+tbT X MU,V (s, t) = E [e ] = E [e ] s[ PPa X ]+t[ b X ] P [(sa +tb )X ] = E [e i i i i i i ] = E [e i i i i ] P ∗ i [ti Xi ] ∗ = E [e ] with t i = (sai + tbi ) n n ∗ n Q [ti Xi ] ∗ = i=1 E [e ] = MXi (ti ) i=1 2 2 n ∗ σ ∗ 2 Pn ∗ σ Pn ∗ 2 Q ti µ+ (ti ) µ( i=1 ti )+ i=1[(ti ) ] = i=1 e 2 = e 2 µ(s×1+t×0)+σ2/2[ Pn [s2 a 2 +t2b2+2sta b ] = e i=1 i i i i sµ+(σ2/n)s2/2 t×0+t2σ2/2 Pn b2 = [e ] × [e i=1 i ] 2 2 Ln 2 = [mgf of N(µ, σ /n)] × [mgf of N(0, σ · i=1 bi ]

MIT 18.443 Distributions Derived From the Normal Distribution 11 χ2 , t, and F Distributions Distributions Derived from Normal Random Variables Statistics from Normal Samples Independence of X and S 2

Proof (continued):

Independence of U = X and each Vk = Xk − X gives 2 Ln 2 X and S = 1 V i are independent. Random variables/vectors are independent if their joint moment generating function is the product of their individual moment generating functions: sU+t1V1+···tnVn MU,V1,...,Vn (s, t1,..., tn) = E [e ] = MU (s) × M(t1,..., tn). Theorem 6.3.B The distribution of (n − 1)S2/σ2 is the Chi-Square distribution with (n − 1) degrees of freedom. Proof:  2 1 Ln 2 Ln Xi −µ Ln 2 2 σ2 1(Xk − µ) = i=1 σ = i=1 Zi ∼ χn. 1 Ln 2 1 Ln � �2 σ2 1(Xk − µ) = σ2 i=1 Xi − X + X − µ 1 L n 2 n � �2 =σ2 i=1 (Xi − X ) + σ2 X − µ

MIT 18.443 Distributions Derived From the Normal Distribution 12 χ2 , t, and F Distributions Distributions Derived from Normal Random Variables Statistics from Normal Samples Proof (continued):

Proof: 1 Ln 2 1 Ln 2 n �� 2 σ2 1(Xk − µ) = σ2 i=1(Xi − X ) + σ2 X − µ 2 2 2 2 χn = [distribution of (nS /σ )] + χ1 By independence: 2 2 2 2 mgf of χn = mgf of [distribution of (nS /σ )] × mgf of χ1 that is −n/2 −1/2 (1 − 2t) = MnS2/σ2 (t) × (1 − 2t) So −(n−1)/2 MnS2/σ2 (t) = (1 − 2t) ,

2 2 2 =⇒ nS /σ ∼ χn−1. Corollary 6.3.B For a X and S2 from a Normal sample of size n, X − µ T = √ ∼ t S/ n n−1

MIT 18.443 Distributions Derived From the Normal Distribution 13 MIT OpenCourseWare http://ocw.mit.edu

18.443 Statistics for Applications Spring 2015

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.