INTERVAL ESTIMATION AND HYPOTHESES TESTING 1. BRIEF REVIEW OF SOME COMMON DISTRIBUTIONS 1.1. Probability Distributions. Table 1 contains information on some common probability distri- butions. TABLE 1. Probability Distributions Distribution pdf Mean Variance mgf Notes 2 2 2 µ =0,σ =1 −1 x−µ 2 t σ −1 2 µt+ 2 1 x Normal, N(µ, σ2) √ 1 · e 2 σ µ σ2 e 2 f(x;µ,σ )= √ ·e 2 2πσ2 2π t2 MX (t)=e 2 1 −x − · θ 2 − 1 Exponential θ e ,θ>0 θ θ (1 θt) Xi ∼N(0, 1),i=1, 2, ..., n n 2 2 ⇒ X Xi ∼ χ (n) i=1 2 − − − Xi ∼N (µ, σ ),i=1, 2, ..., n 1 ν 2 x ν χ2(ν) x 2 e 2 ν 2ν (1 − 2t) 2 Chi-square, ν Γ v n 2 2 2 ( 2 ) X − µ ⇒ X i ∼ χ2(n) σ i=1 n X − X¯ 2 ⇒ X i ∼ χ2(n − 1) σ i=1 −x 1 α−1 β 2 − −α Gamma βαΓ(α) x e αβ αβ (1 βt) N(0,1) ν +1 −(ν +1) t(ν)= Γ 2 2 r 2 √ 2 x ν χ (ν) t(ν) ν 1+ ν 0 ν − 2 ν πν Γ( 2 ) N(0,1) and χ2(ν) are independent ν +ν Γ 1 2 ν1 ν χ2(ν ) 2 ν 1 −1 1 1 · 1 2 · 2 χ2 (ν ) ν1 ν2 ν x 2 ν1 ν2 1 1 Γ Γ 2 ν 2ν (ν +ν −2) F (ν1 ,ν2 )= = · ( 2 ) ( 2 ) 2 2 1 2 χ2(ν ) ν1 χ2 (ν ) F(ν1 ,ν2) 2 2 2 2 2 ν2 −2 ν1 (ν2−2) (ν2−4) −(ν1 +ν2) ν2 ν1 2 2 ×1+ x 2 χ (ν1 ) and χ (ν2 ) are independent ν2 1 1 1.2. Moments. 1.2.1. Population raw moments. ∞ 0 r Z r µr = E(X )= x f(x) dx −∞ ∞ (1) 0 Z µ1 = E(X)= xf(x) dx −∞ Date: December 8, 2005. 1 2 INTERVAL ESTIMATION AND HYPOTHESES TESTING 1.2.2. Population central moments. ∞ r Z r µr = E[(X − µX ) ]= (x − µX ) f(x) dx −∞ ∞ Z µ1 = E[X − µX ]= (x − µX ) f(x) dx (2) −∞ ∞ 2 Z 2 2 µ2 = E[(X − µX ) ]= (x − µX ) f(x) dx = Var(X)=σ −∞ 1.2.3. Sample raw moments. 1 n X¯ r = X Xr n n i i=1 (3) 1 n X¯ = X X n i i=1 1.2.4. Sample central moments. n r 1 0 r C = X Xi − µ r =1, 2, 3,..., n n i,1 i=1 n 1 1 0 C = X Xi − µ (4) n n i,1 i=1 n 2 1 0 2 C = X Xi − µ n n i,1 i=1 1.2.5. Sample moments about the average (sample mean). n r 1 ¯ r M = X Xi − Xn ,r=1, 2, 3,..., n n i=1 n 1 ¯ Mn = X Xi − Xn =0 (5) n i=1 n 2 1 ¯ 2 2 M = X Xi − Xn =ˆσ n n i=1 1.3. Properties of various sample moments and functions thereof. Consider a sample X1,X2, ... 2 ,Xn where the Xi are identically and independently distributed with mean µ and variance σ . Now consider a sample X1,X2, ... ,Xn where the Xi are identically and independently dis- tributed normal random variables with mean µ and variance σ2. 2. THE BASIC IDEA OF INTERVAL ESTIMATES An interval rather than a point estimate is often of interest. Confidence intervals are thus impor- tant in empirical work. To construct interval estimates, standardized normal random variables are often used. INTERVAL ESTIMATION AND HYPOTHESES TESTING 3 TABLE 2. Properties of sample moments Moment Expected Value Variance ¯ 1 n σ2 X = n Pi=1 Xi µ n 1 1 n − σ2 Cn = n Pi=1 (Xi µ) 0 n 2 E(X−µ)4 −[E(X−µ)2 ] 4 2 1 n − 2 2 1 − 2 µ4−σ Cn = n Pi=1 (Xi µ) σ n Var(X µ) = n = n n 2 − 2 − − 4 µ −µ2 2(µ −2µ2) µ −3µ2 2 1 − ¯ n−1 2 (n 1) µ4 − (n 1)(n 3)σ 4 2 − 4 2 4 2 Mn = n Pi=1 Xi X n σ n3 n3 = n n2 + n3 2 4 2 1 n − ¯ 2 µ4 − (n−3)σ S = n−1 Pi=1 Xi X σ n n(n − 1) TABLE 3. Properties of sample moments of a normal distribution Moment Value Expected Value Variance µ1 = E(X − µ) 0 - - 2 2 µ2 = E(X − µ) σ - - 3 µ3 = E(X − µ) 0 - - 4 4 µ4 = E(X − µ) 3σ - - ¯ 1 n σ2 X = n Pi=1 Xi - µ n 2 1 n − ¯ 2 2 2σ4 S = n−1 Pi=1 Xi X - σ n−1 4 2 2 1 n − ¯ 2 n−1 2 2σ (n−1) Mn =ˆσ = n Pi=1 Xi X - n σ n2 3. STANDARDIZED NORMAL VARIABLES AND CONFIDENCE INTERVALS FOR THE MEAN WITH σ KNOWN 3.1. Form of the confidence interval. If X is a normal random variable with mean µ and variance σ2 then X − µ Z = (6) σ is a standard normal variable with mean zero and variance one. An estimate of µ, µˆ is given by the sample mean X¯. This then implies 1 n σ2 X¯ = X X = X¯ ∼ N µ, n i n i=1 (7) X¯ − µ z = ∼ N(0, 1) h √σ i n So if γ1 is the upper α/2 percent critical value of a standard normal variable, i.e. 4 INTERVAL ESTIMATION AND HYPOTHESES TESTING ∞ Z 1 − 1 z2 α √ e 2 dz = then γ1 2π 2 " x¯ − µ # 1 − α = F (γ1) − F (−γ1)=Pr −γ1 ≤ ≤ γ1 √σ n σ σ =Pr−γ √ ≤ x¯ − µ ≤ γ √ 1 n 1 n (8) σ σ =Prγ √ ≥−x¯ + µ ≥−γ √ 1 n 1 n σ σ =Prx¯ − γ √ ≤ µ ≤ x¯ + γ √ 1 n 1 n Therefore, with σ known, σ σ x¯ − γ √ ≤ µ ≤ x¯ + γ √ (9) 1 n 1 n is said to be the (1 − α) 100% confidence interval for µ. 3.2. Example confidence interval for the mean of a normal distribution with σ2 known. Consider the income data for carpenters and house painters in table 4 where the data for both groups of individuals is distributed normally. Further assume that σ2 is known and equal to $600,000. TABLE 4. Income Data for Carpenters and House Painters carpenters painters sample size nc =12 np =15 mean income c¯ = $6000 p¯ = $5400 2 2 estimated variance sc = $565 000 sp = $362 500 Consider a 95% confidence interval for the mean of the distribution which we will denote by µc. α For the normal distribution with 2 = 0.025, γ1 is equal to 1.96. We then have √ √ " 600 000 600 000 # 1 − α =Pr 6000 − √ (1.96) ≤ µc ≤ 6000 + √ (1.96) 12 12 (10) = Pr [6000 − (223.607)(1.96) ≤ µx ≤ 6000 + (223.607)(1.96)] = Pr [5561.73 ≤ µx ≤ 6438.27] 4. CONFIDENCE INTERVALS FOR THE MEAN WITH σ UNKNOWN 4.1. Form of the confidence interval. The previous section gave an interval estimate for the mean of a population when σ was known. When σ is unknown, another method must be used. Recall from the section 1 on probability distributions that the t random variable is defined as INTERVAL ESTIMATION AND HYPOTHESES TESTING 5 z t = (11) q χ2(ν) ν where z is a standard normal and χ2(ν) is a χ2 random variable with ν degrees of freedom and z and χ2(ν) are independent. 2 Also note from the same section 1 that if Xi ∼ N(µ, σ ) then n X − µ 2 X i ∼ χ2(n) σ i=1 (12) n ¯ 2 Xi − X 2 and X ∼ χ (n − 1) σ i=1 If Y ∼ N(β, σ2) and Y¯ = βˆ, then we would have n ˆ 2 Yi − β ! X ∼ χ2(n − 1) (13) σ i=1 (n−1)S2 2 We can use the information in equation 12 to find the distribution of σ2 where S is equal to 1 n S2 = X(X − X¯)2 (14) n − 1 i i=1 2 2 (n−1)S 2 Now substitute for S in σ2 where S and simplify as follows. (n − 1)S2 (n − 1) 1 n = · X(X − X¯)2 σ2 σ2 n − 1 i i=1 1 n = · X(X − X¯)2 σ2 i (15) i=1 n X − X¯ 2 = X i ∼ χ2(n − 1) σ i=1 The last line then indicates that (n − 1)S2 ∼ χ2(n − 1) (16) σ2 X¯ −µ (n−1)S2 We can show that in equation 7 and 2 in equation 16 are independent so that h √σ i σ n 6 INTERVAL ESTIMATION AND HYPOTHESES TESTING X¯ − µ X¯ − µ σ σ √ √ n n = 1 √ s(n − 1)S2 S2 σ σ2(n − 1) (17) √ X¯ − µ n X¯ − µ = = ∼ t(n − 1) S √ S n If γ1 is the upper α/2 percent critical value of a t random variable then ν +1 (ν+1) ∞ Γ 2 − Z 2 t 2 α 1+ dt = (18) √ ν ν 2 γ1 πν Γ 2 and X¯ − µ σ √ n 1 − α = F (γ1) − F (−γ1)=Pr−γ1 ≤ ≤ γ1 2 s(n − 1)S σ2(n − 1) √ n(X¯ − µ) (19) =Pr−γ ≤ ≤ γ 1 S 1 −γ S γ S =Pr √1 ≤ X¯ − µ ≤ √1 n n γ S γ S =PrX¯ − √1 ≤ µ ≤ X¯ + √1 n n This is referred to as (1−α)(100%) confidence interval for µ.