Section 4.6 Order Statistics Order Statistics Let X1; X2; X3; X4; X5 be iid random variables with a distribution F with a Lecture 15: Order Statistics range of (a; b). We can relabel these X's such that their labels correspond to arranging them in increasing order so that Statistics 104 X X X X X (1) ≤ (2) ≤ (3) ≤ (4) ≤ (5) Colin Rundel X(1) X(2) X(3) X(4) X(5) March 14, 2012 a X5 X1 X4 X2 X3 b In the case where the distribution F is continuous we can make the stronger statement that X(1) < X(2) < X(3) < X(4) < X(5) Since P(Xi = Xj ) = 0 for all i = j for continuous random variables. 6 Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 1 / 24 Section 4.6 Order Statistics Section 4.6 Order Statistics Order Statistics, cont. Notation Detour For X1; X2;:::; Xn iid random variables Xk is the kth smallest X , usually For a continuous random variable we can see that called the kth order statistic. f (x) ≈ P(x ≤ X ≤ x + ) = P(X 2 [x; x + ]) X(1) is therefore the smallest X and lim f (x) = lim P(X 2 [x; x + ]) !0 !0 f (x) = lim P(X 2 [x; x + ])/ X(1) = min(X1;:::; Xn) !0 Similarly, X(n) is the largest X and P (x X x + ✏)=P (X [x, x + ✏]) X(n) = max(X1;:::; Xn) 2 f(x) f(x + ✏) x x + ✏ Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 2 / 24 Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 3 / 24 Section 4.6 Order Statistics Section 4.6 Order Statistics Density of the maximum Density of the minimum For X1; X2;:::; Xn iid continuous random variables with pdf f and cdf F For X1; X2;:::; Xn iid continuous random variables with pdf f and cdf F the density of the maximum is the density of the minimum is P(X(n) 2 [x; x + ]) = P(one of the X 's 2 [x; x + ] and all others < x) P(X(1) 2 [x; x + ]) = P(one of the X 's 2 [x; x + ] and all others > x) n n X X = P(Xi 2 [x; x + ] and all others < x) = P(Xi 2 [x; x + ] and all others > x) i=1 i=1 = nP(X1 2 [x; x + ] and all others < x) = nP(X1 2 [x; x + ] and all others > x) = nP(X1 2 [x; x + ])P(all others < x) = nP(X1 2 [x; x + ])P(all others > x) = nP(X1 2 [x; x + ])P(X2 < x) ··· P(Xn < x) = nP(X1 2 [x; x + ])P(X2 > x) ··· P(Xn > x) = nf (x)F (x)n−1 = nf (x)(1 − F (x))n−1 n−1 n−1 f(n)(x) = nf (x)F (x) f(1)(x) = nf (x)(1 − F (x)) Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 4 / 24 Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 5 / 24 Section 4.6 Order Statistics Section 4.6 Order Statistics Density of the kth Order Statistic Cumulative Distribution of the min and max For X1; X2;:::; Xn iid continuous random variables with pdf f and cdf F For X1; X2;:::; Xn iid continuous random variables with pdf f and cdf F the density of the kth order statistic is the density of the kth order statistic is F(1)(x) = P(X(1) < x) = 1 − P(X(1) > x) P(X(k) 2 [x; x + ]) = P(one of the X 's 2 [x; x + ] and exactly k − 1 of the others < x) n = 1 − P(X1 > x;:::; Xn > x) = 1 − P(X1 > x) ··· P(Xn > x) X n = P(Xi 2 [x; x + ] and exactly k − 1 of the others < x) = 1 − (1 − F (x)) i=1 = nP(X1 2 [x; x + ] and exactly k − 1 of the others < x) F(n)(x) = P(X(n) < x) = 1 − P(X(n) > x) = nP(X1 2 [x; x + ])P(exactly k − 1 of the others < x) = P(X < x;:::; X < x) = P(X < x) ··· P(X < x) ! ! ! 1 n 1 n n − 1 n − 1 n = nP(X 2 [x; x + ]) P(X < x)k−1P(X > x)n−k = nf (x) F (x)k−=1(1F (−x)F (x))n−k 1 k − 1 k − 1 ! n − 1 d dF (x) f (x) = nf (x) F (x)k−1(1 − F (x))n−k f (x) = (1 − F (x))n = n(1 − F (x))n−1 = nf (x)(1 − F (x))n−1 (k) k − 1 (1) dx dx d dF (x) f (x) = F (x)n = nF (x)n−1 = nf (x)F (x)n−1 (n) dx dx Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 6 / 24 Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 7 / 24 Section 4.6 Order Statistics Section 4.6 Order Statistics Order Statistic of Standard Uniforms Beta Distribution iid The Beta distribution is a continuous distribution defined on the range Let X1; X2;:::; Xn Unif(0; 1) then the density of X(n) is given by ∼ (0; 1) where the density is given by ! n − 1 1 r−1 s−1 f (x) = nf (x) F (x)k−1(1 − F (x))n−k f (x) = x (1 − x) (k) k − 1 B(r; s) ( nn−1x k−1(1 − x)n−k if 0 < x < 1 where B(r; s) is called the Beta function and it is a normalizing constant = k−1 0 otherwise which ensures the density integrates to 1. Z 1 This is an example of the Beta distribution where r = k and s = n k + 1. 1 = f (x)dx − 0 Z 1 1 1 = x r−1(1 − x)s−1dx B(r; s) X(k) Beta(k; n k + 1) 0 ∼ − 1 Z 1 1 = x r−1(1 − x)s−1dx B(r; s) 0 Z 1 B(r; s) = x r−1(1 − x)s−1dx 0 Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 8 / 24 Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 9 / 24 Section 4.6 Order Statistics Section 4.6 Order Statistics Beta Function Beta Function - Expectation The connection between the Beta distribution and the kth order statistic Let X Beta(r; s) then of n standard Uniform random variables allows us to simplify the Beta ∼ function. Z 1 1 E(X ) = x x r−1(1 − x)s−1dx 0 B(r; s) Z 1 1 Z r−1 s−1 = ; 1x (r+1)−1(1 − x)s−1dx B(r; s) = x (1 − x) dx B(r; s) 0 0 1 B(r + 1; s) B(k; n − k + 1) = = n−1 B(r; s) n k−1 (k − 1)!(n − 1 − k + 1)! r!(s − 1)! (r + s − 1)! = = n(n − 1)! (r + s)! (r − 1)!(s − 1)! (r − 1)!(n − k)! r! (r + s − 1)! = = n! (r − 1)! (r + s)! r (r − 1)!(s − 1)! Γ(r)Γ(s) = = = r + s (r + s − 1)! Γ(r + s) Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 10 / 24 Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 11 / 24 Section 4.6 Order Statistics Section 4.6 Order Statistics Beta Function - Variance Beta Distribution - Summary Let X Beta(r; s) then If X Beta(r; s) then ∼ ∼ Z 1 1 1 E(X 2) = x2 xr−1(1 − x)s−1dx f (x) = x r−1(1 − x)s−1 0 B(r; s) B(r; s) B(r + 2; s) (r + 1)!(s − 1)! (r + s − 1)! Z x 1 B (r; s) = = F (x) = x r−1(1 − x)s−1dx = x B(r; s) (r + s + 1)! (r − 1)!(s − 1)! 0 B(r; s) B(r; s) (r + 1)r = (r + s + 1)(r + s) Z 1 (r − 1)!(s − 1)! Γ(r)Γ(s) B(r; s) = x r−1(1 − x)s−1dx = = (r + s − 1)! Γ(r + s) 2 2 0 Var(X ) = E(X ) − E(X ) Z x r−1 s−1 (r + 1)r r 2 Bx (r; s) = x (1 − x) dx = − 0 (r + s + 1)(r + s) (r + s)2 (r + 1)r(r + s) − r 2(r + s + 1) = r 2 E(X ) = (r + s + 1)(r + s) r + s rs = rs 2 Var(X ) = (r + s + 1)(r + s) (r + s)2(r + s + 1) Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 12 / 24 Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 13 / 24 Section 4.6 Order Statistics Section 4.6 Order Statistics Minimum of Exponentials Maximum of Exponentials iid iid Let X1; X2;:::; Xn Exp(λ), we previously derived a more general result Let X1; X2;:::; Xn Exp(λ) then the density of X(n) is given by where the X 's were∼ not identically distributed and showed that ∼ min(X1;:::; Xn) Exp(λ1 + + λn) = Exp(nλ) in this more restricted n−1 ∼ ··· f(n)(x) = nf (x)F (x) case. n−1 = n λe−λx 1 − e−λx Lets confirm that result using our new more general methods n−1 f(1)(x) = nf (x)(1 − F (x)) n−1 Which we can't do much with, instead we can try the cdf of the maximum. = n λe−λx 1 − [1 − e−λx ] n−1 = nλe−λx e−λx ] n = nλ e−λx ] = nλe−nλx Which is the density for Exp(nλ). Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 14 / 24 Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 15 / 24 Section 4.6 Order Statistics Section 4.6 Order Statistics Maximum of Exponentials, cont.