Lecture 14: Order Statistics; Conditional Expectation
Total Page:16
File Type:pdf, Size:1020Kb
Department of Mathematics Ma 3/103 KC Border Introduction to Probability and Statistics Winter 2017 Lecture 14: Order Statistics; Conditional Expectation Relevant textbook passages: Pitman [5]: Section 4.6 Larsen–Marx [2]: Section 3.11 14.1 Order statistics Pitman [5]: Given a random vector (X(1,...,Xn) on the probability) space (S, E,P ), for each s ∈ S, sort the § 4.6 components into a vector X(1)(s),...,X(n)(s) satisfying X(1)(s) 6 X(2)(s) 6 ··· 6 X(n)(s). The vector (X(1),...,X(n)) is called the vector of order statistics of (X1,...,Xn). Equivalently, { } X(k) = min max{Xj : j ∈ J} : J ⊂ {1, . , n} & |J| = k . Order statistics play an important role in the study of auctions, among other things. 14.2 Marginal Distribution of Order Statistics From here on, we shall assume that the original random variables X1,...,Xn are independent and identically distributed. Let X1,...,Xn be independent and identically distributed random variables with common cumulative distribution function F ,( and let (X) (1),...,X(n)) be the vector of order statistics of X1,...,Xn. By breaking the event X(k) 6 x into simple disjoint subevents, we get ( ) X(k) 6 x = ( ) X(n) 6 x ∪ ( ) X(n) > x, X(n−1) 6 x . ∪ ( ) X(n) > x, . , X(j+1) > x, X(j) 6 x . ∪ ( ) X(n) > x, . , X(k+1) > x, X(k) 6 x . 14–1 Ma 3/103 Winter 2017 KC Border Order Statistics; Conditional Expectation 14–2 Each of these subevents is disjoint from the ones above it, and each has a binomial probability: ( ) X(n) > x, . , X(j+1) > x, X(j) 6 x = (n − j of the random variables are > x and j are 6 x) , so ( ) ( ) n ( )n−j P X > x, . , X > x, X 6 x = 1 − F (x) F (x)j. (n) (j+1) (j) j Thus: The cdf of the kth order statistic from a sample of n is: ( ) ∑n ( ) n ( )n−j F (x) = P X 6 x = 1 − F (x) F (x)j. (1) (k,n) (k) j j=k 14.3 Marginal Density of Order Statistics Pitman [5]: ′ p. 326 If F has a density f = F , then we can calculate the marginal density of X(k) by differentiating the CDF F(k,n): d F(k,n)(x) dx ( ) ∑n d n ( )n−j = F (x)j 1 − F (x) dx j j=k ( ) ∑n n d ( )n−j = F (x)j 1 − F (x) j dx j=k ( ) ∑n ( ) n ( )n−j ( )n−j−1 = jF (x)j−1 1 − F (x) F ′(x) − (n − j)F (x)j 1 − F (x) F ′(x) j j=k ( ) ∑n ( ) n ( )n−j ( )n−j−1 = jF (x)j−1 1 − F (x) − (n − j)F (x)j 1 − F (x) f(x) j j=k ∑n n! ( )n−j = F (x)j−1 1 − F (x) f(x) (j − 1)!(n − j)! j=k n∑−1 n! ( )n−j−1 − 1 − F (x) F (x)jf(x) j!(n − j − 1)! j=k n! ( )n−j = 1 − F (x) F (x)k−1f(x) (k − 1)!(n − k)! ∑n n! ( )n−j + 1 − F (x) F (x)j−1f(x) (j − 1)!(n − j)! j=k+1 n∑−1 n! ( )n−j−1 − 1 − F (x) F (x)jf(t). j!(n − j − 1)! j=k v. 2017.02.08::13.25 KC Border Ma 3/103 Winter 2017 KC Border Order Statistics; Conditional Expectation 14–3 The last two terms above cancel, since using the change of variables i = j − 1, ∑n n! ( )n−j 1 − F (x) F (x)j−1 (j − 1)!(n − j)! j=k+1 n∑−1 n! ( )n−i = 1 − F (x) F (x)i. i!(n − i − 1)! i=k So the density of the kth order statistic from a sample of size n is: ( ) n! − n−k k−1 f(k,n)(x) = − − 1 F (x) F (x) f(x) (k 1)!(n k)! ( ) n − 1 ( )(n−1)−(k−1) = n 1 − F (x) F (x)k−1f(x). (2) k − 1 Note: I could have written n − k instead of (n − 1) − (k − 1), but then the binomial coefficient would have seemed more mysterious. 14.4 Some special order statistics st The 1 order statistic X(1) from a sample of size n is just the minimum X(1) = min{X1,...,Xn}. The event that min{X1,...,Xn} 6 x is just the event that at least one Xi 6 x. The complement ( )n of this event is that all Xi > x, which has probability 1 − F (x) , so the cdf is ( )n F(1,n)(x) = 1 − 1 − F (x) and its density is ( )(n−1) f(1,n)(x) = n 1 − F (x) f(x). th The n order statistic X(n) from a sample of size n is just the maximum X(n) = max{X1,...,Xn}. The event that max{X1,...,Xn} 6 x is just the event that all Xi 6 x, so its cdf is n F(n,n)(x) = F (x) and its density is (n−1) f(n,n)(x) = nF (x) f(x). The (n − 1)st order statistic is the second-highest value. Its cdf is ( ) n−2 n n−1 n F(n−1,n)(x) = n 1 − F (x) F (x) + F (x) = nF (x) − (n − 1)F (x) , and its density is ( ) n(n − 1) 1 − F (x) F (x)n−2f(x). In the study of auctions, the second-highest bid is of special interest. Indeed a second- price auction awards the item to the highest bidder, but the price is the second-highest bid. A standard auction provides incentives for bidders to bid less than their values, so the distribution of bids and the distribution of values is not the same. Nevertheless it can be shown (see my on- line notes) that the expected revenue to a seller in an auction with n bidders with independent and identically distributed values is just the expectation of the second-highest order statistic for the distribution of values. KC Border v. 2017.02.08::13.25 Ma 3/103 Winter 2017 KC Border Order Statistics; Conditional Expectation 14–4 14.5 Uniform order statistics and the Beta function Pitman [5]: For a Uniform[0,1] distribution, F (t) = t and f(t) = 1 on [0, 1]. In this case equation (2) tells pp. 327–328 us: th The density f(k,n) of the k order statistic for n independent Uniform[0, 1] random variables is ( ) n − 1 f (t) = n (1 − t)n−ktk−1. (k,n) k − 1 Since f(k,n) is a density, ∫ ( ) ∫ 1 − 1 n 1 − n−k k−1 f(k,n)(t) dt = n − (1 t) t = 1, 0 k 1 0 or ∫ 1 1 (k − 1)!(n − k)! (1 − t)n−ktk−1 = ( ) = . (3) n−1 n! 0 n k−1 Now change variables by setting r = k and s = n − r + 1 (so s − 1 = n − r and n = r + s − 1). Then rewrite (3) as ∫ 1 − − − s−1 r−1 (r 1)!(s 1)! Γ(s)Γ(r) (1 t) t = − = . 0 (s + r 1)! Γ(r + s) Recall that the Gamma function is a continuous version of the factorial, and has the property that Γ(s + 1) = sΓ(s) for every s > 0, and Γ(m) = (m − 1)! for every natural number m. See Definition 13.1.1. This fact suggests (to at least some people) the following definition: 14.5.1 Definition The Beta function is defined for r, s > 0 (not necessarily integers), by ∫ 1 Γ(s)Γ(r) B(r, s) = tr−1(1 − t)s−1 dt = . 0 Γ(r + s) So for integers r and s, we see that Γ(s + 1)Γ(r + 1) (s)!(r)! (s)!(r)! 1 B(r + 1, s + 1) = = = (r + s) = (r + s)( ). Γ(r + s + 2) (r + s − 1)! (r + s)! r+s r 14.5.2 Definition The beta(r, s) distribution has the density 1 f(x) = xr−1(1 − x)s−1 B(r, s) on the interval [0, 1] and zero elsewhere. v. 2017.02.08::13.25 KC Border Ma 3/103 Winter 2017 KC Border Order Statistics; Conditional Expectation 14–5 Note that for integer rand s, the density of the beta(r + 1, s + 1) distribution is ( ) 1 r + s f(x) = xr(1 − x)s, r + s r which is 1/(r + s) times the Binomial probability of r successes and s failures in r + s trials, where the probability of success is x. The mean of a beta(r, s) distribution is r . r + s Proof : ∫ ∫ 1 1 1 xf(x) dx = x xr−1(1 − x)s−1 dx B(r, s) 0 ∫0 1 1 = xr+1−1(1 − x)s−1 B(r, s) 0 B(r + 1, s) = B(r, s) Γ(s)Γ(r + 1) Γ(r + s) = Γ(r + 1 + s) Γ(s)Γ(r) Γ(s)rΓ(r) Γ(r + s) = (r + s)Γ(r + s) Γ(s)Γ(r) r = . r + s Thus for a Uniform[0,1] distribution, the (k, n) order statistic has a beta(k, n − k + 1) distribution and so has mean k . n + 1 [Application to breaking a unit length bar into n + 1 pieces by choosing n breaking points. The expectation of the kth breaking point is at k/(n + 1), so each piece has expected length 1/n.] 14.6 The war of attrition In the 1970s, the ethologist John Maynard Smith [3] began to apply game theory to problems of animal behavior.