Minimal Sufficient Statistics Ancillary Statistics Location-scale Family Summary . Biostatistics 602 - Statistical Inference Lecture 04 Ancillary Statistics . Hyun Min Kang January 22th, 2013 . Hyun Min Kang Biostatistics 602 - Lecture 04 January 22th, 2013 1 / 23 2. What are examples obvious sufficient statistics for any distribution? 3. What is a minimal sufficient statistic? 4. Is a minimal sufficient statistic unique? 5. How can we show that a statistic is minimal sufficient for θ? Minimal Sufficient Statistics Ancillary Statistics Location-scale Family Summary . Recap from last lecture 1. Is a sufficient statistic unique? . Hyun Min Kang Biostatistics 602 - Lecture 04 January 22th, 2013 2 / 23 3. What is a minimal sufficient statistic? 4. Is a minimal sufficient statistic unique? 5. How can we show that a statistic is minimal sufficient for θ? Minimal Sufficient Statistics Ancillary Statistics Location-scale Family Summary . Recap from last lecture 1. Is a sufficient statistic unique? 2. What are examples obvious sufficient statistics for any distribution? . Hyun Min Kang Biostatistics 602 - Lecture 04 January 22th, 2013 2 / 23 4. Is a minimal sufficient statistic unique? 5. How can we show that a statistic is minimal sufficient for θ? Minimal Sufficient Statistics Ancillary Statistics Location-scale Family Summary . Recap from last lecture 1. Is a sufficient statistic unique? 2. What are examples obvious sufficient statistics for any distribution? 3. What is a minimal sufficient statistic? . Hyun Min Kang Biostatistics 602 - Lecture 04 January 22th, 2013 2 / 23 5. How can we show that a statistic is minimal sufficient for θ? Minimal Sufficient Statistics Ancillary Statistics Location-scale Family Summary . Recap from last lecture 1. Is a sufficient statistic unique? 2. What are examples obvious sufficient statistics for any distribution? 3. What is a minimal sufficient statistic? 4. Is a minimal sufficient statistic unique? . Hyun Min Kang Biostatistics 602 - Lecture 04 January 22th, 2013 2 / 23 Minimal Sufficient Statistics Ancillary Statistics Location-scale Family Summary . Recap from last lecture 1. Is a sufficient statistic unique? 2. What are examples obvious sufficient statistics for any distribution? 3. What is a minimal sufficient statistic? 4. Is a minimal sufficient statistic unique? 5. How can we show that a statistic is minimal sufficient for θ? . Hyun Min Kang Biostatistics 602 - Lecture 04 January 22th, 2013 2 / 23 Minimal Sufficient Statistics Ancillary Statistics Location-scale Family Summary . Minimal Sufficient Statistic . Definition 6.2.11 . A sufficient statistic T(X) is called a minimal sufficient statistic if, for any 0 0 .other sufficient statistic T (X), T(X) is a function of T (X). Why is this called ”minimal” sufficient statistic? . • The sample space X consists of every possible sample - finest partition • Given T(X), X can be partitioned into At where t 2 T = ft : t = T(X) for some x 2 X g • Maximum data reduction is achieved when jT j is minimal. • If size of T 0 = t : t = T0(x) for some x 2 X is not less than jT j, then jT j can be called as a minimal sufficient statistic. Hyun Min Kang Biostatistics 602 - Lecture 04 January 22th, 2013 3 / 23 Minimal Sufficient Statistics Ancillary Statistics Location-scale Family Summary . Theorem for Minimal Sufficient Statistics . Theorem 6.2.13 . • fX(x) be pmf or pdf of a sample X. • Suppose that there exists a function T(x) such that, • For every two sample points x and y, j j • The ratio fX(x θ)/fX(y θ) is constant as a function of θ if and only if T(x) = T(y). • Then T(X) is a minimal sufficient statistic for θ. In other words.. j j ) • fX(x θ)/fX(y θ) is constant as a function of θ = T(x) = T(y). ) j j • T(x) = T(y) = fX(x θ)/fX(y θ) is constant as a function of θ . Hyun Min Kang Biostatistics 602 - Lecture 04 January 22th, 2013 4 / 23 . Solution . P Yn − − exp (− n (x − θ)) j exp( (xi θ)) Q i=1 i fX(x θ) = 2 = n 2 (1 + exp(−(xi − θ))) (1 + exp(−(xi − θ))) i=1 P i=1 exp (− n x ) exp(nθ) = Q i=1 i n − − 2 . i=1(1 + exp( (xi θ))) Minimal Sufficient Statistics Ancillary Statistics Location-scale Family Summary . Exercise from the textbook . Problem . X1; ··· ; Xn are iid samples from e−(x−θ) f (xjθ) = ; −∞ < x < 1; −∞ < θ < 1 X (1 + e−(x−θ))2 .Find a minimal sufficient statistic for θ. Hyun Min Kang Biostatistics 602 - Lecture 04 January 22th, 2013 5 / 23 P − n − Q exp ( i=1(xi θ)) = n 2 (1 + exp(−(xi − θ))) P i=1 exp (− n x ) exp(nθ) = Q i=1 i n − − 2 i=1(1 + exp( (xi θ))) Minimal Sufficient Statistics Ancillary Statistics Location-scale Family Summary . Exercise from the textbook . Problem . X1; ··· ; Xn are iid samples from e−(x−θ) f (xjθ) = ; −∞ < x < 1; −∞ < θ < 1 X (1 + e−(x−θ))2 .Find a minimal sufficient statistic for θ. Solution . Yn exp(−(x − θ)) f (xjθ) = i X (1 + exp(−(x − θ)))2 i=1 i . Hyun Min Kang Biostatistics 602 - Lecture 04 January 22th, 2013 5 / 23 P exp (− n x ) exp(nθ) = Q i=1 i n − − 2 i=1(1 + exp( (xi θ))) Minimal Sufficient Statistics Ancillary Statistics Location-scale Family Summary . Exercise from the textbook . Problem . X1; ··· ; Xn are iid samples from e−(x−θ) f (xjθ) = ; −∞ < x < 1; −∞ < θ < 1 X (1 + e−(x−θ))2 .Find a minimal sufficient statistic for θ. Solution . P Yn exp(−(x − θ)) exp (− n (x − θ)) f (xjθ) = i = Q i=1 i X (1 + exp(−(x − θ)))2 n (1 + exp(−(x − θ)))2 i=1 i i=1 i . Hyun Min Kang Biostatistics 602 - Lecture 04 January 22th, 2013 5 / 23 Minimal Sufficient Statistics Ancillary Statistics Location-scale Family Summary . Exercise from the textbook . Problem . X1; ··· ; Xn are iid samples from e−(x−θ) f (xjθ) = ; −∞ < x < 1; −∞ < θ < 1 X (1 + e−(x−θ))2 .Find a minimal sufficient statistic for θ. Solution . P Yn − − exp (− n (x − θ)) j exp( (xi θ)) Q i=1 i fX(x θ) = 2 = n 2 (1 + exp(−(xi − θ))) (1 + exp(−(xi − θ))) i=1 P i=1 exp (− n x ) exp(nθ) = Q i=1 i n − − 2 . i=1(1 + exp( (xi θ))) . Hyun Min Kang Biostatistics 602 - Lecture 04 January 22th, 2013 5 / 23 P Q exp (− n x ) n (1 + exp(−(y − θ)))2 = Pi=1 i Qi=1 i − n n − − 2 exp ( i=1 yi) i=1(1 + exp( (xi θ))) The ratio above is constant to θ if and only if x1; ··· ; xn are permutations ··· ··· of y1; ; yn. So the order statistic T(X) = (X(1); ; X(n)) is a minimal sufficient statistic. Minimal Sufficient Statistics Ancillary Statistics Location-scale Family Summary . Solution (cont’d) . Applying Theorem 6.2.13 . P Q f (xjθ) exp (− n x ) exp(nθ) n (1 + exp(−(y − θ)))2 X = Pi=1 i Qi=1 i j − n n − − 2 fX(y θ) exp ( i=1 yi) exp(nθ) i=1(1 + exp( (xi θ))) . Hyun Min Kang Biostatistics 602 - Lecture 04 January 22th, 2013 6 / 23 The ratio above is constant to θ if and only if x1; ··· ; xn are permutations ··· ··· of y1; ; yn. So the order statistic T(X) = (X(1); ; X(n)) is a minimal sufficient statistic. Minimal Sufficient Statistics Ancillary Statistics Location-scale Family Summary . Solution (cont’d) . Applying Theorem 6.2.13 . P Q f (xjθ) exp (− n x ) exp(nθ) n (1 + exp(−(y − θ)))2 X = Pi=1 i Qi=1 i f (yjθ) exp (− n y ) exp(nθ) n (1 + exp(−(x − θ)))2 X Pi=1 i Q i=1 i exp (− n x ) n (1 + exp(−(y − θ)))2 = Pi=1 i Qi=1 i − n n − − 2 exp ( i=1 yi) i=1(1 + exp( (xi θ))) . Hyun Min Kang Biostatistics 602 - Lecture 04 January 22th, 2013 6 / 23 Minimal Sufficient Statistics Ancillary Statistics Location-scale Family Summary . Solution (cont’d) . Applying Theorem 6.2.13 . P Q f (xjθ) exp (− n x ) exp(nθ) n (1 + exp(−(y − θ)))2 X = Pi=1 i Qi=1 i f (yjθ) exp (− n y ) exp(nθ) n (1 + exp(−(x − θ)))2 X Pi=1 i Q i=1 i exp (− n x ) n (1 + exp(−(y − θ)))2 = Pi=1 i Qi=1 i − n n − − 2 exp ( i=1 yi) i=1(1 + exp( (xi θ))) The ratio above is constant to θ if and only if x1; ··· ; xn are permutations ··· ··· of y1; ; yn. So the order statistic T(X) = (X(1); ; X(n)) is a minimal .sufficient statistic. Hyun Min Kang Biostatistics 602 - Lecture 04 January 22th, 2013 6 / 23 . Examples of Ancillary Statistics . X ; ··· ; X ∼i.i.d. N (µ, σ2) where σ2 is known. 1 n P 2 1 n − 2 • sX = n−1 i=1(Xi X) is an ancillary statistic 2 • X1 − X2 ∼ N (0; 2σ ) is ancillary. 2 • (X1 + X2)/2 − X3 ∼ N (0; 1:5σ ) is ancillary. (n−1)s2 • X ∼ χ2 is ancillary. σ2 n−1 Minimal Sufficient Statistics Ancillary Statistics Location-scale Family Summary . Ancillary Statistics . Definition 6.2.16 . A statistic S(X) is an ancillary statistic if its distribution does not depend .on θ. Hyun Min Kang Biostatistics 602 - Lecture 04 January 22th, 2013 7 / 23 P 2 1 n − 2 • sX = n−1 i=1(Xi X) is an ancillary statistic 2 • X1 − X2 ∼ N (0; 2σ ) is ancillary. 2 • (X1 + X2)/2 − X3 ∼ N (0; 1:5σ ) is ancillary.