1 M3S3/M4S3 STATISTICAL THEORY II WORKED EXAMPLE: EFFICIENT INFERENCE FOR THE CAUCHY DISTRIBUTION Suppose that X1;:::;Xn are i.i.d random variables having a Cauchy distribution with pdf 1 1 f (xjθ) = x 2 R Xjθ ¼ 1 + (x ¡ θ)2 for parameter θ 2 R. (i) Show that the Fisher Information for θ is I(θ) = 1=2. (ii) Find the an asymptotic approximation to the distribution of the sample median, Mn, that is the quantile corresponding to p = 1=2, and hence comment on the e±ciency of Mn in estimating θ. (iii) Using the one-step estimation approach, construct an e±cient estimator of θ, stating its asymp- totic variance. (iv) Find the an asymptotic approximation to the distribution of the length, Ln, of the central 95 % sample interval, that is Ln = X(k2) ¡ X(k1) where k1 = dnp1e, k2 = dnp2e for p1 = 0:025 and p2 = 0:975, dxe denotes the smallest integer not less than x, and X(k) denotes the kth order statistic. SOLUTION (i) The Fisher information is computed in the usual way. l(θ) = ¡ log ¼ ¡ log(1 + (x ¡ θ)2) 2(x ¡ θ) l_(θ) = 1 + (x ¡ θ)2 1 ¡ (x ¡ θ)2 Äl(θ) = ¡2 (1 + (x ¡ θ)2)2 Thus Z 1 1 ¡ (x ¡ θ)2 1 1 I(θ) = E [¡Äl(θ)] = 2 dx fXjθ 2 2 2 ¡1 (1 + (x ¡ θ) ) ¼ 1 + (x ¡ θ) Z Z 2 1 1 ¡ (x ¡ θ)2 2 1 1 ¡ x2 = 2 3 dx = 2 3 dx ¼ ¡1 (1 + (x ¡ θ) ) ¼ ¡1 (1 + x ) Z 2 ¼=2 1 ¡ tan2 t dx = 2 3 dt x = tan t ¼ ¡¼=2 (1 + tan t) dt Z ¼=2 2 Z ¼=2 2 2 1 ¡ tan t 2 2 1 ¡ tan t 2 = 2 3 sec t dt = 2 3 sec t dt ¼ ¡¼=2 (1 + tan t) ¼ ¡¼=2 (sec t) Z ¼=2 2 Z ¼=2 µ 2 ¶ 2 1 ¡ tan t 2 sin t 4 = 2 2 t dt = 1 ¡ 2 cos t dt ¼ ¡¼=2 (sec t) ¼ ¡¼=2 cos t Z 2 ¼=2 ¡ ¢ = cos4 t ¡ sin2 t cos2 t dt ¼ ¡¼=2 Z Z 8 ¼=2 4 ¼=2 = cos4 t dt ¡ cos2 t dt ¼ 0 ¼ 0 2 By a standard result Z ¼=2 Z ¼=2 C(k) = cosk t dt = cos t cos(k¡1) t dt 0 0 Z h i¼=2 ¼=2 = sin t cos(k¡1) t + (k ¡ 1) sin2 t cos(k¡2) t dt 0 0 Z ¼=2 = 0 + (k ¡ 1) (1 ¡ cos2 t) cos(k¡2) t dt 0 Z ¼=2 Z ¼=2 = (k ¡ 1) cos(k¡2) t dt ¡ (k ¡ 1) cosk t dt 0 0 = (k ¡ 1)C(k ¡ 2) ¡ (k ¡ 1)C(k) and hence µ ¶ k ¡ 1 C(k) = C(k ¡ 2) k Thus, as C(0) = ¼=2, C(2) = ¼=4 and C(4) = 3¼=16, so 8 3¼ 4 ¼ 1 I(θ) = £ ¡ £ = ¼ 16 ¼ 4 2 as required. (ii) Mn is the p = 1=2 quantile, and by the standard result for sample quantiles p L 2 n (Mn ¡ θ) ¡! N(0; σθ ) as the (true) distribution median is θ (as the pdf is symmetric around θ), and where 2 2 p(1 ¡ p) 1=4 ¼ σθ = 2 = 2 = ¼ 2:467 ffXjθ(θ)g f1=¼g 4 ¡1 Thus the estimator Mn is ine±cient, as the lowest possible variance given by I(θ) = 1=2 is I(θ) = 2. Note that an e±cient estimator can be found as the solution to the likelihood equations l_n(θ) = 0, or equivalently Xn (x ¡ θ) i = 0 1 + (x ¡ θ)2 i=1 i but this is not straightforward. Note also that the other natural estimator, the sample mean, is not useful, as it has intractable asymptotic behaviour (the expectation of the Cauchy does not exist). (iii) The one-step approach to e±cient estimation computes one of the following two estimators ³ ´¡1 (1) θb = θen ¡ lÄn(θen) l_n(θen) (N) and ³ ´¡1 1 θb? = θe + I(θe ) l_ (θe ) (S) n n n n n based on the consistent estimators θen. The one-step estimators are asymptotically equivalent to the e±cient estimator, and thus have asymptotic variance 2. 3 Now Mn is a suitable consistent estimator of θ, and Xn (x ¡ θ) Xn 1 ¡ (x ¡ θ)2 l_ (θ) = 2 i Äl (θ) = ¡2 i n 1 + (x ¡ θ)2 n (1 + (x ¡ θ)2)2 i=1 i i=1 i so therefore the one-step estimators are à !¡1 Xn 1 ¡ (x ¡ M )2 Xn (x ¡ M ) θb(1) = M + 4 i n i n n (1 + (x ¡ M )2)2 1 + (x ¡ M )2 i=1 i n i=1 i n and 4 Xn (x ¡ M ) θb? = M + i n n n 1 + (x ¡ M )2 i=1 i n (iv) Using the standard result from lectures, we have that 0 2 31 p1 (1 ¡ p1) p1 (1 ¡ p2) © ª2 µµ ¶ µ ¶¶ B 6 f (xp ) f (xp ) 7C B 6 fXjθ (xp1 ) Xjθ 1 Xjθ 2 7C p X x L B 6 7C n (k1) ¡ p1 ! N B0; 6 7C ´ N(0; §); X(k ) xp B 6 7C 2 2 @ 4 p1 (1 ¡ p2) p2 (1 ¡ p2) 5A © ª2 f (xp ) f (xp ) Xjθ 1 Xjθ 2 fXjθ (xp2 ) say, where here, p1 = 0:025, p2 = 0:975, and xp1 and xp2 are the corresponding true distribution quantiles. Now, here Z x Z x 1 1 1 x¡θ 1 FXjθ(x) = fXjθ(t) dt = 2 dt = [arctan t]¡1 = (arctan(x ¡ θ) + ¼=2) ¡1 ¡1 ¼ 1 + (t ¡ θ) ¼ ¼ so 1 1 (arctan(x ¡ θ) + ¼=2) = 0:025 and (arctan(x ¡ θ) + ¼=2) = 0:975 ¼ p1 ¼ p2 and hence xp1 = θ + tan((0:025¼) ¡ ¼=2) = θ ¡ 12:706 xp2 = θ + tan((0:975¼) ¡ ¼=2) = θ + 12:706: Hence · ¸ 0:025 (1 ¡ 0:025) (1 + 12:7062)2 0:025 (1 ¡ 0:975) (1 + 12:7062)(1 + 12:7062) § = ¼2 0:025 (1 ¡ 0:975) (1 + 12:7062)(1 + 12:7062) 0:975 (1 ¡ 0:975) (1 + 12:7062)2 · ¸ · ¸ 0:025 (1 ¡ 0:025) 0:025 (1 ¡ 0:975) 6348:501 162:782 = ¼2(1 + 12:7062)2 = 0:025 (1 ¡ 0:975) 0:975 (1 ¡ 0:975) 162:782 6348:501 Finally, as Ln = Xk2 ¡ Xk1 , using the Delta method with function g(t1; t2) = t2 ¡ t1, we have p L T n(Ln ¡ ¸) ¡! N(0; g_(xp1 ; xp2 )§g_ ) where ¸ is the true length, that is 2 £ 12:706 = 25:412, and · ¸ @g(t) @g(t) g_(t) = = [¡1 1] : @t1 @t2 Thus T g_(xp1 ; xp2 )§g_ = 12371:44: and hence p L n(Ln ¡ 25:412) ¡! N(0; 12371:44):.