1

M3S3/M4S3 STATISTICAL THEORY II WORKED EXAMPLE: TESTING FOR THE PARETO DISTRIBUTION

Suppose that X1,...,Xn are i.i.d random variables having a Pareto distribution with pdf

θcθ f (x|θ) = x > c X|θ xθ+1 and zero otherwise, for known constant c > 0, and parameter θ > 0.

(i) Find the ML , θbn, of θ, and find the of √ n(θbn − θT )

where θT is the true value of θ. (ii) Consider testing the hypotheses of

H0 : θ = θ0

H1 : θ 6= θ0

for some θ0 > 0. Determine the likelihood ratio, Wald and Rao tests of this hypothesis.

SOLUTION (i) The ML estimate θbn is computed in the usual way:

Yn Yn θcθ θncnθ Ln(θ) = fX|θ(xi|θ) = θ+1 = θ+1 i=1 i=1 xi sn Yn where sn = xi. Then i=1

ln(θ) = n log θ + nθ log c − (θ + 1) log sn

n l˙ (θ) = + n log c − log s n θ n and solving l˙n(θ) = 0 yields the ML estimate

· ¸ " #−1 log s −1 1 Xn θb = n − log c = log x − log c . n n n i i=1 The corresponding estimator is therefore

" #−1 1 Xn θb = log X − log c . n n i i=1 Computing the asymptotic distribution directly is difficult because of the reciprocal. However, consider φ = 1/θ; by invariance, the ML estimator of φ is

1 Xn 1 Xn φb = log X − log c = (log X − log c) n n i n i i=1 i=1 2

which implies how we should compute the asymptotic distribution of θbn - we use the CLT on the random variables Yi = log Xi − log c = log(Xi/c), and then use the Delta Method.

To implement the CLT, we need the expectation and of Y = log(X/c). Now

³ c ´θ F (x) = 1 − x > c X x so that

FY (y) = P [Y ≤ y] = P [log(X/c) ≤ y] = P [X ≤ c exp{y}] = 1 − exp{−θy} y > 0 and hence Y ∼ Exponential(θ). By standard results 1 1 E [Y ] = = φ V ar [Y ] = = φ2, fY θ fY θ2 and therefore, by the CLT, √ b L 2 n(φn − φT ) −→ N(0, φT ) where φT = 1/θT .

Finally, let g(t) = 1/t so thatg ˙(t) = −1/t2. Then, by the Delta Method √ b L 2 2 n(g(φn) − g(φT )) −→ N(0, {g˙(φT )} φT ) so that, as g(φT ) = 1/φT = θT √ b L 4 2 2 n(θn − θT ) −→ N(0, {1/φT }φT ) ≡ N(0, θT ).

(ii) For the likelihood ratio test:

b b b bn nθn θn+1 h i Ln(θn) θnc /sn b b b λn = 2 log = 2 log θ +1 = 2n log(θn/θ0) + (θn − θ0) log c − (θn − θ0)mn L (θ ) n nθ0 0 n 0 θ0 c /sn where mn = (log sn)/n. For the : 2 Wn = n(θbn − θ0)I(θbn)(θbn − θ0) = nI(θbn)(θbn − θ0) where I(θ) is the Fisher information for this model. From first principles

l(θ) = log θ + θ log c − (θ + 1) log x

1 l˙(θ) = + log c − log x θ 1 ¨l(θ) = − θ2 so I(θ) = 1/θ2; in fact, we could have deduced this from the results derived in (i). Thus

à !2 θbn − θ0 Wn = n . θbn 3

Note that, although the Fisher Information is available in this case, it may be statistically advantageous in a finite sample case to replace I(θbn) by Ibn(θbn), derived in the usual way from the first or second derivatives of ln. That is, we might use µ ¶ 1 Xn 1 Xn 1 2 1 Xn Ib (θb ) = S(x , θ)2 = + log c − log x = (y − y)2 n n n i n b i n i i=1 i=1 θn i=1 where yi = log xi − log c. Alternately, using the second derivative,

1 Xn 1 Xn Ib (θb ) = − ¨l (θ) = 1/θb2 = 1/θb2 = I(θb ). n n n i n n n n i=1 i=1 For the Rao test: in the single parameter case

T −1 2 Rn = Zn[I(θ0)] Zn = Zn/I(θ0) where µ ¶ 1 Xn 1 Xn 1 Z = √ S(x , θ ) = √ + log c − log x n n i 0 n θ i i=1 i=1 0 so that if yi = log xi − log c as before ( ) ( ) Xn 2 Xn 2 (yi − 1/θ0) (yi − 1/θ0) i=1 i=1 Rn = = 2 . nI(θ0) n/θ0

SUPPLEMENTARY EXERCISES: Suppose that c is also an unknown parameter. Find

• the ML estimator for c, bc

• the weak-law (ie probability) limit of bc

• an asymptotic (large n) approximation to the distribution of bc.

• the profile likelihood for θ.

Hint: Recall, when considering the likelihood for c, that xi > c for all i. Then, think back to M2S1 Chapter 5, and extreme order .