Pivotal Quantities Pivoting Using the Probability Transform

Topic 16 Building Confidence Intervals

1 / 16 Pivotal Quantities Pivoting Using the Probability Transform

Outline

Pivotal Quantities Constructing Confidence Intervals

Pivoting Using the Probability Transform Optimizing the Length of the Confidence Interval

2 / 16 Pivotal Quantities Pivoting Using the Probability Transform

Pivotal Quantities

A random quantity Q(X, θ) is called a pivotal quality or a pivot if • is a function of X and θ where θ is the only unknown . • The distribution of Q(X, θ) does not depend on θ.

Examples. For X = (X1,..., Xn), independent and identically distributed.

• If the Xi arise from a location family with density f (x − θ), then

n X Q(X, θ) = ai (Xi − θ) i=1

is a pivotal quantity. In particular, X¯ − θ is a pivot.

3 / 16 Pivotal Quantities Pivoting Using the Probability Transform

Pivotal Quantities

1  x  • If the Xi arise from a scale family with density β f β , then

Pn a X Q(X, θ) = i=1 i i β Pn is a pivotal quantity. In particular, i=1 XI /β is a pivot. 1  x−θ  • If the Xi arise from a location-scale family with density β f β

X¯ − θ Q(X, θ) = S is a pivotal quantity. Note. Even though Q(X, θ) does not depend on θ, it is not an ancillary . It is not even a statistic. 4 / 16 Pivotal Quantities Pivoting Using the Probability Transform

Constructing Confidence Intervals To construct a γ-level confidence interval, • Find ` and u so that Pθ{` ≤ Q(x, θ) ≤ u} = γ.

• Define C(x) = {x; ` ≤ Q(X, θ) ≤ u}.

Then Pθ{θ ∈ C(X)} = γ. If, for each x, Q(x, θ) is monotone increasing in θ, then let • θˆ`(x) be the solution to Q(x, θ) = ` and • θˆu(x) be the solution to Q(x, θ) = u. Then. C(x) = {x; θˆ`(x) ≤ θ ≤ θˆu(x)}. 5 / 16 Pivotal Quantities Pivoting Using the Probability Transform

Constructing Confidence Intervals

For Q(x, θ) is monotone decreasing in θ, then reverse the roll of θˆ`(x) and θˆu(x).

Example. For X = (X1,..., Xn) from a location family with density f (x − θ), then take Q(X, θ) = X¯ − θ as the pivot. Then choose ` and u so that

γ = Pθ{X; ` ≤ Q(X, θ) ≤ u} = Pθ{X; ` ≤ X¯ − θ ≤ u}

= Pθ{X : ` − X¯ ≤ −θ ≤ u − X¯} = Pθ{X; X¯ − ` ≥ θ ≥ X¯ − u}

2 For X = (X1,..., Xn) ∼ N(θ, σ0), then take

z σ0 z σ0 ` = − (1−√γ)/2 and u = (1−√γ)/2 . n n

6 / 16 Pivotal Quantities Pivoting Using the Probability Transform

Constructing Confidence Intervals

1  x  Example. For X = (X1,..., Xn) from a scale family with density β f β , take Pn Q(X, β) = i=1 xi /β = T (x)/β as the pivot. Then choose ` and u so that

γ = Pβ{X; ` ≤ Q(X, β) ≤ u}

= Pβ{X; ` ≤ T (X)/β ≤ u}

= Pβ{X; 1/` ≥ β/T (X) ≥ 1/u}

= Pβ{X; T (X)/` ≥ β ≥ T (X)/u}

For Exp(β) , T (X)/β ∼ Γ(n, 1) is a pivot. For an equal tailed 95% confidence interval,

7 / 16 Pivotal Quantities Pivoting Using the Probability Transform

Constructing Confidence Intervals

> gamma<-0.95 0.12 > alpha<-(1-gamma)/2 > qgamma(c(alpha,1-alpha),10) 0.08 dgamma(x, 10) dgamma(x, 0.04 [1] 4.795389 17.084803 0.00

0 5 10 15 20

x Thus,

0.95 = Pβ{X; 4.795 ≤ T (X)/β ≤ 17.085}

= Pβ{X; T (X)/4.795 ≥ β ≥ T (X)/17.085}

8 / 16 Pivotal Quantities Pivoting Using the Probability Transform

Constructing Confidence Intervals Example. For X = (X1,..., Xn) ∼ U(0, θ), take

Q(x, β) = max xi /θ = T (x)/θ i n as the pivot. Then M = T (x)/θ has distribution function FM (t) = t in the interval [0, 1]. For an equal tailed γ-confidence interval, set 1 + γ 1 − γ F (u) = and F (`) = . M 2 M 2

1 + γ 1/n 1 − γ 1/n u = and ` = . 2 2 ( ) 1 − γ −1/n 1 + γ −1/n γ = P X; T (X) ≥ β ≥ T (X) . β 2 2

9 / 16 Pivotal Quantities Pivoting Using the Probability Transform

Pivoting Using the Probability Transform

Theorem. Let T be a continuous (test) statistic with distribution function FT (t|θ), θ ∈ Θ ⊂ R. Let α1 + α2 = 1 − γ be the size of the two tails outside the confidence interval.

For FT (t|θ)a decreasing function of θ, define θˆu(t) and θˆ`(t) by

FT (t|θˆu(t)) = α1 and FT (t|θˆ`(t)) = 1 − α2,

respectively. Then, the random interval

[θˆ`(T ), θˆu(T )]

is a γ-level confidence interval.

10 / 16 Pivotal Quantities Pivoting Using the Probability Transform

Pivoting Using the Probability Transform

Proof. Because, for any given value of t, FT (t|θ)a decreasing function of θ. Thus,

1 − α2 > α1 implies θˆ`(t) < θˆu(t).

Moreover,

FT (t|θ) < α1 ⇔ θ > θˆu(t)

FT (t|θ) > 1 − α2 ⇔ θ < θˆ`(t).

Consequently,

{θ; α1 ≤ FT (T |θ) ≤ 1 − α2} = {θ; θ`(T ) ≤ θ ≤ θu(T )}

Remark. The proof can be modified to handle the case of FT (t|θ) an increasing function of θand for one-sided confidence intervals.

11 / 16 Pivotal Quantities Pivoting Using the Probability Transform

Pivoting Using the Probability Transform

Example. For independent X = (X1,..., Xn) ∼ U(−θ, θ), we will use T (X) = X(n) − X(1) to determine the confidence interval. Set

X + θ U = i . i 2θ

Then the independent random variables U = (U1,..., Un) ∼ U(0, 1). The

T (X) = X(n) − X(1) = 2θ((U(n) − θ) − (U(1) − θ)) = 2θ(U(n) − U(1)) = 2θD.

The distribution

FT (t|θ) = Pθ{T ≤ t} = Pθ{2θD ≤ t} = P{D ≤ t/(2θ)} = FD (t/(2θ)),

which is a decreasing function of θ.

12 / 16 Pivotal Quantities Pivoting Using the Probability Transform

Pivoting Using the Probability Transform

We have seen that the distribution of D = U(n) − U(1) ∼ Beta(n − 1, 2). Let QD be the quantile function for D,  T (X)   1 2θ 1  QD (α1) ≤ ≤ QD (1 − α2) = ≥ ≥ 2θ QD (α1) T (X) QD (1 − α2)  T (X) T (X)  = ≥ θ ≥ 2QD (α1) 2QD (1 − α2) For n = 16 and γ = 0.95, then the confidence interval with equal probability tails, > gamma<-0.95; alpha<-(1-gamma)/2 6 5

> qbeta(c(alpha,1-alpha),15,2) 4 3 dbeta(x, 15, 2) dbeta(x, 2

[1] 0.6976793 0.9844864 1 0

0.6 0.7 0.8 0.9 1.0

x

13 / 16 Pivotal Quantities Pivoting Using the Probability Transform

Optimizing the Length of the Confidence Interval As you see the choice of endpoints has the only constraint that the sum of the tail probabilities equals1 − γ. One goal is to have the shortest expected length.This leads to the following constrained optimization problem.

min{E[θˆu(T ) − θˆ`(T )]; P{θˆu(T ) − θˆ`(T )} ≥ γ}}

For the case where the density of T is unimodal, to maximize the probability over an interval of length c, we differentiate. d (F (a + c) − F (a)) = f (a + c) − f (a) = 0. da T T T T If the tm of the density, then a < tm < a + c. Also, 0 0 fT (a) > 0 and fT (a + c) < 0 showing that the probability is maximized. Notice that if the distribution ofT is symmetric about its mode, then the equal tailed probabilities are also the shortest. 14 / 16 Pivotal Quantities Pivoting Using the Probability Transform

Optimizing the Length of the Confidence Interval Returning to the previous example, we want to find the shortest interval for D that has probability γ. > gamma<-0.95 > diff<-function(a) qbeta(gamma+a,15,2)-qbeta(a,15,2) >(astar<-optimize(diff,interval=c(0.001,0.05),maximum=FALSE)) $minimum [1] 0.04840239 $objective [1] 0.2621124 > (u<-qbeta(gamma+astar$minimum,15,2)) [1] 0.9962875 > (l<-qbeta(astar$minimum,15,2)) [1] 0.7341751 > dbeta(c(l,u),15,2) [1] 0.8433389 0.8457852 15 / 16 Pivotal Quantities Pivoting Using the Probability Transform

Optimizing the Length of the Confidence Interval

• Our first confidence interval had equal tails 6 α1 = α2 = 0.025 and width

0.2868071. 5

• The density at the left 4

endpoint is lower than on the 3 dbeta(x, 15, 2) dbeta(x, right. 2

• This suggests that we can 1

shorten the confidence 0 interval by moving the 0.6 0.7 0.8 0.9 1.0 interval to the right. x

16 / 16