Credible Intervals Comparison of Approaches

Chapter 8 Interval Estimation Bayesian Approches

1 / 9 Credible Intervals Comparison of Approaches Outline

Credible Intervals

Comparison of Approaches

2 / 9 Credible Intervals Comparison of Approaches Credible Intervals

A Bayesian interval estimate is called a credible interval. Recall that for the Bayesian approach to statistics, both the data and the parameter are random thus, the interval estimate is a statement about the posterior probability distribution of the parameter θ.

P{Θ˜ ∈ C(X)|X = x} = γ.

Here Θ˜ is the random variable having a distribution equal to the prior probability π. We have choices in defining this interval. For example, we can • choose the narrowest interval, which involves choosing those values of highest posterior density. • choosing the equal-tail size interval in which the probability of being below the interval is as likely as being above it.

3 / 9 Credible Intervals Comparison of Approaches Credible Intervals Example. For independent flips of a biased coin with a prior distribution π(p) ∼ Beta(α, β),

(α−1) (β−1) π(p) = cα,βp (1 − p) , 0 < p < 1.

If we perform n Bernoulli trials x = (x1,..., xn), then the joint density Pn Pn k=1 xk n− k=1 xk fX |P˜(x|p) = p (1 − p) . Thus the posterior distribution of the parameter P˜ given the data x, Pn Pn k=1 xk n− k=1 xk (α−1) (β−1) fP˜|X (p|x) ∝ fX |P˜(x|p)π(p) = p (1 − p) · cα,βp (1 − p) . Pn Pn α+ xk −1 β+n− xk −1 = cα,βp k=1 (1 − p) k=1 . The posterior also has a beta distribution, parameters α + #successes and β + #failures.

4 / 9 Credible Intervals Comparison of Approaches Credible Intervals 4

Using a Beta(3, 3) prior, we look at credible intervals 3 after (top)6 successes in 10 trials and (bottom) 16 2 1 successes in 25 trials. density posterior 0 Exercise. Explain the posterior distribution and give 0.0 0.2 0.4 0.6 0.8 1.0 95%, 98% and 99% credible intervals after 10 and c(pc[1],c(pc[2],q pc[1])pc[2]) 25 trials using the information below 4 3

> q<-c(0.005,0.01,0.025,0.5,0.975,0.99,0.995) 2

> round(qbeta(q,9,7),3) 1 posterior density posterior

[1] 0.256 0.282 0.323 0.565 0.787 0.821 0.841 0

> round(qbeta(q,19,12),3) 0.0 0.2 0.4 0.6 0.8 1.0 [1] 0.384 0.406 0.439 0.615 0.773 0.799 0.815 Figure: Posterior densities, indicating 95% credible intervals

5 / 9 Credible Intervals Comparison of Approaches Optimizing the Length of a Credible Interval • Since posterior distribution is a density, we do not face the same issues that were involve in finding confidence intervals for discrete random variables. • To choose the narrowest interval, we follow a procedure similar to that used in minimizing the length of a confidence interval.

> gamma<-0.95 > diff<-function(a) qbeta(gamma+a,9,7)-qbeta(a,9,7) > (astar<-optimize(diff,interval=c(0.001,0.05),maximum=FALSE)) $minimum [1] 0.02802784 $objective [1] 0.4639734 > (u<-qbeta(gamma+astar$minimum,9,7)) [1] 0.7924897 > (l<-qbeta(astar$minimum,9,7)) [1] 0.3285163

6 / 9 Credible Intervals Comparison of Approaches Credible Intervals Recall the example of a normal prior onΨ of normal observations X . We take

• The prior density to be N(θ1, 1/λ0) 2 • The observations X1, . . .¸ , Xn are independent N(θ, σ ) • Their mean X¯ ∼ N(θ, σ2/n) 2 2 • The posterior distribution is N(θ1(¯x), σ /(n + λ0σ )) where

2 λ0 n/σ θ1(¯x) = 2 θ1 + 2 x¯. λ0 + n/σ λ0 + n/σ

Thus a γ- level credible interval is symmetric about θ1(¯x).

σ2 θ1(¯x) ± z(1−γ)/2 2 . n + λ0σ Thus, it is both equal-tailed and narrowest. 7 / 9 Credible Intervals Comparison of Approaches Criterion of Choosing Credible Intervals

One approach to choosing a credible interval C = [a, b] a < b is to adopt a loss function. For example, take u to be a strictly increasing function and loss function,

L(b − a, θ) = u(b − a) + cI[a,b]c (θ).

Thus the loss is the sum of • a function, u, of the length of the interval plus • c if the parameter is not in the interval. Risk, the expected loss, is

R(b − a, θ) = u(b − a) + cPθ{θ∈ / [a, b]}.

The credible interval chosen is the one that minimizes risk.

8 / 9 Credible Intervals Comparison of Approaches Comparison of Approaches

Bayesian Approach Classical Approach

• Begins with a prior probability π on • Begins with a family of distributions the parameter spaceΘ. indexed with a parameterΘ. • Uses the data and textcolortealBayes • Uses a significance level to construct formula to compute the posterior to create a random interval probability. [θˆ`(x), θˆu(x)]. • The textcolortealcredible interval • The creation of the confidence interval having textcolortealposterior can be considered as a single probability γ is the probability that θ, textcolortealBernoulli trial with viewed as a random quantity, lies in probability of success γ. the interval.

9 / 9