Statistical Intervals (STT Consulting) March 2010

Statistical intervals (STT Consulting) March 2010

Statistical Intervals

STT Consulting

March 2010

[email protected] www.stt-consulting.com

1 Statistical intervals (STT Consulting) March 2010

Confidence Interval It is often desirable to express or quantify the degree of uncertainty for some estimate of an unknown population parameter. The most common type of interval estimate is a confidence interval (CI). The CI is essentially an estimate for an unknown population parameter expressed as a range of values with some specified level of confidence. The level of confidence describes the probability that the “interval will capture the true parameter in repeated samples”. CI’s should be calculated for every estimate of a parameter (mean, SD, r, slope, power, etc). See also: CI of percentiles >Tolerance Interval

Calculation of confidence intervals Many of the more simple confidence intervals can be calculated with EXCEL and applying its “INVERSE” functions (NORSMINV, TINV, CHIINV, FINV: www.stt-consulting.com >Statistics >Excel Statistical Functions).

Example: confidence interval of the mean

 = xbar ± tP,df * s/SQRT(n) Note: s/SQRT(n) = standard error

R-script {TechingDemos} : Distribution of confidence intervals ci.examp(mean.sim = 100, sd = 3, n = 9, reps = 100, conf.level = 0.95, method = "t") Red vertical lines: z value (units of standard error, here 1.96 * SE) Pink horizontal lines: CI’s that do not contain the parameter = 100 (5 expected).

Example: confidence interval of the mean

 = xbar ± tP,df * s/SQRT(n) Note: s/SQRT(n) = standard error

R-script : Distribution of t-values n=9 loop=500000 t=c(rep(0,loop)) for(i in 1:loop) { x=rnorm(n, mean = 0, sd = 3) xstd=sqrt(var(x)) xbar=mean(x) se=xstd/sqrt(n) t[i]=(0-xbar)/se } hist(t,breaks=550, ylim=c(0,0.5), xlim=c(-6,6), freq=FALSE, lty=0) lines(density(t), col="red", lwd=2) quantile(t,probs=c(0.025,0.5,0.975)) #t = qt(0.975,8) = 2.306

Prediction interval Assume, you have estimated mean (xbar) and standard deviation (s) from n results (for example, 20). Now, you want to predict the upper limit for the next result with 95% probability (also denoted as (100)%;  0.95).

Prediction interval, next n + 1 xi = xbar + kP*s, with kP = t(p=2sided 95%,df=19)*SQRT(1+1/20) = 2.14 (see Table)

Assume, you want to predict the upper limit to contain ALL next m results with 95% probability, or to contain ALL of m-k out of m additional observations. These calculations are complex and one has to consult tables or use software.

Prediction interval, ALL next m-k out of m

Xmk = xbar + kP*s, with kP: Consult tables (Odeh 1990) or use software Factor = 3.16, for n = 20, next 9 of 9

Odeh RE. Two-sided prediction intervals to contain at least k out of m future observations from a normal distribution. Technometrics 1990;32:203-16.

Table 1.A.0.1: Factors for (100)% two-sided prediction intervals, based on a sample of size n, to contain all of m out of m additional observations m  = .95 n 1 2 3 4 5 6 7 8 9 8 2.51 2.97 3.24 3.43 3.57 3.69 3.79 3.88 3.95 9 2.43 2.86 3.11 3.29 3.43 3.54 3.63 3.71 3.78 10 2.37 2.79 3.02 3.19 3.32 3.43 3.51 3.59 3.66 11 2.33 2.72 2.95 3.11 3.24 3.34 3.42 3.49 3.56 12 2.29 2.68 2.90 3.05 3.17 3.27 3.35 3.42 3.48 15 2.22 2.57 2.78 2.92 3.03 3.12 3.19 3.26 3.31 20 2.14 2.48 2.67 2.80 2.90 2.98 3.05 3.11 3.16 25 2.10 2.43 2.61 2.73 2.83 2.91 2.97 3.03 3.08 30 2.08 2.39 2.57 2.69 2.78 2.86 2.92 2.97 3.02 40 2.05 2.35 2.52 2.64 2.73 2.80 2.86 2.91 2.95 60 2.02 2.31 2.48 2.59 2.67 2.74 2.80 2.85 2.89 120 1.99 2.27 2.43 2.54 2.62 2.68 2.74 2.79 2.83 240 1.97 2.25 2.41 2.51 2.59 2.66 2.71 2.76 2.80 480 1.97 2.25 2.40 2.50 2.58 2.64 2.70 2.74 2.78 Infinite 1.96 2.24 2.39 2.49 2.57 2.63 2.68 2.73 2.77

Note, infinitely, these intervals limit to the 97.5% quantile of the distribution of the maximum values sampled from a Normal Distribution with m results (for example, 2.73 for m = 8). This shall be demonstrated with an R-script.

R-script: Distribution of maximum values of a sample with size n taken from a Normal Distribution with  = 0 and  = 1 n=8 loop=500000 Maximum=c(rep(0,loop)) for(i in 1:loop) { x=rnorm(n, mean = 0, sd = 1) Maximum[i]=max(x) } quantile(Maximum, probs = c(0.025,0.5,0.975)) hist(Maximum,breaks=60,xlim=c(-2,5))

Quantiles 2.5% 50% 97.5% 0.3340892 1.3849190Histogram 2.7326133 of Maximum 0 0 0 0 y 3 c n 0 e 0 u 0 q 5 e r 1 F 0

-2 -1 0 1 2 3 4 5

Maximum

The distribution is slightly skewed and its 97.5% quantile is at 2.73 (also consult the Table above).

Prediction interval, high number of next m The more “next” results we want to be included in the prediction interval, the greater the k-factor becomes (see Table).

Odeh RE. Two-sided prediction intervals to contain at least k out of m future observations from a normal distribution. Technometrics 1990;32:203-16.

Table 1.A.0.1: Factors for (100)% two-sided prediction intervals, based on a sample of size n, to contain all of m out of m additional observations m  = .95 n 10 15 20 30 40 50 60 80 100 8 4.02 4.28 4.46 4.70 4.88 5.01 5.11 5.28 5.40 9 3.85 4.09 4.25 4.49 4.65 4.77 4.87 5.02 5.14 10 3.72 3.94 4.10 4.32 4.47 4.59 4.68 4.83 4.94 11 3.62 3.83 3.98 4.19 4.34 4.45 4.54 4.68 4.79 12 3.53 3.74 3.89 4.09 4.23 4.34 4.42 4.56 4.66 15 3.36 3.56 3.69 3.87 4.00 4.10 4.18 4.30 4.40 20 3.21 3.38 3.50 3.67 3.79 3.88 3.95 4.06 4.15 25 3.12 3.28 3.40 3.56 3.67 3.75 3.82 3.93 4.01 30 3.06 3.22 3.33 3.48 3.59 3.67 3.73 3.84 3.92 40 2.99 3.14 3.25 3.39 3.49 3.57 3.63 3.73 3.80 60 2.93 3.07 3.17 3.30 3.40 3.47 3.53 3.62 3.69 120 2.86 3.00 3.09 3.22 3.31 3.38 3.43 3.52 3.58 240 2.83 2.96 3.05 3.18 3.26 3.33 3.38 3.46 3.53 480 2.81 2.94 3.03 3.16 3.24 3.31 3.36 3.44 3.50 Infinite 2.80 2.93 3.02 3.14 3.22 3.28 3.33 3.41 3.47

For high m, we therefore may consider another interval, the Tolerance Interval.

Tolerance interval Assume, you have estimated mean (xbar) and standard deviation (s) from n results (for example, 20). Now you want to know in which interval you will find a certain percentage (100-) % of future measurements with a certain prabability (100-)% (or confidence). For example, a tolerance interval with 95% confidence for 90% of distribution implies 95% of intervals cover at least 90% of distribution.

A tolerance interval can also be viewed as a confidence interval for the pth centile (κp = μ + Zp*σ, for example, z = 1.65, 1.96). The definition of a tolerance interval, unfortunately, is not used consistently in the literature. Also, several oversimplified formulas exist for its calculation (t/sqrt(n)*sqrt(1+nz2(Cn2-1)) * s; t*sqrt(1/n+z2*Cn2*1/2n) * s). The publication “Chakraborti S, Li J. Confidence interval estimation of a normal percentile. Amer Statistician 2007;61:331-6” gives a very good insight into the statistics of the confidence interval of a percentile/tolerance interval.

Other important references are: A. Wald, J. Wolfowitz (1946), Tolerance limits for a normal distribution, Annals of Mathematical Statistics 17, 208-215. R. E. Odeh, D. B. Owen (1980), Tables for Normal Tolerance Limits, Sampling Plans, and Screening, Marcel Dekker, New York.

The 2 different approaches can be expressed in the formulas

TI = xbar + kT*s p= xbar + Cn*Zp*s ± CI (Cn is a factor needed because s is a biased estimator; see Chakraborti) Note: finally, one arrives at the same numbers!

Calculation of kT

The calculation of kT requires values of the non-central t-distribution. Most easily, the calculations can be performed with “R” and the package {tolerance} installed (http://cran.r- project.org/web/packages/tolerance/tolerance.pdf). Otherwise, tables should be consulted. Note, the package {spc} gives slightly different values; {tolerance} with “WBE” (Weissberg- Beatty) is preferred by myself because it gives values consistent with ISO 3207 and many other documents.

Howe, W. G. (1969), Two-Sided Tolerance Limits for Normal Populations - Some Improvements, Journal of the American Statistical Association, 64, 610–620. Weissberg, A. and Beatty, G. (1969), Tables of Tolerance Limit Factors for Normal Distributions, Technometrics, 2, 483–500.

R-script K.factor(n = seq(5,20,1), alpha = 0.05, P = 0.95, side = 2, method = "WBE") (ignore the warning messages). n = 5 6 7 8 9 10 11 12 5.08 4.41 4.01 3.73 3.53 3.38 3.26 3.16 n = 13 14 15 16 17 18 19 20 3.08 3.01 2.95 2.90 2.86 2.82 2.78 2.75

Note, the formulas given in http://www.itl.nist.gov/div898/handbook/prc/section2/prc263.htm work well for the 2-sided case, but not so good for the 1-sided case.

Other useful resources Statistical Intervals: A Guide for Practitioners. Gerald J. Hahn and William Q. Meeker. John Wiley & Sons, 1991. Software related to the book: http://www.public.iastate.edu/~wqmeeker/stint.html Note: the prediction interval (npik) is an approximation, only! http://www.public.iastate.edu/~wqmeeker/anonymous/Papers/HahnMeekerTAS1993.pdf

Applications (for the medical laboratory)

Confidence intervals Confidence intervals are generally applied and relate to statistical null-hypothesis significance testing.

Prediction intervals Prediction intervals are seldom used in the medical laboratory, except in regression analysis. However, they are typically not utilized. One exception is their application for commutability testing in the CLSI EP14 document.

Tolerance intervals Tolerance intervals are applied for the limit of the blank (LoB), the Bland-Altman limits of agreement, and for the reference interval (also IQC limits). Typically, however, they are not named tolerance intervals, but rather seen as confidence intervals of percentiles.