Tolerance Intervals Example 1: Asthma Inhaler

 How often does an asthma inhaler deliver the right dose in a puff?  Is it enough that it delivers about the right dose on average?  Need to know about individual puffs Example 2: Mental Health Testing

 It is known that the top 10% of scorers on the OQ-45 test undergoing a particular therapy are likely to deviate from the recovery track.  A therapist gives his patient the test. How does he know if his patient’s score is in this 10% ? Example 3: Releasing Land around A Uranium Enrichment Plant for Public Use

 A state would like to make some of the land surrounding a uranium enrichment plant available for public use.  Is all of the land safe for public recreational use?

How Do We Determine The Appropriate Cut-off

 Example 1: FDA sets quality criteria for drug administration  85% coverage with high confidence  If the inhaler cannot meet criteria it cannot be sold  Are the points that mark the middle 85% of the sufficient?  Example 2: Need to identify 10% that are at risk  Is the 90th for test scores sufficient?  Example 3: EPA insists that at least 95% of the land is within a safe range for uranium and uranium isotopes  Soil samples collected from the area and analyzed for uranium  Can the 95th percentile be used as the cut-off? Problem with Raw

 Would you make a decision about the of a drug based solely on the sample mean?  Know that ̅ is not accurate  Accuracy is dependent on sample size  How do you know when the mean is too far away from a target to be due to random chance?  Use hypothesis tests to account for this uncertainty  Can use confidence intervals to perform a hypothesis test  Allows a quick comparison between the critical value and sample mean What is a ?

 Interval that we are (1-α)% confident covers μ  Interval gets smaller as n increases  Provides a range for practical interpretation of the mean  Tells us nothing about individuals in the population Will A Confidence Interval Work?

 All three examples require knowledge about individuals, not averages  Example 1: Each puff matters, not the average puff  Example 2: Need to know if my patient is at risk, not the average of everyone at risk  Example 3: How do I know if all, or at least 95%, of the land is safe?  Questions all deal with individuals, not averages  Confidence intervals are not appropriate Tolerance Interval

 Like a confidence interval for individuals  Can cover a certain proportion of the population with a certain degree of confidence  Example: a 99%/95% tolerance interval will include 99% of the population with 95% confidence.  Confidence the same thing it does with confidence intervals With a 90%/95% tolerance interval we want to capture 90% of the population with 95% confidence.

If this was the population only these two guys would be left out with 95% confidence.

In example 2 these are the two guys who are likely to deviate from the recovery track Basic Calculation

 ̅

,  / 1 ,

 p is the coverage  γ is the level of confidence

2  , is the χ with n‐1 degrees of freedom that is exceeded with probability γ  All other notation is standard R Example: Uranium Enrichment Plant

 Essential to examine distribution of the data  cannot help because we are not dealing with averages or sums  This is close enough to work Test Structure

 EPA recreational user limit for uranium = 261 mg/kg

 H0: More than 99% of the land in question is less than 261 mg/kg

 Ha: Less than 99% of the land in question is less than 261 mg/kg

 Compute upper 99%/95% tolerance limit

 If the tolerance limit is greater than 261 mg/kg reject H0 R Example: Uranium Enrichment Plant 99%/95% Tolerance Interval

Confidence level

Coverage

One-sided interval because we don’t care if there is too little uranium R Example: Uranium Enrichment Plant 99%/95% Tolerance Interval

 What does it mean?  Upper tolerance limit = 133 mg/kg  EPA recreational user limit for uranium = 261 mg/kg  The area is safe for the public with respect to uranium SAS Example: Uranium Enrichment Plant data uranium; input Uranium @@ ;  PROC CAPABILITY cards ; computes many different 112 65.7 39.7 coverage intervals 99.7 64.7 39.6 94.1 63.6 39 including tolerance and 82.4 55.8 38.9 confidence intervals 77.7 53.9 37.7 75.1 51.7 30.5 73.6 48.3 27.2 71.8 46.7 21.3 67.3 ; proc capability data=uranium noprint; intervals Uranium; run; SAS Example: Uranium Enrichment Plant

• Two-sided intervals • Want a one- sided interval • Use 90% confidence to get one-side • Many different computational methods so results differ slightly from R What if the Data are Not Normally Distributed

 Non-parametric tests available  Use maximum observed value  Bootstrap  Tolerance package has intervals for other distributions Notes on Tolerance Intervals

 Research is not as developed as confidence intervals  Software typically uses other methods than the basic formula for computing tolerance intervals  There are many different computational methods just to get the basic intervals assuming normality so results may differ between software packages  Can be very useful, but it is wise to know the quirks and limitations of the intervals when using them References

 EPA (2006), Screening-Level Risk Assessment For Recreational Use Of Beaches Upper Columbia River Remedial Investigation and Feasibility Study. Retrieved from http://yosemite.epa.gov/r10/CLEANUP.NSF/82751e55bf4ef18488256ecb0083566 6/0d26a07060578af988256ecc0075d161/$FILE/Draft%20UCR%20Beach-Screen- RA.pdf  EPA, 2013, ProUCL Version 5.0.00 Technical Guide, EPA/600/R-07/041, U.S. Environmental Protection Agency, September 2013.  Gelo, O, et. al. (2014), Psychotherapy Research: Foundations, Process, and Outcome, New York, NY: Springer-Verlag.  Olsson, B. (2013) Parametric Tolerance Interval (PTI) Test for Improved Control of Delivered Dose Uniformity (DDU) in Orally Inhaled and Nasal Drug Products (OINDP), Retrieved from http://www.fda.gov/ohrms/dockets/ac/03/slides/3926S2_04_Olsson.ppt.