XIX IMEKO World Congress Fundamental and Applied Metrology September 6−11, 2009, Lisbon, Portugal

DETERMINING THE 95% CONFIDENCE INTERVAL OF ARBITRARY NON-GAUSSIAN DISTRIBUTIONS

France Pavlovčič, Janez Nastran, David Nedeljković

University of Ljubljana, Faculty of Electrical Engineering, Ljubljana, Slovenia E-mail: [email protected], [email protected], [email protected]

Abstract − Measurements are nowadays permanent at- drift, the time drift, and the resolution, all have the rectan- tendants of scientific research, health, medical care and gular distribution. Sometimes also the A-type uncertainties treatment, industrial development, safety and even global do not match with the as they are result economy. All of them depend on accurate measurements of regulated quantities, or are affected by the resolution of and tests, and many of these fields are under the legal me- measuring device. In the latter case the probability distribu- trology because of their severity. How the measurement is tion consists of two Dirac functions, one at the lower meas- accurate, is expressed by uncertainty, which is obtained by urement reading and the other at the upper measurement multiplication of standard deviations by coverage factors to reading regarding the resolution. If the distribution is un- increase trustfulness in the measured results. These coverage known the coverage factor is calculated from Chebyshev's factors depend on degree of freedom, which is the function inequality [2] and is 4.472 for the 95% confidence interval. of the number of implied repetitions of measurements, and The uncertainty, the extended or the standard one, does not therefore the reliability of the results is increased. The stan- stand by itself, but contributes its portion to a combined un- dard coverage factor is 1.96 for normal (Gaussian) distribu- certainty. The , which corresponds to tions or near-Gaussian distributions, and the obtained ex- the combined uncertainty, is the convolution of the contrib- panded uncertainty has the 95% statistical probability. In uting probability distributions. The convolution of two rec- general, it is not possible to achieve the 95% confidence in- tangular distributions gives the trapezoidal distribution, or in terval by using the standard coverage factor 1.96, neverthe- some cases the , and the next rectan- less of the degree of freedom. The present paper describes gular distribution convoluted to the trapezoidal or to the tri- the method of estimating the expanded uncertainty by an al- angular distribution results in the distribution with the gorithm of this model based on the 95% confidence interval square dependent tails, and the further rectangular distribu- of any probability distribution of any shape, dealing with the tions lead to the distribution with the polynomial dependent A-type or the B-type uncertainty. Furthermore the coverage tails – an arrow-point shape, as defined in this paper. By factor is determined due to the 95% confidence interval of further convolutions, the resulted distribution tends toward the actual probability distribution. The algorithm success- the normal distribution. Nevertheless, there are some not fully copes with adding two or more uncertainties with "well-behaved probability distributions" as quoted by the mathematical properties of sums, and is established in ac- standard [1] to cause the coverage probability of less than cordance with the standards and guides. The model is intro- 95% by using standard coverage factor. Hence, very often duced in procedures carried out in the calibration laboratory. we are to deal with distributions with the large probability around the mean values of measured quantity with some ex- Keywords: coverage factor, distribution shape coeffi- cessive, but still reliable values. The tails of such distribu- cient, uncertainty. tions are fatter than the tails of the normal distribution and the coverage factor must be greater than 1.96 to achieve the 1. BASIC INFORMATION 95% confidence interval. There are several sources of uncertainties with the rec- The expanded uncertainty is calculated according to the tangular probability distributions, and when combined the standard EA-4/02 [1] as the multiplication of the standard resulted probability is either trapezoidal, triangular. The coverage factor 1.96 with the standard uncertainty when the rectangular, trapezoidal and triangular probability are dis- degree of freedom is approaching to infinite value. This cussed in the standard EA-4/02 [1], and further on U-shaped standard also points out the necessity to determine the con- distribution is dealt in NIS3003 [3] used with sinus wave fidence interval of the 95% coverage probability of any dis- measuring signal, but the other distributions and further tribution although it is not normal or Gaussian and further- convolutions of these distributions are rarely described in more it can be far from being normal that is being non- literature. There is an algorithm of combining the normal Gaussian. To see the problem, we must be aware of dealing distributions and the rectangular distributions only, de- with several kinds of distributions not only with the normal scribed in the literature [4]. The symmetrical impulse meas- distribution. Namely, the distributions of the B-type uncer- uring signal is very common in measuring systems, for in- tainties are mostly not normal, for instance the temperature stance the measurement of the contact resistance, the tem- perature measurement of the resistance temperature sensors with the DC current and several measurements where the in- 5.0 fluence of hysteresis is being avoided. This measuring signal A gives the symmetrical Dirac shaped distribution. 4.0 Gaussian

2. THE CONTRIBUTING DISTRIBUTIONS 3.0 95%

The measured signal is continuous function depended on 2.0 Gaussian one independent variable, such as time or a counter. Its range has the supremum and the infimum, which are the 1.0 P <95% EA-4/02 NIS 3003 trapezium bounds of the domain of definition of the corresponding shaped U shaped Coverage factor (1.96×C) over standard deviation standard over (1.96×C) factor Coverage distribution. The upper and the lower bounds are finite val- 0.0 012345678unknown arrow triangular rectangular V shaped U shaped Dirac shaped shaped ues. Amplitude A is defined as the maximum of the absolute Distributions values of the difference between the mean value of the measured signal throughout its whole definition interval and Fig. 1: The cover factors of several distributions and statistical the lower and the upper bounds respectively. The corre- confidence of the acquired data interval with the following legend: sponding kind of distributions follows by the following A ... the cover factors of the 100% confidence level deter- equation: mined by (1) and (4), B ... the cover factors of the 95% confidence level using the +∞ + A presented model and so far calculated by (9), ∫∫p(X ) ⋅ dX = p(X ) ⋅ dX =1 (1) C ... the cover factors of the 95% confidence level calcu- −∞ − A lated by integration (2).

We are looking for the coverage factor ˆ of any distribu- K There is no problem to determine the coverage factor of tion, applied to the standard deviation, which gives the 95% the probability distribution defined by the probability den- confidence interval as follows: sity p(X) described by the analytic function, by the tabled +Kˆ ⋅σ values or by the geometrical definition. But, determining the p(X ) ⋅ dX ≥ 0.95 (2) coverage factor out of statistically acquired data, we have to ∫ establish the model, which solution gives results within the −Kˆ ⋅σ mentioned area with the probability levels from 95% up to at the infinite degree of freedom. 100%. The solution given by the presented model is one of There are upper limits of this coverage factor as follows: many possible solutions, and it is shown in Fig. 1 as the • any, even unknown distribution corresponds to Cheby- function (B). shev's inequality [2], therefore: 3. THE KURTOSIS AND MODELLING THE 1 COVERAGE FACTOR Kˆ ≤ = 20 (3) 1− P P=0.95 The kurtosis is the parameter of the descriptive , which gives information about the probability distributions • any, even unknown distribution with the finite upper and of acquired data that are created by our measurements. It is lower bounds corresponds to (1), therefore according to the classical measure of non-gaussianity, and it expresses (1) and (2), the upper limit of the coverage factor is: the similarity to the normal or Gaussian distribution. The A distribution shape is quantified by it, but the mapping of the Kˆ ≤ (4) set of the shapes to the set of their kurtosis numerical values σ is surjection. There is no rule to get the distribution shape where the equality is present only, when the cover factor of out of the kurtosis. From the kurtosis, it can be concluded the 100% confidence level is considered. only that: The coverage factor of each known distribution is cal- • a certain distribution is peaked around its mean and have culated by using the (2). The results of the calculation for the fat tails – leptokurtic distributions; the dealt distributions are presented in Fig. 1 as two func- • it is flat (could be the rectangular distribution) or even tions: the first one is the function (C), which is calculated concave (could be the U-shaped distribution) with the for the 95% probability level by (2), and the second one is thin tails or without them – platykurtic distributions; the function (A) for the 100% probability level calculated by • it could be very similar to the normal distribution – (1) and (4). The upper limits are also shown in this figure: mesokurtic distributions. the maximum of the function (C) for the 95% probability The kurtosis is the fourth standardized k about level of an unknown distribution due to Chebyshev's ine- 4 the mean, and is defined as the quotient of the fourth mo- quality due to (3). The functions (A) and (C) in Fig. 1 make ment m about the mean and the fourth power of the stan- the boundary of the area of the probability levels from 95% 4 up to 100%. There is also the position of Gaussian distribu- dard deviation σ, and it is: tion marked in Fig. 1. m 4. THE CONVOLUTION AND THE ADDITION k = 4 (5) ALGORITHM 4 σ 4 The kurtosis of the normal distribution is taken as 2.45, When combining several probability distributions in un- and the skewness of the normal distribution or any symmet- certainty calculations the resulted probability distribution is rical distributions is zero. the convolution of all participant distributions. The convo- The basis of the present modelling of the coverage factor lution of two probability distributions is: of the 95% confidence interval is the kurtosis, because it is the statistical parameter that quantifies the shape of the p12 (X ) = p1(X )⊗ p2 (X ) = analysed probability distribution. The coverage factor, in- +∞ (10) cluding all contributed coefficients of the presented model, = p X −τ ⋅ p τ ⋅dτ ∫ 1()()2 is basically the square root of the ratio between the kurtosis −∞ k4 of the analyzed distribution and the kurtosis of the normal distribution, corrected by the empirical coefficient κ and and further on for the N convoluted distribution written just multiplied with the coverage factor of the normal distribu- symbolically as: tion at the infinite degree of freedom: (11) pΣ (X ) = p1 (X )⊗K⊗ pi (X )⊗K⊗ pN (X ) ⎛ k ⎞ Each distribution contributes the amplitude A , the stan- ˆ 4 (6) i Kbasic = 1.96⋅⎜κ ⋅ ⎟ ⎜ ⎟ dard deviation σi about the mean and the shape coefficient ⎝ 2.45 ⎠ Ci to the resulting shape coefficient CΣ that is obtained by where the empirical coefficient κ is a function of the abso- the following addition algorithm: lute value of skewness k3, which is determined empirically:

1 ⎛ N N −1 N ⎞ ⎜ ⎛ ⎞ N ⎟ κ =1 ⇐ k ≤ C 2 ⋅σ 4 + 2.45⋅ ⎜κ ⋅σ 2 ⋅ κ ⋅σ 2 ⎟ 3 ⎜ ∑ i i ∑∑⎜ i i j j ⎟ ∑ Ai ⎟ 3 ⎜ i=1 i=+11⎝ j=i ⎠ i=1 20 ⎟ (7) CΣ = min , , N 2 N ⎜ ⎛ ⎞ 2 1.96 ⎟ 2 1.96⋅ σ 1 ⎜ ⎜∑σ i ⎟ ∑ i ⎟ κ >1 ⇐ k > ⎜ ⎝ i=1 ⎠ i=1 ⎟ 3 ⎝ ⎠ 3 (12) The empirical coefficient is necessary to correct the ba- The addition of the shape coefficients is commutative sic coverage factors of skewed distributions with the finite and associative and the resulted shape coefficient is a mem- bounds of their domain. This is not the case with the normal ber of the same set of values as the participant shape coeffi- distribution or near-normal distributions or any approxi- cients in the evaluating process, which all are the necessary mately symmetrical distribution, where this empirical coeffi- mathematic conditions for the applied methods of deter- cient is unit. mining the combined uncertainties as it is prescribed by the Due to (6) the coverage factor of the probability distribu- standard [7] as universality, internal consistency and trans- tion with the finite bounds of its domain actually is: ferability. ˆ (8) The expanded uncertainty is obtained by multiplying the K = K(ν ) ⋅C standard deviation or the combined uncertainty, which is and further on taking into account the upper limits of the appropriate, by the coverage factor [1] and the shape coeffi- coverage factor due to (3) and (4) the shape coefficient C of cient, so that the expanded uncertainty is estimated to have the probability distribution is: at least the 95% confidence level: U = K(ν ) ⋅C ⋅σ ⎛ m A 20 ⎞ P=95% C = min⎜κ ⋅ 4 , , ⎟ (9) (13) ⎜ 4 ⎟ U = K(ν ) ⋅C ⋅u ⎝ 2.45⋅σ 1.96⋅σ 1.96 ⎠ c P=95% eff Σ c

This coverage factor at the infinite degree of freedom is therefore the expanded uncertainty intervals ±U or ±Uc graphically shown as the function (B) in Fig. 1. Its values about the mean of the measured or calculated results are the are in the lower range of the area indicating the confidence 95% confidence interval. interval from 95% up to 100%, which is very good. The ad- vantage of this coverage factor is, that it consists of two 5. THE CASE STUDY multiplicands as in (8): the first one – K(ν) is dependent on degree of freedom, as it is generally known as the coverage Although the presented method was used in the calibra- factor, and the other - C depends on the shape of the prob- tion laboratory and it was considered as very effective, the ability distribution, mainly on the kurtosis and we named it authors decide to demonstrate the usage of this method in a as the shape coefficient. very specialized scientific field of electron impacts to gas So far, the shape coefficient is defined, but if it should be molecules and to steady material such as an electrode, and useful, a mathematic operation(s) between several shape co- further on the voltages needed for ionization by these two efficients must be established. kinds of the impacts were determined probabilistically with the appropriate expanded uncertainties and further on the re- terval is determined too. These data, which are the results of sultant expanded uncertainty. Nevertheless, what the scien- the addition algorithm, are presented in the fifth column of tific problem was, we have two probability density distribu- Table 1. tions named as the first and the second probability distribu- So far, the fourth and the fifth columns of Table 1 repre- tion in Fig. 2. The domains of these two distributions were sent the data of the same distribution, namely of the convo- to be added together, hence these two distributions were lution probability distribution, and hence are comparable, convoluted and a resultant distribution, also in Fig. 2, is although the data in the fourth column are obtained by the named as the convolution probability distribution. convolution integral and the data of the fifth column by the addition algorithm. There is quite a large difference between 100% 25% the shape coefficients, but it is not significant in this case to the combined uncertainty and its confidence level. As we 1st probability 2nd probability 80% 20% distribution distribution see, the confidence levels attained by the convolution and by the addition algorithm match very well, but the latter one is

60% 15% a little bit greater, but it belongs also to the wider uncer- convolution probability tainty interval. In Fig. 2 are seen the intervals, which are distribution

40% 10% distributions outside the uncertainty intervals with the confidence levels probability distribution probability

st for the fist, the second and the convolution distributions, as 1

20% 5%

probability and Convolution prbability prbability Convolution and probability stated in Table 1. But the positions of minima and maxima nd 2 of these distributions domains in view of the uncertainty in-

0% 0% tervals are described also in Table 1. 0 5 10 15 20 25 30 35 40 45 Voltage - [V] The uncertainty intervals of the 95% confidence level is established by presented method with the shape coefficients,

st nd obtained by kurtosis, which is the fourth standardized mo- Fig. 2: The 1 , 2 and convolution probability densities with the ment about the mean, without calculating the probability intervals oudside 95% conidence interval. integrals as in (2), or previously calculating the convolution integral of the participatory probability distributions as in For the first and the second distributions, and further on, (10) when the sum of the uncertainties are required. as their convolution is practically carried out, also for the convolution distribution, the shape coefficients C are calcu- lated by (9) and the uncertainty intervals on the basis 6. CONCLUSIONS U = 1.96⋅C⋅σ are established for all three distributions, and The presented method of evaluating the expanded un- the confidence levels of these uncertainty intervals are ob- certainty of the measurand on the basis of the 95% confi- tained by the integration of these probability density distri- dence interval has the following features: butions throughout the uncertainty intervals. The results are shown in the second, third and fourth column of Table 1. • it determinines uncertainty intervals of the 95% confi- dence level of arbitrary non-Gausian probability distri- Table 1: Comparison of results achieved by convolution and by butions without resolving its borders by the integration addition algorithm. of their densities or calculating convolution integrals when adding them together; • it is universal [7], because this algorithm is applicable to 1st 2nd convolu- addition distribution distribution tion algorithm all kinds of measurements, to the A and B-type of the uncertainty evaluation and to all type of input data dis- C 0.948 0.918 0.945 0.993 tribution; confidence 96.18% 96.12% 95.41% 96.93% • it is internally consistent [7], which mathematically of ±U means being commutative and associative, so that com- mean 5.754 10.496 18.672 18.597 bined uncertainty is independent of grouping and de- -1.96·C·σ composing the contributing components; mean 9.663 19.241 28.631 28.905 • it is transferable [7], which mathematically means that mean 13.573 27.987 38.591 39.212 +1.96·C·σ the resulted shape coefficient and the participant shape minimum coefficients are the members of the same set of values or in view of outside inside outside outside are fitting the same function, so the one result can be di- ±U rectly used as a component in evaluating the uncertainty maximum of another measuring process; in view of inside outside outside outside • the convolution of many normal distributions gives nor- ±U mal distribution and so does the resulted shape coeffi- cient; further on, the convolution of great number of Further on, the shape coefficient CΣ is determined by ad- whatever distributions leads to mesokurtic distribution dition algorithm (12) of the presented method and also the and even to normal distribution and so also does the re- uncertainty interval ±Uc defined by (13) on the basis of sulted shape coefficient, hence the K(νeff) = 1.96 are recalculated for the convolution probabil- is met by this method [1]; ity distribution. The confidence level of this uncertainty in- • the expanded uncertainty depends on its effective de- tional Workshop: From Data Acquisition to Data Processing grees of freedom as in (13) so that the proper reliability and Retrieval, Republic of Slovenia, Ministry of Education, is achieved [1]; Science and Sport, Metrology Institute and European The- matic Network, Advanced Mathematical and Computational • the expanded uncertainty estimated by this method, as in Tools in Metrology, Ljubljana, Sep. 13-15, 2004. (13), takes into account the effective degree of freedom of the output estimates and the non-normality or non- [5] F. Pavlovčič, D. Groselj: An establishment and a propaga- gaussianity of the probability distributions and so far tion of a measuring uncertainty of a medium by processing meets regulations [1] about the 95% confidence interval. Gaussian random signal (in Slovenian); V: ZAJC, Baldomir (ed.). Zbornik šeste Elektrotehniške in računalniške konfe- rence ERK '97, 25. - 27. september 1997, Portorož, REFERENCES Slovenija. Ljubljana: IEEE Region 8, IEEE Slovenian sec- tion, 1997, pp. 415-418. [1] EA-4/02: Expression of the Uncertainty of Measurement in Calibration, European co-operation for Accreditation, Publi- [6] F. Pavlovčič: A dynamic modelling of measuring uncertainty cation Reference, December 1999. of industrial and environmental media using a signal proc- essing technique; In: ILIĆ, Damir (ed.), BORŠIĆ, Mladen [2] G. A. Korn, T. M. Korn: Mathematical Handbook for scien- (ed.), BUTORAC, Josip (ed.). XVII IMEKO World Con- tists and engineers, McGraw-Hill Inc., 2nd edition, 1968. gress, June 22-27, 2003, Dubrovnik, Croatia. Metrology in the 3rd millennium: proceedings. Zagreb: HMD - Croatian [3] NIS3003: The expression of Uncertainty and Confidence in Metrology Society, 2003, p. 1663. Measurement for Calibrations, NAMAS Executive, National Physical Laboratory, Teddington, Middlesex, TW110LW, [7] Guide to the expression of uncertainty in measurement, Inter- England, Edition 8, May 1995. national Organisation for Standardisation, 1st edition, 1995.

[4] M. J. Korczynski, A. Hetman: Direct method of coverage in- terval calculation: advantages and disadvantages, Interna-