NUREG/CR-0033 PROCEDURES FOR ROUNDING MEASUREMENT RESULTS IN NUCLEAR MATERIALS CONTROL AND ACCOUNTING R. J. Brouns F. P. Roberts Battelle Pacific Northwest Laboratory . ' 1 8 1'2 3 4 6'' ' Prepared for U. S. Nuclear Regulatory Commission 7912200f]O NOTICE This report was prepared as an account of work sponsored by the United States Government. Neither the United States nor the United States Nuclear Regulatory Commission, nor any of their employees, nor any of their contractors, subcontractors, or their employees, makes any warranty, express or implied, nor assumes any legal liability or responsibility fer the accuracy, completeness or usefulness of any information, apparatus, pro- duct or process disclosed, nor represents that its use would not infringe privately owned rights. 1 1812 347 Available from National Technical Information Service Springfield, Virginia 22161 Price: Printed Copy $4.50 ; Microfiche $3.00 The price of this document for requesters outside of the North American Continent ;an be obtained from the National Technical Information Service. NUREG/CR-0033 PN L-2565 PROCEDURES FOR ROUNDING MEASUREMENT RESULTS IN NUCLEAR MATERIALS CONTROL AND ACCOUNTING R. J. Brouns F. P. Roberts Manuscript Completed: November 1977 Date Published: March 1978 Battelle Pacific Northwest Laboratory Battelle Boulevard ' Richland, WA 99352 . i* , Prepared for Office of Standards Development U. S. Nuclear Regulatory Commission Under Contract No. EY-76-C-06-1830 ~ A8STRACT This report defines procedures for rounding measurement results for nuclear material ontrol and accounting. Considerations for the applica- tions of these procedures are discussed. ' :. 1812 349 i . CONTENTS ABSTRACT. .... ii CONTENTS. .... iii 1.0 INTRODUCTION. .... I 2.0 ROUNDING PROCEDURE. .... 3 3.0 STATISTICAL ASPECTS OF ROU."10ING . .... 4 4.0 MINIMIZING THE EFFECTS OF ROUNDING ON COMPUTED RESULTS. ... 7 4.1 ADDITIONS AND SUBTRACTIONS . .... 7 4.2 MULTIPLICATIONS AND DIVISIONS . .... 7 4.3 POWERS AND ROOTS. .... 8 4.4 LOGARITHMS. .... 8 4.5 COMBINED OPERATIONS. .... 8 5.0 CONCLUSIONS . .... 9 6.0 REFERENCES . .... 10 ACKNOWLEDGEMENT . ..... 10 . , 1812 350 ~ iii PROCEDURES FOR ROUNDING MEASUREMENT RESULTS IN NUCLEAR MATERIALS CONTROL AND ACCOUNTING 1.0 INTRODUCTION Careful consideration of the potential for introducing error is impor- tant in establishing procedures for rounding Special Nuclear Material (SNM) measurement results. The consequences of rounding can affect some of the decisions required of SNM licensees in accordance with 10 CFR Part 70 and 10 CFR Part 73. The decisions involved are those based on measurements of SNM received, shipped, held, transferred or discarded, and on the magnitudes of MUFs, LEMUFs, shipper-receiver differences and the uncertainty of deter- minations of scrap quantities retained at the licensed facility.(a) Rounding affects two kinds of numbers that occur in the computations associated with material balances derived from measuremer.c data; they are arithmetic numbers and measurement numbers.(b) Arithmetic numbers are exact in the sense that they have fixed or " accepted" values. Examples are C 1/3, e, and n. Some arithmetic numbers cannot be expressed exactly in a finite number of decimal places, e.g. , n and 1/3. Therefore, in the decimal form they are always approximate; their degree of exactness depends on the number of digits used to express them. Thus n may be approximated by 3.14, 3.142, 3.1416, 3.141593, etc. , and 1/3 by 0.3, 0.33, 0.333 etc., depending on the number of digits desired or the size of the error that can be accepted. (a) Examples of specific sections of the Regulations in which numerical values are the basis of an action decision are 70.22(b) and (h); 70.51(c), (d), (e), (f), (g), and (h); and 70.58(e) and (g). (b)See Ku, Reference 4 or Eisenhart, Reference 1. - , -1812 351 , - . a.. ' , , I Rounding an arithmetic number results in r fixed, known error. For example, if we accepted the value of n given in 7 digits (3.141593), e n rounded to 3.1416 is in error by 0.000007 * n rounded to 3.142 is in error by 0.000407 * n rcunded to 3.14 is in errer by -0.001593. However, the error is known and the approximation to the correct value can be made as close as desired by retaining enough digits. Measurement numbers, on the other hand, are subject to uncertainty in the sense that they approximate continuous variables and are subject to the errors inherent in measurement processes. Measurement devices are usually graduated or scaled so that at least as many digits can be re- corded as are necessary to take advantage of the full sensitivity of the measurement system. These digits may not all be meaningful, in the sense that some of them may not be reproducible in repeated measurements of an item under the same conditions, i.e., such digits are in decimal places that are affected appreciably by measurement random error. Sometimes measurement numbers are rounded when they are recorded. A number may be rounded electronically by an instrument in converting analog values to digital values for readout. Computers and calculators either round or truncate (#) numbers when the limit for the number of digits that the machine can handle is reached. Sometimes measurement data are rounded by an observer, either in reading or recording the data. Pre- ferably, only the final result to be reported is rounded. For example, the reported quantity of 350 in a lot may be a rounded value but the recorded measurement values for gross weights, chemical assays and isotopic assays used to calculate it should not be rounded. Usually rounding is done to eliminate superfluous digits. Too many digits can occur when: 1) INmbers contain nonmeaningful digits; i.e., digits whose values are determined by the random error in the measurement process. (a)To drop excess digits without changing the last digit retained. , ' 2 hk 2) Numbers are combined arithmetically with other numbers having much larger relative uncertainties. As an example of the first situation, consider the data from a measurement instrument that is readable to a much smaller value of a variable than the standard deviation of the measurement process. For example, a scale readable to 0.1 g may have a weighing standard deviation of 10 g under the circumstances of the scale's use. Two examples of the second situation are: * A plant inventory consists of some types of items measured to only 1 g and other types measured to 0.01 g. The inventory in this case could be rounded to the nearest I g, although some of the individual items in the inventory would have been measured to two additional decimal places. * Two measurement systems whose data will be combined in subsequent calculations have widely different measurement precisions. For example, some SNM-bearing scrap may be weighable to a relative standard deviation of 0.1% while the assay of the material can be made to only 10%. 2.0 ROUNDING PROCEDURE A rounding interval may be defined as the range of all the values in a set of data that would be rounded to the same value. For example, if all values betweeti 13.55 and 13.65 in a data set were rounded to 13.6, the range, and hence tite rounding interval, would be 0.1. Alternatively, a rounding interval could be defined as the difference between any two consecutive rounded values in a data set. The following three-step procedure is commonly used for rounding numbers (1,3-5). (a) If the first digit to be dropped is less than five, the last digit retained is left unchenged. (b) If the first digit to be dropped is greater than five, or a five followed by some digits greater than zero, the last digit re- tained is increased by one. - ,- , 1812 353 '1 3 . .. (c) If the first digit to be dropped is a five or a five followed by only zeros, the last digit retained is rounded to the near- est even number (i."e., if the lait * digit retained is even, it is left unchanged, and if it is odd, it is increased by one). This procedure provides for a balance between rounding down and rounding up. Therefore, the overall effect of rounding is unbiased if the variation in the data is sufficiently large relative to the rounding inter- val. The provision of (c) above that rounding be only to the even digit is arbitrary; rounding to the nearest odd digit would establish the balance just as well, but rule (c) is common practice.(l' - When rounding is done in intervals of 50, 5, 0.5, etc., bias can be avoided by use of the procedure recommended in ASTM standards.(5,7) This is done by doubling the number, rounding by the rules in (a), (b) and (c) to an interval double that desired (i.e. , round to the nearest 100,10, 1.0, 0.1, etc.), and dividing the rounded value by 2. For example, to round 6025 to the nearest 50 units, multiply by 2, obtaining 12050; round to the nearest 100 by rule (c), obtaining 12000; and divide by 2, obtain- ing 6000 as the rounded value of 6025. 3.0 STATISTICAL ASPECTS OF ROUNDING The standard deviation of a set of numbers can be affected by rounding. The magnitude of the effect is generally proportional to the width of thc rounding interval, but if the rounding interval is too large, the varia- bility in the original data may be masked by the rounding. For example, if the measurement numbers 2.42, 2.43, 2.37, 2.39, and 2.41 are rounded to the first decimal, all values become 2.4 and the variance estimate is zero.