Volume 106, Number 1, January–February 2001 Journal of Research of the National Institute of Standards and Technology [J. Res. Natl. Inst. Stand. Technol. 106, 279–292 (2001)] Statistics and Measurements Volume 106 Number 1 January–February 2001 M. Carroll Croarkin For more than 50 years, the Statistical En- survey of several thematic areas. The ac- gineering Division (SED) has been in- companying examples illustrate the im- National Institute of Standards and strumental in the success of a broad spec- portance of SED in the history of statistics, Technology, trum of metrology projects at NBS/NIST. measurements and standards: calibration Gaithersburg, MD 20899-8980 This paper highlights fundamental contribu- and measurement assurance, interlaboratory tions of NBS/NIST statisticians to statis- tests, development of measurement meth- [email protected] tics and to measurement science and tech- ods, Standard Reference Materials, statisti- nology. Published methods developed by cal computing, and dissemination of SED staff, especially during the early years, measurement technology. A brief look for- endure as cornerstones of statistics not ward sketches the expanding opportunity only in metrology and standards applica- and demand for SED statisticians created tions, but as data-analytic resources used by current trends in research and devel- across all disciplines. The history of statis- opment at NIST. tics at NBS/NIST began with the forma- tion of what is now the SED. Examples from the first five decades of the SED il- Key words: calibration and measure- lustrate the critical role of the division in ment assurance; error analysis and uncer- the successful resolution of a few of the tainty; design of experiments; history of highly visible, and sometimes controversial, NBS; interlaboratory testing; measurement statistical studies of national importance. methods; standard reference materials; A review of the history of major early pub- statistical computing; uncertainty analysis. lications of the division on statistical methods, design of experiments, and error analysis and uncertainty is followed by a Available online: http://www.nist.gov/jres 1. Introduction For more than 50 years, Statistical Engineering Divi- expanded and new methods have been developed to ad- sion (SED) staff have played a critical role in the success dress recent challenges and take advantage of the statis- of a broad spectrum of metrology projects at NBS/ tical literature and the tremendous surge in statistical NIST. During this time, statistics at NBS/NIST has pro- computing capability. gressed with the constant goal of improving and charac- SED research contributions cover: quantification of terizing measurement methods. Methods and uncertainty in measurements, statistical design of exper- publications which were developed early in the life of imental investigations, monte carlo modeling, parameter the division are still cornerstones for statistical analyses estimation, stochastic modeling, exploratory data analy- and are applied across all disciplines and metrologies. sis and empirical modeling, model validation, computer Over the years, existing methods have been refined and intensive statistical methods, reliability analysis, statisti- 279 Volume 106, Number 1, January–February 2001 Journal of Research of the National Institute of Standards and Technology cal signal processing, image analysis, time series analy- sis, hypothesis testing, and quality control. Statisticians participate in the planning of experimen- tal studies and conduct rigorous uncertainty analysis of results and develop theoretical models to augment ex- perimental work done by NIST collaborators. Examples of such work include Monte Carlo simulation of physi- cal processes, such as neutron scattering, and stochastic differential modeling of aerosol particle spectrometers. Typically, SED staff develop long term relationships with collaborators in the other NIST laboratories and develop intimate knowledge of the scientific field in which they work. Here we highlight areas where SED contributes to metrology work at NIST with examples from recent collaborations along with an historical per- spective for viewing statistical contributions to metrol- ogy. 2. History 2.1 Early Days Churchill Eisenhart (see Fig. 1) came to NBS from the University of Wisconsin in 1946 when Edward Con- don, Director of NBS, resolved to establish a statistical consulting group to “substitute sound mathematical analysis for costly experimentation” [26]. As the first Chief of the Statistical Engineering Laboratory (SEL), Fig. 1. Churchill Eisenhart. he shaped the direction that statistics would take at NBS for many years and personally laid the foundation for differentiate effects caused by AD-X2 from effects due error analysis related to measurement science. to the charging line. They also insisted on a formal In its early days SEL, in its work with scientists at randomization scheme for selecting the batteries for NBS, was drawn into several interesting activities as the treatment in order to avoid conflicts in the analysis. Secretary of Commerce encouraged NBS to become After this accomplishment, there ensued a brief moment involved in outside activities. The most important of of panic when they realized that a design was also these was the controversy over battery additive AD-X2 needed for a visual test where the electrical plates would [13]. The NBS Director, A. V. Astin, had been pressured be disassembled and 45 paired comparisons would be by various senators and the additive producer to test the made of treated and untreated batteries. Fortunately, additive for its ability to improve battery performance. Joseph Cameron found a suitable incomplete block de- The statisticians, under severe time constraints, were sign, thus avoiding the risk of having to construct such responsible for recommending experimental designs for a design in the urgency of the moment [26]. testing the additive. The resulting analysis by SEL of this experiment, There were 32 batteries available for the test, and the conducted by the Electrochemistry Section, confirmed manufacturer wanted to put all 16 batteries that were to that the additive had no significant positive effect on be treated with AD-X2 on one charging line. The statis- batteries, but in what was to quickly become an interest- ticians disagreed with the manufacturer and Jack You- ing sidelight of history, the Assistant Secretary of Com- den (see Fig. 2) proposed a design with the 32 batteries merce for Domestic Affairs announced that Astin had grouped in pairs for testing on three charging lines. On not considered the “play of the marketplace” in his lines 1 and 2, both batteries of a pair were to be treated judgment and relieved him as Director of NBS. Eventu- with AD-X2 or both were to be untreated. On line 3, ally, the National Academy of Sciences was called in to there was one treated and one untreated battery in each review NBS’s work, which was labeled first rate, and pair. The statisticians were satisfied that this design for Astin was reinstated [24]. testing the electrical performance of the batteries could 280 Volume 106, Number 1, January–February 2001 Journal of Research of the National Institute of Standards and Technology was established to address the number one cause of fire deaths in this country, namely, cigarette ignition of up- holstered furniture and bedding. Extensive testing by federal, private, and industrial laboratories of lit cigarettes on furniture mock-ups resulted in heated dis- cussions as might be expected when an industry is fac- ing potential regulation. NIST was asked to intervene, and Eberhardt led in the evaluation of screening tests conducted by the Fire Science Division of the Building and Fire Research Laboratory (BFRL). This work led to the development of two test methods and a carefully-de- signed interlaboratory evaluation of the test methods. Because standard statistical procedures for analyzing interlaboratory studies do not apply to the analysis of proportions, a methodology based on a simple model for “extra-binomial variation” [25] was developed specifi- cally for analyzing this data. The cigarette industry re- sponded with a new study that seemed to imply that fabrics selected by NIST for the study were atypical. Careful re-analysis of this data by Eberhardt demon- strated to a Technical Advisory Group made up of repre- sentatives from government, industry, consumer advo- cates, and testing laboratories the flaws in the industry’s analysis. SED collaboration with BFRL on cigarette ig- nition continues as the industry tries to respond to Con- gressional mandates for less fire-prone cigarettes and interlaboratory tests are mounted to assess progress in this direction. Fig. 2. Jack Youden. 2.3 Publications on Statistical Methods 2.2 High Visibility Studies One of the first large-scale contributions of the SEL NIST continues to rely on the Statistical Engineering to measurement science at NBS was the publication of Division for advice on experimental design, analysis, NBS Handbook 91 [23] which has guided researchers at and interpretation whenever the institution is called upon NIST for four decades in the planning and analysis of as arbiter in technical conflicts of national importance. scientific experiments. In 1954, Eisenhart was ap- In 1970, when the Selective Service System was roundly proached by the Army’s Office of Ordnance Research and properly criticized for allowing birthdate to bias the and asked to produce