FUNDAMENTAL THEORY OF SCIENTIFIC COMPUTER SIMULATION REVIEW GOOD FAIR AVERAGE POOR BAD NUREG/KM-0006 November 2013 Office of Nuclear Reactor Regulation AVAILABILITY OF REFERENCE MATERIALS IN NRC PUBLICATIONS NRC Reference Material Non-NRC Reference Material As of November 1999, you may electronically access Documents available from public and special technical NUREG-series publications and other NRC records at libraries include all open literature items, such as books, NRC’s Public Electronic Reading Room at journal articles, transactions, Federal Register notices, http://www.nrc.gov/reading-rm.html. Publicly released Federal and State legislation, and congressional reports. records include, to name a few, NUREG-series Such documents as theses, dissertations, foreign reports publications; Federal Register notices; applicant, and translations, and non-NRC conference proceedings licensee, and vendor documents and correspondence; may be purchased from their sponsoring organization. NRC correspondence and internal memoranda; bulletins and information notices; inspection and investigative Copies of industry codes and standards used in a reports; licensee event reports; and Commission papers substantive manner in the NRC regulatory process are and their attachments. maintained at— The NRC Technical Library NRC publications in the NUREG series, NRC Two White Flint North regulations, and Title 10, “Energy,” in the Code of 11545 Rockville Pike Federal Regulations may also be purchased from one Rockville, MD 20852–2738 of these two sources. 1. The Superintendent of Documents These standards are available in the library for reference U.S. Government Printing Office use by the public. Codes and standards are usually Mail Stop SSOP copyrighted and may be purchased from the originating Washington, DC 20402–0001 organization or, if they are American National Standards, Internet: bookstore.gpo.gov from— Telephone: 202-512-1800 American National Standards Institute nd Fax: 202-512-2250 11 West 42 Street 2. The National Technical Information Service New York, NY 10036–8002 Springfield, VA 22161–0002 www.ansi.org www.ntis.gov 212–642–4900 1–800–553–6847 or, locally, 703–605–6000 Legally binding regulatory requirements are stated only A single copy of each NRC draft report for comment is in laws; NRC regulations; licenses, including technical available free, to the extent of supply, upon written specifications; or orders, not in NUREG-series request as follows: publications. The views expressed in contractor- Address: U.S. Nuclear Regulatory Commission prepared publications in this series are not necessarily Office of Administration those of the NRC. Publications Branch The NUREG series comprises (1) technical and Washington, DC 20555-0001 administrative reports and books prepared by the staff E-mail: [email protected] (NUREG–XXXX) or agency contractors (NUREG/CR– Facsimile: 301–415–2289 XXXX), (2) proceedings of conferences (NUREG/CP– XXXX), (3) reports resulting from international Some publications in the NUREG series that are agreements (NUREG/IA–XXXX), (4) brochures posted at NRC’s Web site address (NUREG/BR–XXXX), and (5) compilations of legal http://www.nrc.gov/reading-rm/doc-collections/nuregs decisions and orders of the Commission and Atomic and are updated periodically and may differ from the last Safety Licensing Boards and of Directors’ decisions printed version. Although references to material found on under Section 2.206 of NRC’s regulations (NUREG– a Web site bear the date the material was accessed, the 0750). material available on the date cited may subsequently be DISCLAIMER: This report was prepared as an account removed from the site. of work sponsored by an agency of the U.S. Government. Neither the U.S. Government nor any agency thereof, nor any employee, makes any warranty, expressed or implied, or assumes any legal liability or responsibility for any third party’s use, or the results of such use, of any information, apparatus, product, or process disclosed in this publication, or represents that its use by such third party would not infringe privately owned rights. NUREG/KM-0006 Fundamental Theory of Scientific Computer Simulation Review Manuscript Completed: July 2013 Date Published: November 2013 Prepared by: Joshua Stephen Kaizer Office of Nuclear Reactor Regulation FUNDAMENTAL THEORY OF SCIENTIFIC COMPUTER SIMULATION REVIEW By Joshua Stephen Kaizer Dissertation submitted to the Faculty of the Graduate School of the University of Maryland, College Park, in partial fulfillment of the requirements for the degree of Doctor of Philosophy 2013 Advisory Committee: Professor Marino di Marzo, Chair Professor James Baeder, Deans Representative Professor Kenneth Kiger Professor Gary Pertmer Dr. Ralph Landry Dr. William Oberkampf Abstract “How much work is required to trust the results of a scientific computer simulation when human lives are at stake?” For many, this question is purely academic. But for those dealing with high consequence simulations, such as the simulations performed to demonstrate the safety of a nuclear power plant, the question is very pertinent and answering it is the domain of scientific computer simulation review. While simulation review has been performed for many years, it is rarely seen as a field of study in its own right. Consequently, requirements are often developed for a specific simulation and a particular use. If either the simulation or the use changes, new requirements must developed. To help solve this problem, some fundamental theory of scientific computer simulation review is proposed. This fundamental theory included the generation of a basic vocabulary, a formalization of the concept of maturity, the creation of simulation component hierarchy, and the development of an assessment framework. A basic vocabulary was generated to define the many common and important concepts used in simulation review. By focusing on review in general, the resulting vocabulary captures many of the ideas important to all simulation reviews and provides a better means of discussing the trustworthiness of their results. The concept of maturity was formalized into Maturity Theory. This theory provides a detailed analysis of maturity, which was used to better understand the tools available in simulation review. The Hierarchy of Scientific Computer Simulation Components was created to capture the various components of a generic simulation and define the relationships between those components. This Hierarchy provides a methodology which can be used to organize and represent many simulations. The Theoretical/Logical Maturity Assessment Framework for Scientific Computer Simulations was developed using the concepts from Maturity Theory and the Hierarchy. This framework provides a structure which establishes the boundaries of simulation review, highlights many of the assumptions of a simulation which need to be supported if the results of the simulation are to be trusted, and establishes a method for its continually evolution. By contributing this fundamental theory, these advancements better established the field scientific computer simulation review. iii iv Table of Contents Abstract ....................................................................................................................... iii Table of Contents ........................................................................................................ v List of Figures ............................................................................................................. xi List of Tables ............................................................................................................ xiii Dedication .................................................................................................................. xv Acknowledgments ................................................................................................... xvii Nomenclature ........................................................................................................... xix 1. Introduction to Scientific Computer Simulation Review ................................ 1 1.1. What is a Scientific Computer Simulation? .................................................. 1 1.1.1. Introduction to Model and Simulation .................................................. 2 1.1.2. Introduction to Scientific Computer Simulation ................................... 3 1.2. What is Scientific Computer Simulation Review? ....................................... 5 1.2.1. The Fundamental Question of a Scientific Computer Simulation ........ 6 1.2.2. The Fundamental Question on the Requirements of a Scientific Computer Simulation ............................................................................ 7 1.2.3. Scientific Computer Simulation Review: Answering the Ultimate Question ................................................................................................ 8 1.3. History of Scientific Computer Simulation Review ..................................... 9 1.3.1. History of Scientific Computer Simulations ......................................... 9 1.3.2. Informal Scientific Computer Simulation Review ............................. 10 1.3.3. Formal Scientific Computer Simulation Review ................................ 11 1.3.4. Latest Developments in Scientific Computer Simulation Review ..... 12 1.4. Maturity Assessment in Scientific Computer Simulation Review ............. 13 1.4.1. Survey of Early Maturity Assessment Frameworks ..........................