Verification and Validation in Computational Fluid Dynamics1
Total Page:16
File Type:pdf, Size:1020Kb
SAND2002 - 0529 Unlimited Release Printed March 2002 Verification and Validation in Computational Fluid Dynamics1 William L. Oberkampf Validation and Uncertainty Estimation Department Timothy G. Trucano Optimization and Uncertainty Estimation Department Sandia National Laboratories P. O. Box 5800 Albuquerque, New Mexico 87185 Abstract Verification and validation (V&V) are the primary means to assess accuracy and reliability in computational simulations. This paper presents an extensive review of the literature in V&V in computational fluid dynamics (CFD), discusses methods and procedures for assessing V&V, and develops a number of extensions to existing ideas. The review of the development of V&V terminology and methodology points out the contributions from members of the operations research, statistics, and CFD communities. Fundamental issues in V&V are addressed, such as code verification versus solution verification, model validation versus solution validation, the distinction between error and uncertainty, conceptual sources of error and uncertainty, and the relationship between validation and prediction. The fundamental strategy of verification is the identification and quantification of errors in the computational model and its solution. In verification activities, the accuracy of a computational solution is primarily measured relative to two types of highly accurate solutions: analytical solutions and highly accurate numerical solutions. Methods for determining the accuracy of numerical solutions are presented and the importance of software testing during verification activities is emphasized. The fundamental strategy of 1Accepted for publication in the review journal Progress in Aerospace Sciences. 3 validation is to assess how accurately the computational results compare with the experimental data, with quantified error and uncertainty estimates for both. This strategy employs a hierarchical methodology that segregates and simplifies the physical and coupling phenomena involved in the complex engineering system of interest. A hypersonic cruise missile is used as an example of how this hierarchical structure is formulated. The discussion of validation assessment also encompasses a number of other important topics. A set of guidelines is proposed for designing and conducting validation experiments, supported by an explanation of how validation experiments are different from traditional experiments and testing. A description is given of a relatively new procedure for estimating experimental uncertainty that has proven more effective at estimating random and correlated bias errors in wind-tunnel experiments than traditional methods. Consistent with the authors’ contention that nondeterministic simulations are needed in many validation comparisons, a three-step statistical approach is offered for incorporating experimental uncertainties into the computational analysis. The discussion of validation assessment ends with the topic of validation metrics, where two sample problems are used to demonstrate how such metrics should be constructed. In the spirit of advancing the state of the art in V&V, the paper concludes with recommendations of topics for future research and with suggestions for needed changes in the implementation of V&V in production and commercial software. 4 Acknowledgements The authors sincerely thank Frederick Blottner, Gary Froehlich, and Martin Pilch of Sandia National Laboratories, Patrick Roache consultant, and Michael Hemsch of NASA/Langley Research Center for reviewing the manuscript and providing many helpful suggestions for improvement of the manuscript. We also thank Rhonda Reinert of Technically Write, Inc. for providing extensive editorial assistance during the writing of the manuscript. 5 Contents 1. Introduction................................................................................................. 8 1.1 Background........................................................................................... 8 1.2 Outline of the paper................................................................................. 10 2. Terminology and Methodology..........................................................................11 2.1 Development of terminology for verification and validation................................... 11 2.2 Contributions from fluid dynamics................................................................16 2.3 Methodology for verification.......................................................................17 2.4 Methodology for validation........................................................................ 19 3. Verification Assessment..................................................................................24 3.1 Introduction..........................................................................................24 3.2 Fundamentals of verification.......................................................................24 3.2.1 Definitions and general principles......................................................... 24 3.2.2 Developing the case for code verification.................................................27 3.2.3 Error and the verification of calculations..................................................28 3.3 Role of computational error estimation in verification testing..................................31 3.3.1 Convergence of discretizations.............................................................31 3.3.2 A priori error information.................................................................. 34 3.3.3 A posteriori error estimates................................................................ 36 3.4 Testing................................................................................................42 3.4.1 Need for verification testing................................................................42 3.4.2 Algorithm and software quality testing....................................................44 3.4.3 Algorithm testing............................................................................ 48 3.4.4 Software quality engineering...............................................................54 4. Validation Assessment....................................................................................56 4.1 Fundamentals of validation.........................................................................56 4.1.1 Validation and prediction................................................................... 56 4.1.2 Validation error and uncertainty............................................................59 4.2 Construction of a validation experiment hierarchy.............................................. 61 4.2.1 Hierarchy strategy...........................................................................61 4.2.2 Hierarchy example...........................................................................63 4.3 Guidelines for validation experiments............................................................ 67 4.4 Statistical estimation of experimental error....................................................... 74 4.5 Uncertainty quantification in computations.......................................................77 4.6 Hypothesis testing...................................................................................80 4.7 Validation metrics................................................................................... 82 4.7.1 Recommended characteristics..............................................................82 4.7.2 Validation metric example.................................................................. 83 4.7.3 Zero experimental measurement error.....................................................85 4.7.4 Random error in experimental measurements............................................ 87 5. Recommendations for Future Work and Critical Implementation Issues...........................89 References.....................................................................................................93 6 Figures 1 Phases of Modeling and Simulation and the Role of V&V............................................12 2 Verification Process........................................................................................18 3Validation Process......................................................................................... 20 4Validation Tiers.............................................................................................21 5 Demonstration of Extrapolated Error Estimation for Mixed First and Second Order Schemes... 41 6 Integrated View of Verification Assessment for CFD................................................. 45 7 Relationship of Validation to Prediction.................................................................57 8 Validation Hierarchy for a Hypersonic Cruise Missile................................................ 64 9 Validation Pyramid for a Hypersonic Cruise Missile.................................................. 66 10 Domain of Boundary Value Problem.................................................................... 84 11 Proposed Validation Metric as a Function of Relative Error.......................................... 86 12 Validation Metric as a Function of Relative Error and Data Quantity................................ 89 Table 1 Major Software Verification Activities...................................................................55 7 1. Introduction 1.1 Background During the last three or four decades, computer simulations of physical processes have been used