SANDIA REPORT SAND2001-0417 Unlimited Release Printed November 2002 Latin Hypercube Sampling and the Propagation of Uncertainty in Analyses of Complex Systems J. C. Helton and F. J. Davis Prepared by Sandia National Laboratories Albuquerque, New Mexico 87185 and Livermore, California 94550 Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy under Contract DE-AC04-94AL85000. Approved for public release; further dissemination unlimited. Issued by Sandia National Laboratories, operated for the United States Department of Energy by Sandia Corporation. NOTICE: This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government, nor any agency thereof, nor any of their employees, nor any of their contractors, subcontractors, or their employees, make any warranty, express or implied, or assume any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represent that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government, any agency thereof, or any of their contractors or subcontractors. The views and opinions expressed herein do not necessarily state or reflect those of the United States Government, any agency thereof, or any of their contractors. Printed in the United States of America. This report has been reproduced directly from the best available copy. Available to DOE and DOE contractors from U.S. Department of Energy Office of Scientific and Technical Information P.O. Box 62 Oak Ridge, TN 37831 Telephone: (865)576-8401 Facsimile: (865)576-5728 E-Mail: [email protected] Online ordering: http://www.doe.gov/bridge Available to the public from U.S. Department of Commerce National Technical Information Service 5285 Port Royal Rd Springfield, VA 22161 Telephone: (800)553-6847 Facsimile: (703)605-6900 E-Mail: [email protected] Online order: http://www.ntis.gov/ordering.htm SAND200 1-0417 Unlimited Release Printed November 2002 Latin Hypercube Sampling and the Propagation of Uncertainty in Analyses of Complex Systems Jon C. Helton Program Development and Environmental Decisions Department Sandia National Laboratories P.O.Box 5800 Albuquerque, NM 87 185-0779 Freddie J. Davis Department of Mathematics, Physical Sciences, and Engineering Technology West Texas A&M University Canyon, TX 79016-0001 Abstract The following techniques for uncertainty and sensitivity analysis are briefly summarized: Monte Carlo analysis, differential analysis, response surface methodology, Fourier amplitude sensitivity test, Sobol’ variance decomposition, and fast probability integration. Desirable features of Monte Carlo analysis in conjunction with Latin hypercube sampling are described in discussions of the following topics: (i) properties of random, stratified and Latin hypercube sampling, (ii) comparisons of random and Latin hypercube sampling, (iii) operations involving Latin hypercube sampling (i.e., correlation control, reweighting of samples to incorporate changed distributions, replicated sampling to test reproducibility of results), (iv) uncertainty analysis (i.e,, cumulative distribution functions, complementary cumulative distribution functions, box plots), (v) sensitivity analysis (i.e., scatterplots, regression analysis, correlation analysis, rank transformations, searches for nonrandom patterns), and (vi) analyses involving stochastic (i.e., aleatory) and subjective (i.e., epistemic) uncertainty. Contents 1 . Introduction ............................................................................................................................................................ 1 2. Techniques for Uncertainty and Sensitivity Analysis ............................................................................................. 3 2.1 Monte Carlo Analysis................................................................................................................................... ,.3 2.2 Differential Analysis ...................................................................................................................................... 4 2.3 Response Surface Methodology..................................................................................................................... 6 2.4 Fourier Amplitude Sensitivity Test (FAST) and Sobol’ Variance Decomposition ....................................... 6 2.5 Fast Probability Integration............................................................................................................................ 8 2.6 Comparison of Techniques............................................................................................................................. 8 3. Random. Stratified and Latin Hypercube Sampling............................................................................................. 11 3.1 Description of Sampling Techniques .......................................................................................................... 11 3.2 Properties of Sampling Techniques............................................................................................................. 15 3.3 Historical Development of Latin Hypercube Sampling................................................................................ 18 4. Comparison of Random and Latin Hypercube Sampling...................................................................................... 21 4.1 Monotonic Function ..................................................................................................................................... 21 4.2 Nonmonotonic Function............................................................................................................................... 21 5 . Operations Involving Latin Hypercube Sampling................................................................................................ 25 5.1 Correlation Control ...................................................................................................................................... 25 5.2 Reweighting of Samples........ .” ...................................................................................................................... 27 5.3 Replication of Samples ................................................................................................................................ 29 6. Example Uncertainty and Sensitivity Analysis .................................................................................................... 31 6.1 Analysis Problem ......................................................................................................................................... 31 6.2 Uncertainty Analysis ................................................................................................................................... 31 6.3 Examination of Scatterplots ......................................................................................................................... 34 6.4 Regression-Based Techniques...................................................................................................................... 35 6.5 Searches for Nonrandom Patterns ................................................................................................................ 37 7. Uncertainty in Analyses for Complex Systems (adapted from Ref . 275. Chapt. 10) ........................................... 43 7.1 Stochastic and Subjective Uncertainty ........................................................................................................ 43 7.2 Performance Assessment for the WIPP ........................................................................................................ 43 7.3 Implementation of 1996 WIPP PA ............................................................................................................... 47 7.4 Uncertainty and Sensitivity Analysis Results in 1996 WIPP PA ................................................................. 49 8 . Discussion............................................................................................................................................................. 53 8.1 Popularity of Latin Hypercube Sampling..................................................................................................... 53 8.2 Additional Thoughts..................................................................................................................................... 55 References.................................................................................................................................................................... 57 Appendix A: On a Better Method for Selecting Input Variables ............................................................................. A-1 Appendix B: The Evolution of Latin Hypercube Sampling ...................................................................................... B-1 11 Figures 1.1. Use of CDFs and CCDFs to represent uncertainty in model predictions. .............................................................. 2 2.1. Scatterplots produced in a Monte Carlo analysis with a Latin hypercube sample of size nS