Uncertainty and Sensitivity Analysis Methods for Improving Design Robustness and Reliability by Qinxian He Bachelor of Science in Engineering, Duke University, 2008 Master of Science, Massachusetts Institute of Technology, 2010 Submitted to the Department of Aeronautics and Astronautics in partial fulfillment of the requirements for the degree of Doctor of Philosophy at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY June 2014 c Massachusetts Institute of Technology 2014. All rights reserved. Author.............................................................. Department of Aeronautics and Astronautics May 7, 2014 Certified by. Karen E. Willcox Professor of Aeronautics and Astronautics Thesis Supervisor Certified by. Ian A. Waitz Jerome C. Hunsaker Professor of Aeronautics and Astronautics Committee Member Certified by. Douglas L. Allaire Assistant Professor of Mechanical Engineering, Texas A&M University Committee Member Accepted by . Paulo C. Lozano Associate Professor of Aeronautics and Astronautics Chair, Graduate Program Committee 2 Uncertainty and Sensitivity Analysis Methods for Improving Design Robustness and Reliability by Qinxian He Submitted to the Department of Aeronautics and Astronautics on May 7, 2014, in partial fulfillment of the requirements for the degree of Doctor of Philosophy Abstract Engineering systems of the modern day are increasingly complex, often involving nu- merous components, countless mathematical models, and large, globally-distributed design teams. These features all contribute uncertainty to the system design process that, if not properly managed, can escalate into risks that seriously jeopardize the design program. In fact, recent history is replete with examples of major design set- backs due to failure to recognize and reduce risks associated with performance, cost, and schedule as they emerge during the design process. The objective of this thesis is to develop methods that help quantify, understand, and mitigate the effects of uncertainty in the design of engineering systems. The design process is viewed as a stochastic estimation problem in which the level of uncertainty in the design parameters and quantities of interest is characterized prob- abilistically, and updated through successive iterations as new information becomes available. Proposed quantitative measures of complexity and risk can be used in the design context to rigorously estimate uncertainty, and have direct implications for system robustness and reliability. New local sensitivity analysis techniques facilitate the approximation of complexity and risk in the quantities of interest resulting from modifications in the mean or variance of the design parameters. A novel complexity- based sensitivity analysis method enables the apportionment of output uncertainty into contributions not only due to the variance of input factors and their interactions, but also due to properties of the underlying probability distributions such as intrinsic extent and non-Gaussianity. Furthermore, uncertainty and sensitivity information are combined to identify specific strategies for uncertainty mitigation and visualize tradeoffs between available options. These approaches are integrated with design budgets to guide decisions regarding the allocation of resources toward improving system robustness and reliability. 3 The methods developed in this work are applicable to a wide variety of engineering systems. In this thesis, they are demonstrated on a real-world aviation case study to assess the net cost-benefit of a set of aircraft noise stringency options. This study reveals that uncertainties in the scientific inputs of the noise monetization model are overshadowed by those in the scenario inputs, and identifies policy implementation cost as the largest driver of uncertainty in the system. Thesis Supervisor: Karen E. Willcox Title: Professor of Aeronautics and Astronautics Committee Member: Ian A. Waitz Title: Jerome C. Hunsaker Professor of Aeronautics and Astronautics Committee Member: Douglas L. Allaire Title: Assistant Professor of Mechanical Engineering, Texas A&M University 4 Acknowledgments The journey toward a PhD is one that cannot be undertaken alone. It is only with help from countless people along the way that I have made it thus far, and I am eternally grateful for their support. First and foremost, I would like to thank my advisor, Professor Karen Willcox, for her guidance, mentorship, and encouragement over the past four years. It has truly been a privilege to work with Karen, and I am greatly indebted to her for everything she has taught me about research, life, and making it all balance together. I also extend thanks to my committee members, Dean Ian Waitz and Professor Doug Allaire, for lending their knowledge and support to every aspect of this research, and for providing invaluable feedback and guidance along the way. I would also like to acknowledge my thesis readers, Professor John Deyst and Professor Olivier de Weck for contributing insightful comments and suggestions that have helped to improve this thesis. To my labmates in the ACDL, past and present, thank you so much for sharing this journey with me. It has been an honor to work with and learn from all of you. I would like to especially recognize Leo Ng, Andrew March, Sergio Amaral, Chad Lieberman, Giulia Pantalone, Harriet Li, Laura Mainini, R´emi Lam, Marc Lecerf, Tiangang Cui, Patrick Blonigan, Alessio Spantini, Eric Dow, Hemant Chaurasia, and Xun Huan for making my stay in Building 37 such a memorable experience. I am also grateful to the larger AeroAstro community for providing a welcoming and dynamic environment in which to learn and grow. I have made so many friends in AeroAstro and will always fondly remember the good times we've shared. A special thank you to the ladies of the Women's Graduate Association of Aeronautics and Astronautics (WGA3) | Sunny Wicks, Sameera Ponda, Farah Alibay, Whitney Lohmeyer, Abhi Butchibabu, Sathya Silva, and Carla Perez Martinez | it has been such a joy getting to know you and I can't wait to see the amazing things you will continue to accomplish in the future. I also owe a great deal of thanks to Philip Wolfe. From a technical standpoint, thank you for helping me set up and run the CAEP/9 noise stringency case study on such short notice; from a personal perspective, thank you for your 5 friendship over the past 10 years | I have really appreciated our project meetings, commiseration lunches, and random adventures both at Duke and at MIT. Finally, I would like to acknowledge the AeroAstro staff members who work tirelessly to make life easier and more enjoyable for graduate students. Jean Sofronas, Joyce Light, Meghan Pepin, Marie Stuppard, Beth Marois, Bill Litant, Robin Palazzolo, and Sue Whitehead | thank you for being so generous with your time, help, smiles, and words of encouragement. Outside of research, I have been extremely fortunate to have the support of a diverse group of friends who help me keep things in perspective. Matt and Amanda Edwards, Jim Abshire, David Rosen, Emily Kim, Audrey Fan, Lori Ling, Yvonne Yamanaka, Tiffany Chen, Tatyana Shatova, and Robin Chisnell are fellow graduate students in the Boston area, and dear friends who have accompanied me on this journey. Thank you for the potluck dinners, ski trips, hiking outings, gym workouts, and runs along the Charles River | you have done so much to lift my spirits and bring joy to my life. Another huge source of strength comes from the Sidney-Pacific Graduate Community, without a doubt the best graduate dormitory in the world. Living at SP was one of the highlights of my time at MIT, and has taught me so much about friendship, leadership, and the spirit of community. In particular, I wish to thank the former Housemaster team | Professor Roger Mark and Mrs. Dottie Mark, and Professor Annette Kim and Dr. Roland Tang | thank you for welcoming me into your home and making sure that I am always well-fed. To my friends from SP | Amy Bilton, Mirna Slim, Ahmed Helal, Jit Hin Tan, Brian Spatocco, George Lan, Po-Ru Loh, Kendall Nowocin, George Chen, Boris Braverman, Fabi´anKozynski, and many, many others | thank you for sharing your kindness and talents, for inspiring me to set ambitious goals, and for showing me that there are few problems in life that can't be solved through teamwork and more bacon. Finally, I would like to thank my family for their steadfast love and support. My parents, Jie Chen and Helin He, left behind friends, family, and successful careers in China to immigrate to the United States in pursuit of better opportunities. Without their sacrifice, I would not be where I am today. I credit them for instilling in me the 6 importance of a good education, the determination to overcome whatever challenges may arise, and the confidence to forge my own path. Last but not least, I want to thank Tim Curran for being my co-pilot on this incredible journey. Thank you for always believing in me, for teaching me to be patient with myself and others, and for giving me the strength to push through the tough times. I look forward to starting the next chapter of our lives together. The work presented in this thesis was funded in part by the International De- sign Center at the Singapore University of Technology and Design, by the DARPA META program through AFRL Contract Number FA8650-10-C-7083 and Vanderbilt University Contract Number VU-DSR #21807-S7, by the National Science Founda- tion Graduate Research Fellowship Program, and by the Zonta International Amelia Earhart Fellowship Program. 7 Contents 1 Introduction 19 1.1 Motivation for Uncertainty Quantification . 20 1.2 Terminology and Scope . 21 1.2.1 System Design Process . 21 1.2.2 Characterizing Uncertainty . 24 1.2.3 Some Common Probability Distributions . 26 1.3 Designing for Robustness and Reliability . 28 1.3.1 Defining Robustness and Reliability . 29 1.3.2 Background and Current Practices . 30 1.3.3 Opportunities for Intellectual Contribution . 33 1.4 Thesis Objectives . 33 1.5 Thesis Outline . 34 2 Quantifying Complexity and Risk in System Design 37 2.1 Background and Literature Review .