A Framework for Decision Support in Systems Architecting by Willard Lennox Simmons B.S., University of New Hampshire, 1997 M.S., University of Colorado, 1999 M.S.E., Princeton University, 2005 Submitted to the Department of Aeronautics and Astronautics in partial fulfillment of the requirements for the degree of Doctor of Philosophy at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY February 2008 c Massachusetts Institute of Technology 2008. All rights reserved. Author............................................................................ Department of Aeronautics and Astronautics January 11, 2008 Certified by........................................................................ Edward F. Crawley Ford Professor of Engineering Thesis Supervisor Certified by........................................................................ Brian C. Williams Professor of Aeronautics and Astronautics Committee Member Certified by........................................................................ Benjamin H. Y. Koo Associate Professor of Industrial Engineering, Tsinghua University Committee Member Accepted by....................................................................... David L. Darmofal Associate Professor of Aeronautics and Astronautics Chair, Committee on Graduate Students 2 A Framework for Decision Support in Systems Architecting by Willard Lennox Simmons Submitted to the Department of Aeronautics and Astronautics on January 11, 2008, in partial fulfillment of the requirements for the degree of Doctor of Philosophy Abstract The objective of this thesis is to provide a method and tool to leverage computational resources to empower a systems architect to reason about architectural decisions more comprehensively and effectively compared to traditional approaches. This thesis pro- vides a computational framework for decision support called the Architecture Decision Graph framework. It supports human decision-making by providing a methodology for generating and analyzing architectures as the result of a set of interrelated deci- sions. ADG’s explicit representation of an interconnected decision problem is a bipar- tite graph of decision variables, property variables, logical constraints, and property functions. The Architecture Decision Graph’s framework provide tools for reasoning about the structure of a decision problem, generating the set of feasible combinations of decisions, and simulating their outcome. The underlying computational engine used by ADG is the Object-Process Network (OPN) kernel. The contribution of this thesis to the field of systems architecting falls into three areas: First, the thesis contributes the ADG representation of an architectural candi- date space as a set of interrelated decision variables. Second, the thesis contributes the ADG framework, which leverages the ADG representation of architecture to transform an architecting problem into a computational problem. Third, this thesis contributes decision space viewing tools, which present the potential impact of changes in the assignments of the decision variables to an architect. The ADG representation, analysis methodology, and tools are demonstrated with two applications. The first application is a retrospective study of the architectural decisions related to the development of the Apollo moon project of the 1960’s. The second application is a study of decisions in support of NASA’s lunar outpost ar- chitecting effort. The applications include discussions of the practical considerations related to the use of ADG as a decision representation method, the efficiency of the simulation algorithm, and a discussion of the architecting insights that can be drawn from the results. Thesis Supervisor: Edward F. Crawley Title: Ford Professor of Engineering 3 4 Acknowledgments I would like to thank my committee, Edward F. Crawley, Brian C. Williams, and Ben- jamin H. Y. Koo for their support of my research. Professor Crawley’s enthusiasm for this research was infectious. He introduced me to the field of systems architecting research when I first came to MIT, and I haven’t looked back. As an advisor, Profes- sor Crawley was a perfect match for me. He was hands-off for weeks at a time, which allowed me to work on new ideas on my own. Then, when the time was right, we would spend hours at his home on a Saturday, sitting around the dinner table refining the details. I appreciate the confidence he instilled in me. Professor Brian Williams introduced me to the world of autonomous reasoning and Artificial Intelligence re- search, which is filled with clever ideas for representing and processing knowledge in order to generate new knowledge. Many of the ideas in this thesis had their genesis in his Introduction to Principles of Autonomy and Decision Making. I credit Professor Benjamin H. Y. Koo with changing the way I think about design problems. Over the course of three years, he trained me to look for solutions at the highest level of abstraction first, rather than getting lost in the details. My two thesis readers, Professor Olivier de Weck and Dr. Jana Schwartz, put a tremendous amount of time and thought into giving me feedback on my thesis. It seems almost unfair that their names are not on the cover of my thesis. Their comments helped me focus the argument for my contribution. Professor Nick Roy was my minor advisor. I’m grateful that he was always willing to set aside the time in his busy schedule to discuss my research. This achievement would not be possible without the support of my family. Striving for a doctoral degree meant five years of missed dinners, canceled weekend plans, and holiday plans invaded by the clicking sound of a laptop keyboard. When my stress- levels peaked, my wife, Madeline T. Pham, was always there to calm me down. When I was stuck in the office until late at night, she was always willing to pick me up and drive me home. Madeline is a constant source of encouragement and I could not have done this without her. My parents’ influence molded me into the person I am today. My father, Theodore R. Simmons, taught me to question everything I am told and everything I have read. Inquisitiveness is the most important trait of a researcher. My mother, Janet L. Simmons, inspired me to overcome bumps in the road of life. Whenever I had a tough day, she always said, Tomorrow’s another day. It sounds silly, but it’s really true. Tomorrow is another day, and I am there! My colleagues at MIT have been tremendously helpful. Wilfried Hofstetter took the time read every section of my thesis with his characteristic precision. His feedback was invaluable. The extensive research by Paul Wooster and Wilfried Hofstetter formed the basis for the NASA Lunar Outpost Study in Chapter 5 and helped refine the Apollo study in Chapter 4. Getting to know Ryan Boas has been a pleasure. He was always willing to give professional, sharp, targeted advice on my research. Jaeymung Ahn has been a tremendous help with the details of my work ever since the first day I arrived at MIT. Jaeymung and Lars Blackmore were my study partners for the qualifying exam. Maokai Lin, who arrived at MIT just five months ago, has been helped with many software implementation details. Kathi Cofield, who has been 5 Ed Crawley’s assistant for five years, has always been helpful taking care of scheduling snafus and making sure I get the time I needed with Ed. It’s been a lot of fun with her. I can’t name all the members of the Space Architects Group who contributed to this effort, but I will say that I was especially moved when about fifteen people showed to my thesis defense practice the Friday before my defense. Five more people came to the office in the middle of blizzard on the following Sunday to help me put the final touches on my talk. In the past year, I’ve met two students from Professor Daniel Jackson’s lab in EECS. Rob Seater and Derek Rayside, two brilliant computer scientists, have gener- ously spent many hours of their time talking about my research instead of making progress on theirs. I hope I can return the favor soon. This work was made possible by the generous support of Draper Laboratory under a University IR&D grant. I’ve thoroughly enjoyed my exchanges with Phil Babcock, Jose Lopez, Jasjit Heckathorn, and Jana Schwartz. Each of them gave me a mixture of encouragement and targeted comments and helped me focus on the key unanswered questions. My semi-annual briefings at Draper have always been able to snap this academic research back to reality. 6 Contents Abstract 3 Acknowledgments 5 Contents 7 List of Figures 11 List of Tables 15 1 Introduction 17 1.1 Overview .................................. 17 1.2 Decisions and Decision Support ..................... 19 1.3 Systems Architecture and Systems Architecting ............ 23 1.4 Needs of the Architectural Decision-Maker ............... 26 1.5 Summary and Synopsis .......................... 27 2 Literature Review 29 2.1 Overview .................................. 29 2.2 Definition of Decision Support System ................. 29 2.2.1 Aspects of an Architectural DSS ................. 30 2.3 State of Practice ............................. 31 2.3.1 Table and Matrix-Based Decision Support ........... 31 2.3.2 Tree and Directed Graph-Based Decision Support ....... 33 2.3.3 Constraint Graph-Based Decision Support ........... 41 2.3.4 Meta-Language-Based Decision Support ............ 43 2.4 Summary ................................. 47 3 Architecture Decision Graph 49 3.1 Overview .................................. 49 3.2 The Architecture Decision Graph Framework