ASSESSING AND MITIGATING VULNERABILITY CHAINS IN MODEL- CENTRIC ACQUISITION PROGRAMS by Jack Burnett Reid B.S. Mechanical Engineering and B.A. Philosophy Texas A&M University, 2015 Submitted to the MIT Institute for Data, Systems, and Society And the Department of Aeronautics and Astronautics in Partial Fulfillment of the Requirements for the Degrees of Master of Science in Technology and Policy And Master of Science in Aeronautics and Astronautics at the Massachusetts Institute of Technology June 2018 2018 Massachusetts Institute of Technology. All rights reserved. Signature of Author……………………………………………………………………………………………………... MIT Institute for Data, Systems, and Society Department of Aeronautics and Astronautics May 18, 2018 Certified by……………………………………………………………………………………………………………... Donna H. Rhodes Principal Research Scientist, Sociotechnical Systems Research Center Thesis Supervisor Certified by……………………………………………………………………………………………………………... Daniel E. Hastings Cecil and Ida Green Education Professor of Aeronautics and Astronautics and Engineering Systems Chief Executive Officer and Director, Singapore MIT Alliance for Research and Technology Thesis Supervisor Accepted by…………………………………...………………………………………………………………………… Munther Dahleh W. Coolidge Professor of Electrical Engineering and Computer Science Director, MIT Institute for Data, Systems, and Society Accepted by…………………………………...………………………………………………………………………… Hamsa Balakrishnan Associate Professor of Aeronautics and Astronautics Chair, Graduate Program Committee This material is partially based upon work by the Naval Postgraduate School Acquisition Research Programs under Grant No. N00244-17-1-0011. 2 ASSESSING AND MITIGATING VULNERABILITY CHAINS IN MODEL-CENTRIC ACQUISITION PROGRAMS by Jack Burnett Reid Submitted to the MIT Institute for Data, Systems, and Society and Department of Aeronautics and Astronautics on May 18, 2018 in Partial Fulfillment of the Requirements for the Degrees of Master of Science in Technology and Policy and Master of Science in Aeronautics and Astronautics Abstract Acquisition programs increasingly use model-centric approaches, generating and using digital assets throughout the lifecycle. Model-centric practices have matured, yet in spite of sound practices there are uncertainties that may impact programs over time. The emergent uncertainties (policy change, budget cuts, disruptive technologies, threats, changing demographics, etc.) and related programmatic decisions (e.g., staff cuts, reduced training hours) may lead to cascading vulnerabilities within model-centric acquisition programs, potentially jeopardizing program success. Program managers are increasingly faced with novel vulnerabilities. They need to be equipped with the means to identify model-centric program vulnerabilities and determine where interventions can most effectively be taken. In this research, Cause-Effect Mapping (CEM), a vulnerability assessment technique, is employed to examine these vulnerabilities. Using a combination of literature investigation, expert interviews, and usability testing, a CEM is created to represent the novel vulnerabilities posed by model-centric practices in acquisition programs. Particular attention is paid to cybersecurity vulnerabilities, which pose a serious threat to the successful implementation of model-centric practices. From this CEM, key gaps in program manager knowledge and organizational policies are identified and potential responses proposed. Thesis Supervisor: Donna H. Rhodes Title: Principal Research Scientist, Sociotechnical Systems Research Center Thesis Supervisor: Daniel E. Hastings Title: Cecil and Ida Green Education Professor of Aeronautics and Astronautics and Engineering Systems 3 God, grant me the insight to find and use models to understand the world around me, The wisdom to acknowledge that they will someday fail, And the strength to rid myself of them when it is apparent they no longer work. -inspired by Ze Frank & the Serenity Prayer 5 Acknowledgements Before proceeding onto this work, I would like to thank several individuals. First and foremost is my wife, Rebecca, who has been unfailingly supportive of me throughout the endeavor that is an MIT graduate program. First in a long-distance relationship and then in person, she has consistently buoyed my hopes and sense of self-worth when I needed it most. Next, I would like to thank Dr. Donna Rhodes, one of my thesis advisors. She has taught me a great deal about how to think critically about my research work and how to present it to an audience. Her level of engagement and enthusiasm for my work has been much appreciated. I regret only that I believe that this thesis insufficiently represents what she enabled me to accomplish. As I transition into a doctoral program and may no longer be working for her, I look forward to continuing to work with her. Thanks also goes to Prof. Daniel Hastings, for agreeing to serve as my AeroAstro advisor despite not knowing me too well and while working in Singapore. I can only hope that this thesis proves worthy of his trust. My fellow researchers at SEAri deserve gratitude as well. In particular, I would like to thank Sarah Rovito, Parker Vascik, Shane German, Lucie Reymondet, and Brian Mekdeci (though I never met him). This work is built upon theirs (as a count of the number of times I cite them and the number of hours going over their theses can demonstrate). Lucie in particular set the stage for the emergence chapter, though she never published on the topic. Beyond mere citations, their support was integral as peers, in classes, at conferences, and (in Shane’s case) in diagnosing that, yes, in fact my hand was broken and I needed to go to the doctor. I would like to thank my fellow housemates at pika. I do not believe that I have ever lived in a true community until I joined this continuing experiment in cooperative living. Thanks for (almost) always having dinner ready at the end of the day and for filling the house with laughter. Finally, I would be remiss without acknowledging the financial support for this research from the MIT-SUTD Fellowship Program, the Systems Engineering Research Center, and the Naval Postgraduate School. Without their generosity, this work literally would not have happened 7 Table of Contents ABSTRACT ................................................................................................................................... 3 ACKNOWLEDGEMENTS ......................................................................................................... 7 TABLE OF CONTENTS ............................................................................................................. 9 LIST OF FIGURES .................................................................................................................... 13 LIST OF TABLES ...................................................................................................................... 15 LIST OF ACRONYMS AND INITIALISMS .......................................................................... 17 CHAPTER 1 : INTRODUCTION ............................................................................................. 19 1.1 MOTIVATION ........................................................................................................................ 19 1.2 RESEARCH APPROACH ......................................................................................................... 19 1.3 RESEARCH QUESTIONS ......................................................................................................... 20 1.4 SCOPE .................................................................................................................................. 20 1.5 RESEARCH CONTRIBUTION................................................................................................... 21 1.6 THESIS STRUCTURE .............................................................................................................. 22 CHAPTER 2 : DEFENSE ACQUISITION AND MODEL-CENTRIC ENGINEERING .. 23 2.1 WHAT IS DEFENSE ACQUISITION? ........................................................................................ 23 2.1.1 Historical Acquisition ............................................................................................... 23 2.1.2 Acquisition in the 20th Century ................................................................................. 24 2.1.3 Problems in Acquisition ............................................................................................ 25 2.2 WHAT IS MODEL-CENTRIC ENGINEERING/ACQUISITION? .................................................... 28 2.2.1 Aspects of MCE ......................................................................................................... 29 2.2.2 Examples of MCE Implementation ........................................................................... 33 2.2.3 Benefits of MCE ........................................................................................................ 35 2.2.4 MCE Implementation Barriers ................................................................................. 36 2.3 PROGRAM VS PROJECT, END-SYSTEMS ................................................................................ 37 CHAPTER 3 : VULNERABILITY ASSESSMENT ............................................................... 39 3.1 DEFINITIONS ........................................................................................................................ 39 9 3.2 METHODS OF VULNERABILITY,