Complexity of System Configuration Management A Dissertation submitted by Yizhan Sun In partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science TUFTS UNIVERSITY August 2006 c Yizhan Sun, June, 2006 ADVISOR: Alva L. Couch Abstract System administration has often been considered to be a “practice” with no theoretical underpin- nings. In this thesis, we begin to define a theory of system administration, based upon two activities of system administrators: configuration management and dependency analysis. We formalize and explore the complexity of these activities, and demonstrate that they are intractable in the general case. We define the concepts of system behavior, kinds of configuration operations, a model of configuration management, a model of reproducibility, and proofs that several parts of the process are NP-complete or NP-hard. We also explore how system administrators keep these tasks tractable in practice. This is a first step toward a theory of system administration and a common language for discussing the theoretical underpinnings of the practice. ii Acknowledgements This thesis is the result of four years of work whereby I have been accompanied and supported by many people. It is a pleasant aspect that I have now the opportunity to express my gratitude to all of them. The first person I would like to thank is my advisor Alva L. Couch. I have been working with him since 2001 when I started my Master’s project. His enthusiasm and integral view of research and his humor when things get tough have made a deep impact upon me. He patiently guided me and supported me throughout the course of my study. I would like to thank Professor Kofi Laing and Professor Lenore Cowen for their help on com- putation theory and for being committee members. I also thank Professor Ricardo Pucella for reviewing my work. I am grateful for my fellow students: Ning Wu, Marc Chiarini, Hengky Susanto, Josh Danziger and Bill Bogstad for their insightful comments and discussions of this work. I would like to thank the Tufts University Computer Science Department for giving me financial support to finish the study. I owe a lot of gratitude to my parents and my parents-in-law. My parents stayed with us for three years in the U.S. to help me with my two children. Without their help, I could not even start this thesis. My parents-in-law supported us financially out of their limited resources. They have shown me what unconditional love is. I am very grateful for my husband for his love and encouragement during the Ph.D. period. I thank my sons Samuel and David for their smiles and countless precious joyful moments in our life. iii DEDICATION To my parents and my parents-in-law, for their endless love. iv Contents 1 Introduction 2 2 Landscape of System Administration 5 2.1 Definition of System Administration . 5 2.2 Taxonomy of System Administration . 6 2.3 Some System Administration Tasks . 8 2.3.1 Backup and Restore . 8 2.3.2 User Management . 9 2.3.3 Service Management . 11 2.3.4 Security . 13 2.3.5 Testing and Quality Assurance . 16 3 Introduction to Configuration Management 18 3.1 Software Configuration Management . 19 3.2 System Configuration Management . 20 3.3 How Configuration Controls Behavior . 20 4 Challenges of Configuration Management 22 4.1 Change . 22 4.2 Scale............................................. 23 4.3 Interdependence of Software and Hardware . 23 4.4 Heterogeneity . 24 4.5 Contingency . 25 4.6 Diverse Users . 26 4.7 Ineffective Collaboration of Multiple System Administrators . 26 v 4.8 Mobile Environments . 27 4.9 Service Guarantees . 27 5 Automation and Autonomic Computing 28 5.1 History of Automation in Configuration Management . 29 5.2 Current Strategies of System Configuration . 32 5.2.1 Manual Configuration . 32 5.2.2 Custom Scripting . 32 5.2.3 Structured Scripting . 32 5.2.4 File Distribution . 34 5.2.5 Declarative Syntax . 35 6 The Configuration Process 36 6.1 Documentation . 36 6.2 Experience . 37 6.3 The Configuration Process . 38 7 A Model of Configuration Management 45 7.1 Closed- vs. Open-world Models of Systems . 45 7.2 Observed Behavior . 46 7.3 Actual State and Observed State . 47 7.4 Configuration Operations . 48 7.5 Two Configuration Management Automata . 49 8 Reproducibility 51 8.1 Local Reproducibility . 52 8.1.1 Properties of Locally Reproducible Operations . 55 8.1.2 Constructing Locally Reproducible Operations . 57 8.2 Population Reproducibility . 60 9 Limits on Configuration Operations 62 9.1 Limits on Configuration Operations . 62 9.2 Relationship Between Limits . 66 vi 10 Complexity of Configuration Composition 69 10.1 Composability . 69 10.2 Complexity of Operation Composability . 70 10.2.1 Component Selection . 70 10.2.2 General Composability . 72 10.2.3 Atomic Operation Composability . 74 10.2.4 Composability of Partially Ordered Operations . 75 10.2.5 Composability of Convergent Operations . 76 10.2.6 Summary of The Proofs . 77 10.3 Discussion . 77 11 Dependency Analysis 79 11.1 Dependency Analysis Techniques . 80 11.1.1 Instrumentation . 80 11.1.2 Perturbation . 80 11.1.3 Data Mining . 81 11.1.4 Requirements Analysis . 81 11.1.5 Dependency Control . 81 11.2 Basic Concepts . 82 11.2.1 White Box . 83 11.2.2 Black Box . 83 11.2.3 Functional Expectations and Tests . 84 11.2.4 State and Behavior . 85 11.3 Dependence Definition . 86 11.3.1 Dependency in a Closed-world System . 86 11.3.2 Dependency in an Open-world System . 88 11.4 Complexity of Dependency Analysis . 89 11.4.1 Black Box Dependency Analysis . 89 11.4.2 White Box Dependency Analysis . 90 11.5 Discussion . 91 12 Configuration Management Made Tractable in Practice 92 12.1 Experience and Documentation . 92 vii 12.2 Reduction of State Space . 93 12.3 Abstraction . 94 12.4 Re-baselining . 95 12.5 Orthogonality . 95 12.6 Closure . 95 12.7 Current Strategies Make Configuration Management Tractable . 96 13 Ways to Reduce Complexity 99 13.1 Choosing Easy Instances to Solve . 99 13.1.1 Reduction of State Space . 100 13.1.2 Using Simple Operations . 100 13.1.3 Forming Hierarchies and Relationships . 100 13.2 Approximation . 101 13.3 Dynamic Programming . 102 14 Conclusions 104 14.1 Review . 104 14.2 Future Work . 106 viii List of Figures 6.1 The configuration stages . 40 6.2 The configuration process . 41 6.3 Asynchronous interactions between system administrators and the environment . 42 9.1 Stateless operations can have if statements . ..