Monitoring and Managing Heterogeneous Middleware Günther Rackl Institut für Informatik Lehrstuhl für Rechnertechnik und Rechnerorganisation Monitoring and Managing Heterogeneous Middleware Günther Rackl Vollständiger Abdruck der von der Fakultät für Informatik der Technischen Universität München zur Erlangung des akademischen Grades eines Doktors der Naturwissenschaften (Dr. rer. nat.) genehmigten Dissertation. Vorsitzender: Univ.-Prof. Dr. H. M. Gerndt Prüfer der Dissertation: 1. Univ.-Prof. Dr. A. Bode 2. Univ.-Prof. Dr. E. Jessen Die Dissertation wurde am 13. Dezember 2000 bei der Technischen Universität München eingereicht und durch die Fakultät für Informatik am 29. Januar 2001 angenommen. Abstract The development and deployment of distributed applications still represents a major problem of computer science due to the complex and heterogeneous architectures of modern computing systems. The goal of this thesis is to improve the development and deployment of distributed applications by enhancing the on-line tool support for heterogeneous middleware environments. Our approach to reach this goal is to provide a monitoring and management system that is tailored to the needs of modern middleware applications. Starting with an analysis of current middleware platforms and on-line tools for various application domains, the shortcomings of existing on-line monitoring and management approaches are worked out. Existing on-line tools mostly represent proprietary solutions for particular middleware platforms and are tailored to homo- geneous scenarios, while their monolithic and static architecture limits their flexi- bility and extensibility. From these insights, several requirements for the development of a monitoring and management infrastructure for heterogeneous middleware arise. Approaches for systematically managing complex environments as well as mechanisms for deal- ing with heterogeneity are essential in this context. Moreover, concepts supporting different kinds of on-line tools and enabling the extensibility of on-line tool envi- ronments are indispensable. From these considerations, the MIMO MIddleware MOnitor approach is de- rived. MIMO is based on a multi-layer monitoring model that classifies observed entities by means of several abstraction layers, a generic monitoring infrastructure, and a sophisticated usage methodology. The prototypical implementation of the MIMO system is evaluated with respect to its applicability within current middle- ware platforms. Besides the observation of general purpose middleware platforms like CORBA, the application to metacomputing systems, a distributed interactive simulator, and a medical image reconstruction programme is demonstrated. The benefits of the MIMO approach result from the systematic approach for monitoring heterogeneous environments, its capability to support tools for all on- line phases of the software lifecycle, and the extensible architecture of the monitor- ing system. Furthermore, the rapid tool development methodology allows to con- struct new tools or integrate new middleware platforms efficiently. Altogether, the results obtained within thesis represent a major contribution to an enhanced on-line tool support in heterogeneous middleware environments. v Für meine Eltern & für Beate. Acknowledgements This thesis would not have been possible without the support of several people. First of all, I would like to thank Prof. Arndt Bode for supporting me throughout the years and for providing an excellent research environment at the LRR. I enjoyed the great freedom to carry out my research and the opportunity to participate in var- ious scientific conferences all over the world. In spite of his numerous obligations Prof. Bode always had time for a short discussion with me. Prof. Eike Jessen deserves gratitude for being the co-referee of my thesis. From him I received helpful feedback in a constructive and unbureaucratic way. The heads of the tools group at the LRR deserve thanks for encouraging me and my research within the past years. PD Dr. habil. Thomas Ludwig has con- tributed a lot to my scientific work and my personal development within the past years. Dr. Roland Wismüller has always been a good advisor regarding scientific and technical questions. I would also like to thank my current and former colleagues in the tools group, Markus Lindermeier, Dr. Jörg Trinitis, Dr. Christian Röder, and Dr. Michael Ober- huber. Together, we had a lot of discussions concerning scientific as well as non- scientific topics. My room mates Dr. Ivan Zoraja, Philipp Drum, and Martin Schulz as well as our system administrator Klaus Tilk deserve credit for innumerable technical, political, and philosophical conversations. Without them, my time at the LRR would not have been that exciting for me. I am also grateful to Dr. Ivan Zoraja, PD Dr. habil. Thomas Ludwig, Dr. Roland Wismüller, Dr. Wlodzimierz Funika, Markus Lindermeier, Dr. Sabine Rathmayer, and Ulrich Rackl for thoroughly reviewing my manuscript, or parts of it. Furthermore, this thesis has gained a lot from students working for the MIMO project. I would like to thank Michael Rudorfer, Alexi Stamatakis, Bernd Süß, and Lars Orta for contributing to the success of the project with their diploma theses. Prof. Ian Foster from Argonne National Laboratory deserves gratitude for taking me up as a guest scientist in the Distributed Systems Laboratory at the ANL. Gregor von Laszewski, PhD, supported me immensly during my stay at Argonne. It was a great experience for me to participate in their research projects, and I personally and scientifically learned a lot during that time. Last not least, I would like to thank my parents, my brothers, my love Beate, and my friends for supporting me with the great private background needed to finish such a thesis succesfully. München, November 2000 Günther Rackl vii Contents 1 Introduction 1 1.1 Motivation and Goals 1 1.2 Methodology and Outline 3 1.3 Research Contribution 5 1.4 Background 7 2 Middleware 9 2.1 General Definitions 9 2.1.1 Client/Server Systems 9 2.1.2 The Gartner Model 10 2.1.3 Three-Tier Applications 10 2.1.4 Middleware 11 2.1.5 Heterogeneity and Interoperability 12 2.2 Common Middleware Systems 13 2.2.1 Historically Relevant Approaches 14 2.2.2 Parallel Programming Environments 15 2.2.3 Metacomputing Infrastructures 17 2.2.4 Distributed Object-Oriented Systems 18 2.2.5 Enterprise Middleware 26 2.3 Summary 28 3 Monitoring and Management 31 3.1 General Principles 31 3.1.1 Software Development and On-line Tools 31 3.1.2 Monitoring 32 3.1.3 Management 33 3.1.4 Classification of Tools 34 3.1.5 Monitoring Mechanisms 35 3.1.6 Problems of Monitoring 36 3.2 Common Approaches and Systems 37 3.2.1 Monitoring Systems for Parallel Programming 37 3.2.2 Tools and Monitors for Distributed Object Computing 39 3.2.3 Enterprise Tools and Standards 46 3.2.4 Network Management Approaches 49 3.3 Conclusion 52 ix 4 Requirements for a Generic Monitoring Approach 55 4.1 Drawbacks of Existing Systems 55 4.1.1 Proprietary Solutions 55 4.1.2 Homogeneous Scenarios 56 4.1.3 Simple Scenarios 56 4.1.4 Missing Interoperability Support 57 4.1.5 Inflexible Architecture 57 4.2 Criteria for a Generic Monitoring Approach 58 4.2.1 Systematic Concept for Complex Environments 58 4.2.2 Tool Support for all On-Line Phases 59 4.2.3 Cover System Heterogeneity 60 4.2.4 Flexibility and Extensibility 61 4.3 Summary 62 5 The MIMO Approach 65 5.1 Multi-Layer Monitoring 65 5.1.1 Hierarchical Information Model 65 5.1.2 Abstract Entity-Relationship Information Model 66 5.1.3 The Multi-Layer Monitoring Model 67 5.1.4 Formal Description 68 5.1.5 Application to Concrete Middleware 71 5.1.6 Algorithms for Accessing the MLM 74 5.1.7 Summary 76 5.2 Monitoring Infrastructure 77 5.2.1 Core Monitoring Framework 78 5.2.2 Interfaces and Events 79 5.2.3 Conclusion 81 5.3 Usage Methodology and Tool Frameworks 81 5.3.1 Usage Methodology 82 5.3.2 Tool Frameworks 82 5.4 Summary 83 6 MIMO Architecture and Implementation 85 6.1 Tool Development and Usage Process 85 6.2 Monitoring Architecture 87 6.2.1 Basic System Structure 87 6.2.2 Components 88 6.2.3 Interaction Patterns 92 6.2.4 Distribution and Assignment of MIMO Instances 94 6.2.5 Summary 98 6.3 MIMO Access and Usage 98 6.3.1 Entity Definition 98 6.3.2 MIMO Core Start-Up 99 6.3.3 Tool View 99 6.3.4 Instrumentation View 101 6.3.5 Example 104 6.3.6 Summary 106 6.4 MIMO Implementation 106 6.4.1 MIMO Core 106 6.4.2 Tools 112 6.4.3 Instrumentation 115 6.4.4 Conclusion 119 6.5 Summary 120 7 Tool Development and Application Scenarios 121 7.1 Tool Development Methodology 121 7.1.1 Motivation 122 7.1.2 Rapid Tool Development Methodology 123 7.1.3 Implementation Aspects 126 7.1.4 Conclusion 128 7.2 Application Scenarios 128 7.2.1 Globus 129 7.2.2 LSD — Latency Sensitive Distribution 133 7.2.3 SEEDS 135 7.2.4 Managing a Load-Balanced Medical Application 136 7.2.5 Summary 139 7.3 Conclusion 139 8 Evaluation and Results 141 8.1 Evaluation 141 8.1.1 MIMO Approach 141 8.1.2 Implementation and Performance 144 8.1.3 Tool development and Middleware Integration 147 8.2 Results 148 8.2.1 Problem Review 148 8.2.2 Conceptional Results 148 8.2.3 Lessons Learned 150 8.3 Summary 151 9 Conclusion 153 9.1 Summary 153 9.2 Outlook 155 Bibliography 159 Index 173 List of Figures 2.1 Client/Server Computing Model 10 2.2 Gartner Group Distributed Application Model 11 2.3 Middleware Layer between Applications and Platforms 13 2.4 Broker Pattern 19 2.5 Proxy Pattern 20 2.6 Remote Invocation 21 2.7 OMA Reference Model 22 2.8 X/Open Transaction