BENCHMARK SYSTEM FOR A STORAGE RESOURCE BROKER A Dissertation Submitted In Partial Fulfilment Of The Requirements For The Degree Of MASTER OF SCIENCE In NETWORK CENTERED COMPUTING, HIGH PERFORMANCE COMPUTING in the FACULTY OF SCIENCE THE UNIVERSITY OF READING by Carsten Koebernick March 3, 2006 University Supervisor: Prof. Vassil Alexandrov Placement Supervisor: Dr. Adil Hasan Abstract This dissertation deals with the design and development of a measurement system for a "Storage Resource Broker" (SRB). The SRB is a client-server middleware that provides an interface for connecting to heterogeneous data resources over a network. The aim of the project is to enable long-run measurements of the SRB server and standard applications. The document starts with comparing two state of the art monitoring system to find one that helps with gathering and transmitting measurement data. Afterwards the document focuses on other necessary tools, which help to measure applications and to store them in a relational database. Two basic approaches will be visualised, which will meet the task of the project. The measurement system has been developed to gather measurements from more than one server, so both approaches base on a client server system. Furthermore, measurement graphs have been developed to compare the load and memory efficiency of different machines and measurement cycles. Finally, the measurement system should help the researchers of the SRB to find possible leaks and ease the further development. Contents List of Figures VI List of Tables VIII 1 Introduction 1 1.1 Project Description . 2 1.2 Motivation . 2 1.3 Restrictions . 3 1.4 Document Structure . 3 2 Monitoring Systems 5 2.1 Ganglia . 6 2.2 Nagios . 10 2.3 Comparison . 12 3 Technologies 13 3.1 Storage Resource Broker . 13 3.1.1 Scommands . 15 3.2 User Management of Unix based systems . 16 3.3 Python . 18 3.4 Tkinter . 19 3.5 Bash . 20 3.6 Database system . 20 3.6.1 Database models . 21 3.6.2 Entity Relationship Model (ERM) . 22 3.6.3 Structured Query Language - SQL . 23 3.6.4 SQLite . 26 3.6.5 Pysqlite . 27 II Contents 3.7 XML . 27 3.7.1 Structure and Rules . 28 3.7.2 Parser . 29 3.7.3 Meta languages . 29 4 Analysis 32 4.1 Ganglia . 32 4.2 SRB System . 34 4.3 Basic Approach . 35 4.4 Solutions . 37 4.4.1 Solution with Network Functionality of Ganglia . 37 4.4.2 Solution with Data Transmission over the Socket . 38 4.4.3 Comparison and Decision . 38 5 Design 40 5.1 System Architecture . 40 5.1.1 Monitoring Server . 40 5.1.2 Monitoring Client . 42 5.1.3 Measurement Sequence . 45 5.2 Database . 47 5.2.1 Entity Relationship Model - ERM . 48 5.2.2 Relational Data model - (RDM) . 49 5.3 GUI Design . 51 5.3.1 Main Frame . 51 5.3.2 Configuration Frame . 52 5.3.3 Diagrams . 53 5.4 Modularisation . 55 5.5 Object Models . 57 5.5.1 Monitoring Server . 57 5.5.2 Monitoring Client . 59 6 Implementation 66 6.1 Socket Connection . 66 6.2 Application Measurements . 69 6.2.1 Standard Application . 69 III Contents 6.2.2 SRB Application . 71 6.3 XML Parser . 73 6.4 Client Measurement Thread . 74 6.5 Multiple Measurement Diagram . 77 7 Tests 82 7.1 Test with one Server . 82 7.1.1 Configuration . 82 7.1.2 Results . 84 7.2 Test with three Servers . 86 7.2.1 Configuration . 86 7.2.2 Results . 87 8 Conclusion 89 9 Future prospect 91 Abbreviations IX Bibliography X A Handbook XII A.1 Files . XII A.1.1 Monitoring server . XII A.1.2 Monitoring client . XII A.2 Installation and Configuration . XIII A.2.1 Ganglia . XIV A.2.2 Python . XV A.2.3 SQLite and Pysqlite . XV A.2.4 Measurement system . XVI A.3 Monitoring Server . XVII A.4 Console Client . XVII A.5 GUI Client . XVII B Python Scripts XXI B.1 python_server.py . XXI B.2 socket_connection.py . XXIV IV Contents B.3 ganglia.py . XXVIII B.4 python_client.py . XXXII B.5 console.py . XLVII B.6 gui2.py . LIII B.7 gui2_ui.py . LV B.8 myplot.py . LXXVII B.9 tooltip.py . LXXXIX C Bash Scripts XC C.1 average.sh . XC C.2 average_mem.sh . XC C.3 num_fd.sh . XCI V List of Figures 2.1 Ganglia Output of University Karlsruhe, Germany . 6 3.1 SRB network . 14 3.2 GNU-project logo . 20 3.3 Database models . 21 3.4 Chen notation . 23 3.5 Bachmann notation . 23 3.6 IDEF1X notation . 23 4.1 SRB server connection . 35 4.2 Client-Server connection . 36 4.3 Solution 1 . 37 4.4 Solution 2 . 38 5.1 Server interfaces . 41 5.2 Client interfaces . 44 5.3 Programme flow chart . 46 5.4 Entity Relationship Model . 48 5.5 Relational Database Model . 49 5.6 Main Frame . 51 5.7 Configuration Frame . 53 5.8 Single View Diagram . 54 5.9 Multiple View Diagram . 55 5.10 Modules . ..