Project Report - CERN Summer Student Program 2014 Icinga Monitoring System Interface The University of Manchester - Department of Computer Science Author: Supervisor: Alina-Georgiana Neculae Olivier Chaze August 23, 2014 Abstract The aim of this project is to develop a web interface that would be used by the Icinga monitoring system to manage the CMS online cluster, in the experimental site. The interface would allow users to visualize the informa- tion in a compressed and intuitive way, as well as modify the information of each individual object and edit the relationships between classes. Acknowledgments I would like to thank my supervisors Olivier Chaze and Lavinia Darlea for their continued support and careful guidance over the course of this project. 1 Contents 1 Introduction 3 1.1 Motivation . 3 2 Database motivation and configuration 5 2.1 Database . 5 3 Tools used for development 8 3.1 Front end . 8 3.2 Back end . 9 4 Interface 10 4.1 Read-only interface . 10 4.2 Read-write interface . 12 5 Conclusions and future work 15 2 Chapter 1 Introduction Icinga is an open source network and computer system monitoring applica- tion, which was originally created as a fork to Nagios, in 2009. Since then, the developers of Icinga have tremendously improved the software, in order to overcome some of Nagios's perceived short-comings. Through a continuous reflection of the community's needs, some of the fea- tures added to Icinga are: a modern Web 2.0 style user interface, additional database connectors for MySQL, Oracle and PostgreSQL and a REST API that allows the integration of numerous extensions without any additional modifications to the Icinga core. The CMS System Administration Team uses Icinga to manage between 2500 and 3000 devices and their services, for the online cluster in the experimen- tal site. At the moment, the software relies on a flat file configuration to retrieve the cluster organization and any specific device information. Motivation The need for this project arises from the fact that, each time the system is updated or a new device is introduced, the Icinga flat files would have to be modified manually for that change to be perceived by the software. In response to this problem, a database containing all the relevant information about the cluster, was created, and through a series of scripts, the database can be used to generate the series of flat files required by Icinga. The project presented in the following chapters represents a front end to the database, and it enables the users to view the environment in a more struc- tured and compact way. Furthermore, it allows operations like the insertion 3 or removal of hosts to be monitored, the addition, removal or modification of checks and so on. In addition to this, given the time, the interface developed needs to be inte- grated with the central management system, used to install and remember the profile of each machine, Puppet. Puppet, is then used to determine the processes that need to be monitored and would update the database accord- ingly, using the developed interface. 4 Chapter 2 Database motivation and configuration The CMS System Administration Team currently monitors over 2500 de- vices, with a total of 50 000 checks. The Icinga flat files document what hosts need to be monitored and which checks need to be performed on each of those, for example: available disk space, number of processes running and so on. When a new machine has to be introduced into the environment, or the role of a machine has to be modified, the changes have to be made manually to each file, thus leaving room for errors or omissions, as a result what is currently monitored does not cover the entire infrastructure anymore. Due to the fact that the configuration file structure is very similar to a database schema, it was decided that translating the files into such a for- mat would greatly facilitate the application of any required modifications to the system. Especially since the database structure could be automatically translated to the flat files required by Icinga, by applying a series of scripts. Database The database structure presented above describes the current configuration of the CMS cluster. The diagram shows how the main group of objects interact and form dependencies, as well as the main characteristics that de- fine each class. The central classes that are used by the system are: Host, Service, Command, Contacts. 5 Each fundamental class is then organized into groups, depending on the properties they have in common as well as the requirements they share. An- other use of the groups is to describe dependencies between large groups of basic objects, one example of this is shown by the ContactGroupHost table, which suggests that a group of contacts is responsible for a particular host and should be contacted in case of emergencies. Understanding the complexity of the relationships that can be described by the central classes, was the first step required for the completion of this project. One of the main requirements of this project was to develop a user friendly and intuitive way of displaying the complex relationships created, while maintaining the readability and usability of the system. 6 ScheduledDowntime ScheduledDowntimeID INT(10) HostScheduledDowntime SDTComment CHAR(100) FK_HSDT_HName CHAR(100) 1..* 1 SDTAuthor CHAR(100) FK_HSDT_ScheduledDowntimeID INT(10) SDTStartTime DATETIME Indexes SDTEndTime DATETIME 1..* Indexes HostDependency TimePeriod FK_HD_HName CHAR(100) TPName CHAR(100) FK_HD_PName CHAR(100) TPMondayStart TIME Indexes TPMondayEnd TIME TPTuesdayStart TIME 1..* 1..* HostHostGroup TPTuesdayEnd TIME 1..* FK_HHG_HGName CHAR(100) TPWednesdayStart TIME FK_HHG_HName CHAR(100) TPWednesdayEnd TIME Indexes 1 1 1 1 TPThursdayStart TIME 1..* TPThursdayEnd TIME Host 1 TPFridayStart TIME 1..* HName CHAR(100) 1 HostGroupDependency 1..*1 TPFridayEnd TIME FK_H_TPName CHAR(100) HostGroup FK_HGD_HGName CHAR(100) TPSaturdayStart TIME Indexes 1..*1 HGName CHAR(100) FK_HGD_MemberName CHAR(100) TPSaturdayEnd TIME Indexes Indexes TPSundayStart TIME 1..* 1 1 0..1 1 TPSundayEnd TIME ContactGroupHost Indexes ServiceHost 1..* FK_CGH_CGName CHAR(100) 1..* FK_SH_HName CHAR(100) FK_CGH_HName CHAR(100) FK_SH_SName CHAR(100) 1 Indexes ContactGroup Indexes CGName CHAR(100) 1 Indexes 1..* 1..* 1 ContactGroupService Service FK_CGS_CGName CHAR(100) SName CHAR(100) FK_CGS_SName CHAR(100) SService_description CHAR(100) Indexes 1..* SActive_checks_enabled TINYINT(4) SPassive_checks_enabled TINYINT(4) ContactContactGroup 1..* SParallelize_check TINYINT(4) FK_CCG_CName CHAR(100) 1 1..* SObsess_over_service TINYINT(4) FK_CCG_CGName CHAR(100) SCheck_freshness TINYINT(4) Indexes SNotifications_enabled TINYINT(4) 1..* SEvent_handler_enabled TINYINT(4) 1 SFlap_detection_enabled TINYINT(4) Contact SFlap_detection_options CHAR(100) CName CHAR(100) 0..1 SFailure_prediction_enabled TINYINT(4) 1 1..* CAlias CHAR(100) SProcess_perf_data TINYINT(4) SRetain_status_information TINYINT(4) CMail CHAR(100) ServiceHostGroup SRetain_nonstatus_inf7 ormation TINYINT(4) CPhone CHAR(100) 1..* FK_SHG_HGName CHAR(100) Indexes SIs_volatile TINYINT(4) FK_SHG_SName CHAR(100) SCheck_period CHAR(100) Indexes 0..1 SMax_check_attempts TINYINT(4) 1..* 1 1..* SNormal_check_interval INT(10) SCheck_interval INT(10) CommandContact SRetry_check_interval INT(10) ServiceGroup FK_CC_ContactName CHAR(100) SContact_groups CHAR(100) SGName CHAR(100) FK_CC_CommandName CHAR(100) SNotification_options CHAR(100) Indexes CC_NotificationType CHAR(100) SNotification_interval INT(11) Indexes SNotification_period CHAR(100) 1 1..* 1 SRegister TINYINT(4) 1..* SAction_url CHAR(100) 1..* 0..1 SParameter CHAR(100) ServiceServiceGroup FK_S_SName CHAR(100) Command 1..* FK_SSG_SName CHAR(100) FK_S_CName CHAR(100) CName CHAR(100) 0..1 FK_SSG_SGName CHAR(100) FK_S_HGName CHAR(100) CLine VARCHAR(500) Indexes Indexes Indexes , 0 Figure 2.1: Database schema used to generate the Icinga configuration files. Chapter 3 Tools used for development The tools used for the development of this project, were chosen after careful consideration. The main areas that were investigated, in order to determine the perfect candidates were: the documentation or resources provided, since the system has to be easily maintainable; the scalability of the application developed; the stability and speed offered, as well as the level of complexity of the environment, the tools used require to function. Front end For this part of the interface, a number of tools were investigated, but after careful consideration, the JavaScript InfoVis Toolkit was chosen as the main library used for visualizing the data and the relationships formed. This li- brary provides web standard based tools to create interactive data visualiza- tions for web applications. Besides the fact that it provides a large number of visualization techniques, such as PieChart, Sunsurst, Force-Directed graphs and many more, it is also very fast, relying on json objects to generate dia- grams. Furthermore, it allows a large degree of specialization such as touch events, drag and drop and a large number of other complex animations that can have any style property required. The remaining part of the interface elements were developed using HTML5 and the Twitter Bootstrap library, due to their large usability and online resources.These tools were selected due to their ability to deliver rich con- tent without needing any external plugins, their speed of execution and the increased stability they offer the application. 8 Back end Even though a NoSQL