Solving Big Data Challenges for Enterprise Application Performance Management Tilmann Rabl Mohammad Sadoghi Hans-Arno Jacobsen Middleware Systems Middleware Systems Middleware Systems Research Group Research Group Research Group University of Toronto, Canada University of Toronto, Canada University of Toronto, Canada
[email protected] [email protected] [email protected] Sergio Gomez-Villamor´ Victor Muntes-Mulero´ Serge Mankovskii DAMA-UPC CA Labs Europe CA Labs Universitat Politecnica` de Barcelona, Spain San Francisco, USA Catalunya, Spain
[email protected] [email protected] [email protected] ABSTRACT 1. INTRODUCTION As the complexity of enterprise systems increases, the need for Large scale enterprise systems today can comprise complete data monitoring and analyzing such systems also grows. A number of centers with thousands of servers. These systems are heteroge- companies have built sophisticated monitoring tools that go far be- neous and have many interdependencies which makes their admin- yond simple resource utilization reports. For example, based on istration a very complex task. To give administrators an on-line instrumentation and specialized APIs, it is now possible to monitor view of the system health, monitoring frameworks have been de- single method invocations and trace individual transactions across veloped. Common examples are Ganglia [20] and Nagios [12]. geographically distributed systems. This high-level of detail en- These are widely used in open-source projects and academia (e.g., 1 ables more precise forms of analysis and prediction but comes at Wikipedia ). However, in industry settings, in presence of stringent the price of high data rates (i.e., big data). To maximize the benefit response time and availability requirements, a more thorough view of data monitoring, the data has to be stored for an extended period of the monitored system is needed.