Attributed Graph Rewriting for Complex Event Processing Self-Management Wilson A
Total Page:16
File Type:pdf, Size:1020Kb
Attributed Graph Rewriting for Complex Event Processing Self-Management Wilson A. Higashino, Cédric Eichler, Miriam Capretz, Luiz F. Bittencourt, Thierry Monteil To cite this version: Wilson A. Higashino, Cédric Eichler, Miriam Capretz, Luiz F. Bittencourt, Thierry Monteil. At- tributed Graph Rewriting for Complex Event Processing Self-Management. ACM Transactions on Autonomous and Adaptive Systems, Association for Computing Machinery (ACM), 2016, 11 (3), pp.article n° 19. 10.1145/2967499. hal-01369701 HAL Id: hal-01369701 https://hal.archives-ouvertes.fr/hal-01369701 Submitted on 26 Sep 2016 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Distributed under a Creative Commons Attribution - NonCommercial - ShareAlike| 4.0 International License Attributed Graph Rewriting for Complex Event Processing Self-Management WILSON A. HIGASHINO, Western University/University of Campinas CEDRIC´ EICHLER, INSA CVL MIRIAM A. M. CAPRETZ, Western University LUIZ F. BITTENCOURT, University of Campinas THIERRY MONTEIL, LAAS-CNRS/INSA Toulouse The use of Complex Event Processing (CEP) and Stream Processing (SP) systems to process high-volume, high-velocity Big Data has renewed interest in procedures for managing these systems. In particular, self- management and adaptation of runtime platforms have been common research themes, as most of these systems run under dynamic conditions. Nevertheless, the research landscape in this area is still young and fragmented. Most research is performed in the context of specific systems, and it is difficult to generalize the results obtained to other contexts. To enable generic and reusable CEP/SP system management procedures and self-management policies, this research introduces the Attributed Graph Rewriting for Complex Event 19 Processing Management (AGeCEP) formalism. AGeCEP represents queries in a language- and technology- agnostic fashion using attributed graphs. Query reconfiguration capabilities are expressed through stan- dardized attributes, which are defined based on a novel classification of CEP query operators. By leveraging this representation, AGeCEP also proposes graph rewriting rules to define consistent reconfigurations of queries. To demonstrate AGeCEP feasibility, this research has used it to design an autonomic manager and to define a selected set of self-management policies. Finally, experiments demonstrate that AGeCEP can indeed be used tor develop algorithms that can be integrated into diverser CEP systems. CCS Concepts: Information systems → Data streaming;r Computer systems organization → Self-organizingr autonomic computing; Cloud computing; Theory of computation → Rewrite sys- tems; Software and its engineering → Specialized application languages Additional Key Words and Phrases: Complex event processing, autonomic computing, self-management, attributed graph, graph rewriting ACM Reference Format: Wilson A. Higashino, Cedric´ Eichler, Miriam A. M. Capretz, Luiz F. Bittencourt, and Thierry Monteil. 2016. Attributed graph rewriting for complex event processing self-management. ACM Trans. Auton. Adapt. Syst. 11, 3, Article 19 (September 2016), 39 pages. DOI: http://dx.doi.org/10.1145/2967499 1. INTRODUCTION Current technological trends such as the Internet of Things and mobile computing are causing a massive explosion in the volume and velocity of data generated every This work was partially supported by an NSERC CRD at Western University (CRDPJ 453294-13). Authors’ addresses: W. A. Higashino and M. A. M. Capretz, Dept. of Electrical and Computer Engineering, Thompson Engineering Building, Western University, London ON, Canada, N6A 5B9; emails: {whigashi, mcapretz}@uwo.ca; C. Eichler, INSA Centre Val de Loire, LIFO, 88 boulevard Lahitolle CS 60013 18022 Bourges, France; email: [email protected]; T. Monteil, CNRS, LAAS, 7 avenue du Colonel Roche, F- 31400 Toulouse, France / Universite´ de Toulouse, CNRS, INSA, F-31400 Toulouse, France; email: monteil@ laas.fr; L. F.Bittencourt, Institute of Computing, University of Campinas, Av.Albert Einstein 1251, Campinas SP, Brazil, 13083-852; email: [email protected]. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. 2016 Copyright is held by the owner/author(s). Publication rights licensed to ACM. ACM 1556-4665/2016/09-ART19 $15.00 DOI: http://dx.doi.org/10.1145/2967499 ACM Transactions on Autonomous and Adaptive Systems, Vol. 11, No. 3, Article 19, Publication date: September 2016. 19:2 W. A. Higashino et al. day. Clearly, processing and understanding this enormous amount of data requires innovative approaches. Complex Event Processing (CEP) and Stream Processing (SP) are technologies com- monly applied in this new Big Data reality. Both CEP and SP are concerned with processing continuous data flows coming from distributed sources to obtain timely re- sponses to queries [Cugola and Margara 2012]. These queries are defined by the user through a set of operators and aim to extract knowledge from incoming data. Because of their generality, the tasks to which CEP and SP are applied range from simple mon- itoring to highly complex financial applications such as fraud detection and automated trading [Grolinger et al. 2014]. Despite this recent surge of interest, the current CEP/SP research and development landscape is still young and particularly fragmented. A large variety of solutions exist, and they often use inconsistent terminology and different query definition languages. Consequently, most ongoing research is performed in the context of specific systems and languages. Of particular interest for this research, algorithms and techniques aimed at query lifecycle management have often been developed in such a system-specific fashion. For instance, Aurora* can dynamically move processing load to neighbouring servers [Cherniack et al. 2003], and Nephele can dynamically resize the output buffers of query operators [Lohrmann et al. 2013]. Both these examples illustrate important query management techniques, in which the system self-adapts to changing conditions. However, they were developed in the context of their respective systems and therefore cannot be easily generalized. More recently, these limitations have become even more pronounced due to two recent trends in CEP/SP research: —Many modern systems [Gulisano et al. 2012; Qian et al. 2013] use cloud infrastruc- tures as runtime environments to leverage cloud performance and inherent elasticity. Management of such large deployments has led to more complex algorithms and re- inforces the need to reuse them; —Recent CEP/SP systems are being offered as services [Amazon 2015; Google 2015] and accept user-defined operators. In this context, the ability to integrate these operators in the query management loop becomes essential to these systems. This research presents Attributed Graph Rewriting for Complex Event Processing Management (AGeCEP), a novel formalism that enables creation of generic and reusable procedures for CEP/SP query management. To achieve this goal, AGeCEP pro- vides a language-agnostic extensible representation of queries and of reconfiguration actions that specify how queries can be transformed. Once an algorithm is expressed by this formalism, it can be integrated into diverse modern cloud-based CEP/SP systems and applied to an extensible set of query operators. In this way, AGeCEP overcomes the specificity of current CEP/SP research and enables creation of universal algorithms and strategies. More specifically, this research develops the following contributions: —The query lifecycle management (QLM) concept as a composition of four main steps: single-query optimization, multi-query optimization, operator placement, and run- time management; —A novel classification of query operators focused on their reconfiguration capabilities; —AGeCEP, the formalism used to represent CEP/SP queries and reconfiguration ac- tions in a way that is both language-agnostic and appropriate for query management. This formalism is based on the classification just mentioned, attributed directed acyclic graphs, and graph rewriting rules; —The design of an autonomic manager based on AGeCEP and the MAPE-K frame- work [IBM 2006] that can be used to control modern CEP systems. ACM Transactions on Autonomous and Adaptive Systems, Vol. 11, No. 3, Article 19, Publication date: September 2016. Attributed Graph Rewriting for CEP Self-Management 19:3 The fitness of this proposal is then investigated with regard to the following: —its feasibility as demonstrated by a general methodology that can be used to adapt operator