DOCSLIB.ORG

Data Streams, Complex Events, and BI Seth Grimes Alta Plana Corporation +1 301-270-0795

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Data Streams, Complex Events, and BI Seth Grimes Alta Plana Corporation +1 301-270-0795 Data Streams, Complex Events, and BI Seth Grimes Alta Plana Corporation +1 301-270-0795 -- http://altaplana.com International Data Warehouse & Business Intelligence Summit 2008 June 11-13, 2008 Agenda BI at the speed of thought → Analytics at the speed of data: the business case. Technologies and Solutions for real-time data and real-time analytics. Evaluating and implementing: Best Practices, strategy, and resources. Data Streams, Complex Events, and BI 3 Data Streams I worked for 3 years for NASA... Programming simulations of satellites that would map the earth‟s gravity field. Data involved simulated orbit altitude, inclination, duration, and the accuracy and measurement rate of instruments. http://media.skyandtelescope.com Instruments captured acceleration with 6 degrees of freedom – 3 spatial dimensions plus “attitude”: roll, pitch & yaw. I also worked on magnetic-field mapping where satellites would be ranged by ground-based lasers. Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 4 Data Streams Then I worked for 5 years for the U.S. National Highway Traffic Safety Administration... Vehicle crash tests (at that time) included up to 80-90 sensors on vehicles and dummies, 8000x/second data rate. Data was correlated to video. http://www.nhtsa.dot.gov/ Analyses were off-line. Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 5 Data Streams I then worked with economic data, especially time series... Observations are at calendar frequencies: yearly, monthly, weekly, etc. Econometric models link disparate data and enable forecasting. Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 6 Data Streams Data streams come from many sources. Data volumes, types, frequencies vary. Processing approach depends on use:Data-stream correlation; analysis techniques. Communications networks and capital markets created the market for new classes of software. Data stream processing (DSP). Complex event processing (CEP). Builds on BI and Operations Research. OR=mathematical modelling for automated decision making. Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 7 Data Streams Capital markets involve trades, bids/offers, news, statistics (e.g., P/E ratio, float), competitive intelligence, and economic & market data. Trading data includes symbol, volume, price, time. Many derivatives: Put & call options. Indexes & baskets. ForEx hedging. Monetizations. Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 8 Breakthrough Analysis Traditional BI is not enough. Historically focused rather than “operational” and predictive. Designed for reporting and interactive analysis rather than automated, embedded analytics. Issues with real-time (RT) data and real-time response. Tilt toward performance management, financials. http://www.dundasconsulting.com/ On the plus side, BI has: Dashboards. KPIs. Ability to handle time-based data. Growing RT capabilities. Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 9 Breakthrough Analysis Traditional DBMSes and DW approaches – Load → Index → Query – do not meet the speed need. Standard SQL and standard SQL engines don‟t support continuous queries. Common APIs (interfaces) – ODBC/JDBC, Analysis Services – are set/record oriented and do not provide adequate real-time query capabilities. Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 10 Breakthrough Analysis We need to cope with – Bigger data. Faster data. New, complex information types. Low-latency (“real-time”) processing. Analysis over data streams. Complex, statistically rooted algorithms. Automated decision making. => Complex Event Processing. Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 11 Problem Statement More data sources, more & faster data: RFID (radio frequency identifier) chips, especially for inventory and logistics. News and social media. Retail transactions. Communications networks. Sensor networks, security devices. Trading markets. Analytics driver: efficiency, speed, accuracy. Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 12 Problem Statement More data types: Surveillance video and audio. Sensor state information (e.g., temperature, humidity). Geolocation data, e.g., from GPS devices. Complex transactional information. Bids and offers. Full (“unstructured”) text. Coded and tagged (“structured”) text. Timestamps. Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 13 Problem Statement Challenges: Timely response. Capture of high-frequency data. Correlation of data from multiple sources. Long-running and time-aware queries. Issues: RDBMSes were designed for transaction processing. Load, structuring, and indexing time given traditional datamarts and data warehouses. Analytical styles: Traditional BI is historically focused. Algorithms: Scalability of traditional serial algorithms. Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 14 Responses Operational/Real-Time BI? Data Stream Processing (DSP). Complex Event Processing (CEP). ... Adaptive Data Warehousing. Column stores. Massively Parallel (database) Processing (MPP). In-memory analytics. Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 15 Operational/Real-time Data/BI Real Time BI: Fast response required. E.g., model scoring. Supports automated decision making. Operational BI on real-time data – What’s happening right now? – involves a bit more. Dashboards provide a great visualization tool. BI differs from monitoring: it contextualizes information. Be careful of “twinkling” data. But cleansing, structuring, indexing add latency. And we’re still not handling events or complexity. Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 16 Operational/Real-time Data/BI Most BI dashboards are not for RT data. Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 17 http://www.perceptualedge.com/blog/?p=154 Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 18 Events The concepts of events became familiar in the „90s in association with object-oriented (OO) programming. C++, Smalltalk, Java, and other languages. Software “objects” are invoked in response to program events such as the user’s pressing a button. Objects and events are managed in an application server. This conception did not cover data events. We did have DBMS based event handlers: stored procedures, triggers, alerts, etc. Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 19 Events Data events: • A shop in Bangkok seeks to authorize a charge on a credit card that was used one hour ago in Nairobi. • Inventory of widget X fall below five days‟ average sales. • The spread between the AU.L price in London and the AUY.F price in Frankfort exceeds 3%. Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 20 Data Stream Concepts Ordering: A stream is a sequence of observations over time. Correlation: Multiple streams may be related. Time and observation windows. Continuous/long-running queries. Use of SQL, treating streams like tables. Use of SQL joining streams and historical data. Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 21 Data Stream Concepts “Traditional ETL processes are limited to reading data at rest: from databases, mainframes, and files extracted from other operational systems. “Data in flight exists in other formats: messages on message- oriented middleware, web service calls, TCP network packets, and so forth... “A stream is analogous to a table in a relational database; but … contains an infinite sequence of rows that arrive whenever the producer decides to send them.” -- Julian Hyde, SQLstream architect (http://julianhyde.blogspot.com/2008/02/streaming-sql-meets-olap.html) Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 22 Data Streams and CEP Extend the concept to event streams. My definition: An “event” involves a data condition that is notable in some context. Those BI dashboards showed events (and more) but don‟t handle streams. Complex Event Processing. Feeds on data and event streams and “clouds.” Transforms input streams – whether data are atomic or complex – into output streams. Supports automated decisions via rules systems. Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 23 BI versus CEP http://www.eventstreamprocessing.com/ Copyright © 2008 Alta Plana Corporation International BI-DW Summit Data Streams, Complex Events, and BI 24 Data Stream Pioneers AT&T Labs created Hancock: A Language for Extracting Signatures from Data Streams, in 1999-2000. “Massive transaction streams present a number of opportunities for data mining techniques. Transactions might represent calls on a telephone network, commercial credit card purchases, stock market trades, or
Load more

Recommended publications
  • Distributed Complex Event Processing in Multiclouds
    Distributed Complex Event Processing in Multiclouds Vassilis Stefanidis1, Yiannis Verginadis1, Ioannis Patiniotakis1 and Gregoris Mentzas1 1 Institute of Communications and Computer Systems, National Technical University of Ath- ens, Greece {stefanidis, jverg, ipatini, gmentzas}@mail.ntua.gr Abstract. The last few years, the generation of vast amounts of heterogeneous data with different velocity and veracity and the requirement to process them, has significantly challenged the computational capacity and efficiency of the modern infrastructural resources. The propagation of Big Data among different pro- cessing and storage architectures, has amplified the need for adequate and cost- efficient infrastructures to host them. An overabundance of cloud service offer- ings is currently available and is being rapidly adopted by small and medium enterprises based on its many benefits to traditional computing models. However, at the same time the Big Data computing requirements pose new research chal- lenges that question the adoption of single cloud provider resources. Nowadays, we discuss the emerging data-intensive applications that necessitate the wide adoption of multicloud deployment models, in order to use all the advantages of cloud computing. A key tool for managing such multicloud applications and guarantying their quality of service, even in extreme scenarios of workload fluc- tuations, are adequate distributed monitoring mechanisms. In this work, we dis- cuss a distributed complex event processing architecture that follows automati- cally the big data application deployment in order to efficiently monitor its health status and detect reconfiguration opportunities. This proposal is examined against an illustrative scenario and is preliminary evaluated for revealing its performance results. Keywords: Distributed CEP, Cloud Monitoring, Multiclouds, Big Data.
    [Show full text]
  • Darko Anicic Event Processing and Stream Reasoning with ETALIS
    Darko Anicic Event Processing and Stream Reasoning with ETALIS Zur Erlangung des akademischen Grades eines Doktors der Wirtschaftswissenschaften (Dr. rer. pol.) von der Fakultät für Wirtschaftswissenschaften des Karlsruher Instituts für Technologie (KIT) genehmigte Dissertation. Event Processing and Stream Reasoning with ETALIS Dipl.-Ing. Darko Aniciˇ c´ Referent: Prof. Dr. Rudi Studer Korreferent: Prof. Dr. Opher Etzion Prüfer: Prof. Dr. Detlef Seese Vorsitzender der Prüfungskommission: Prof. Dr. Andreas Geyer-Schulz Tag der mündlichen Prüfung: 09. November 2011. Institut für Angewandte Informatik und Formale Beschreibungsverfahren (AIFB) Fakultät Wirtschaftswissenschaften Karlsruher Institut für Technologie (KIT) Karlsruhe 2011 i To Vedrana and Andre Abstract Event Processing (EP) is concerned with detection of situations under time constraints that are of a particular business interest. We face today a paradigm shift toward the real time information processing, and EP has therefore spawned significant attention in science and technology. Due to omnipresence of events, EP is becoming a central as- pect of new distributed systems such as cloud computing and grid systems, mobile and sensor-based systems, as well as a number of application areas including financial ser- vices, business intelligence, social and collaborative networking, click stream analysis and many others. However, there are a number of issues to be considered in order to enable effective event-based computation. A language for describing event patterns needs to feature a well-defined semantics. It also needs to be rich enough to express important classes of event patterns. Pattern matching should be supported in both, query-driven and event- driven modes. A number of other event operations, such as event aggregation, filtering, translation, enrichment and splitting, should be supported too.
    [Show full text]
  • Requirements and State of the Art Overview on Flexible Event Processing
    ICT, STREP FERARI ICT-FP7-619491 Flexible Event pRocessing for big dAta aRchItectures Collaborative Project D4.1 Requirements and state of the art overview of flexible event processing 01.02.2013 – 31.01.2014(preparation period) Contractual Date of Delivery: 31.01.2015 Actual Date of Delivery: 31.01.2015 Author(s): Fabiana Fournier and Inna Skarbovsky Institution: IBM Workpackage: Flexible Event Processing Security: PU Nature: R Total number of pages: 48 Project funded by the European Community under the Information and Communication Technologies Programme Contract ICT-FP7-619491 Project coordinator name Michael Mock Revision: 1 Project coordinator organisation name Fraunhofer Institute for Intelligent Analysis and Information Systems (IAIS) Schloss Birlinghoven, 53754 Sankt Augustin, Germany URL: http://www.iais.fraunhofer.de Abstract The goal of the FERARI (Flexible Event pRocessing for big dAta aRchItectures) project is to pave the way for efficient real-time Big Data technologies of the future. The proposed framework aims at enabling business users to express complex analytics tasks through a high-level declarative language that supports distributed complex event processing as an integral part of the system architecture. Work package 4 “Flexible Event Processing” deals with all the developments around event processing technologies in order to achieve this goal. In order to be flexible, event processing engines need to tackle the two following requirements in a satisfactory way: • The easy adaptability to non-functional requirements, specially, the way the tool copes with scalability issues in a distributed environment. • The easy definition and maintenance of the event-driven logic. The task of work package 4 is to provide a model and methodology to cope with these limitations.
    [Show full text]
  • Complex Event Processing As a Service in Multi-Cloud Environments
    Western University Scholarship@Western Electronic Thesis and Dissertation Repository August 2016 Complex Event Processing as a Service in Multi-Cloud Environments Wilson A. Higashino The University of Western Ontario Supervisor Dr. Miriam A. M. Capretz The University of Western Ontario Graduate Program in Electrical and Computer Engineering A thesis submitted in partial fulfillment of the equirr ements for the degree in Doctor of Philosophy © Wilson A. Higashino 2016 Follow this and additional works at: https://ir.lib.uwo.ca/etd Part of the Databases and Information Systems Commons Recommended Citation Higashino, Wilson A., "Complex Event Processing as a Service in Multi-Cloud Environments" (2016). Electronic Thesis and Dissertation Repository. 4016. https://ir.lib.uwo.ca/etd/4016 This Dissertation/Thesis is brought to you for free and open access by Scholarship@Western. It has been accepted for inclusion in Electronic Thesis and Dissertation Repository by an authorized administrator of Scholarship@Western. For more information, please contact [email protected]. Abstract The rise of mobile technologies and the Internet of Things, combined with advances in Web technologies, have created a new Big Data world in which the volume and velocity of data gen- eration have achieved an unprecedented scale. As a technology created to process continuous streams of data, Complex Event Processing (CEP) has been often related to Big Data and used as a tool to obtain real-time insights. However, despite this recent surge of interest, the CEP market is still dominated by solutions that are costly and inflexible or too low-level and hard to operate. To address these problems, this research proposes the creation of a CEP system that can be o↵ered as a service and used over the Internet.
    [Show full text]
  • Join Query Optimization Techniques for Complex Event Processing Applications
    Join Query Optimization Techniques for Complex Event Processing Applications Ilya Kolchinsky Assaf Schuster Technion, Israel Institute of Technology Technion, Israel Institute of Technology Haifa 32000 Israel Haifa 32000 Israel [email protected] [email protected] ABSTRACT Keywords Complex event processing (CEP) is a prominent technology Stream Processing, Complex Event Processing, Lazy Eval- used in many modern applications for monitoring and track- uation, Query Optimization ing events of interest in massive data streams. CEP engines inspect real-time information ows and attempt to detect 1. INTRODUCTION combinations of occurrences matching predened patterns. Complex event processing has become increasingly im- This is done by combining basic data items, also called portant for applications in which arbitrarily complex pat- primitive events, according to a pattern detection plan, terns must be eciently detected over high-speed streams in a manner similar to the execution of multi-join queries of events. Online nance, security monitoring, and fraud in traditional data management systems. Despite this sim- detection are among the many examples. Pattern detection ilarity, little work has been done on utilizing existing join generally consists of collecting primitive events and combin- optimization methods to improve the performance of CEP- ing them into potential (partial) matches using some type based systems. of detection model. As more events are added to a partial In this paper, we provide the rst theoretical and experi- match, a full pattern match is eventually formed and re- mental study of the relationship between these two research ported. Popular CEP mechanisms include nondeterministic areas. We formally prove that the CEP Plan Generation nite automata (NFAs) [5, 18, 51], nite state machines [6, problem is equivalent to the Join Query Plan Generation 45], trees [36], and event processing networks [21, 42].
    [Show full text]
  • From Complex Event Processing to Simple Event Processing
    From Complex Event Processing to Simple Event Processing Sylvain Hallé1 Abstract Many problems in Computer Science can be framed as the computation of queries over sequences, or “streams” of data units called events. The field of Complex Event Processing (CEP) relates to the techniques and tools developed to efficiently process these queries. However, most CEP systems developed so far have concentrated on relatively narrow types of queries, which consist of sliding windows, aggregation functions, and simple sequential patterns computed over events that have a fixed tuple structure. Many of them boast throughput, but in counterpart, they are difficult to setup and cumbersome to extend with user-defined elements. This paper describes a variety of use cases taken from real-world scenarios that present features seldom considered in classical CEP problems. It also provides a broad review of current solutions, that includes tools and techniques going beyond typical surveys on CEP. From a critical analysis of these solutions, design principles for a new type of event stream processing system are exposed. The paper proposes a simple, generic and extensible framework for the processing of event streams of diverse types; it describes in detail a stream processing engine, called BeepBeep, that implements these principles. BeepBeep’s modular architecture, which borrows concepts from many other systems, is complemented with an extensible query language, called eSQL. The end result is an open, versatile, and reasonably efficient query engine that can be used in situations that go beyond the capabilities of existing systems. Keywords: event processing, software testing, query languages, runtime verification 1. Introduction taken from domains as varied as bug detection in video games and network intrusion detection.
    [Show full text]
  • BPMN Extension for Business Process Monitoring
    BPMN Extension forBusiness Process Monitoring∗ Anne Baumgrass, Nico Herzberg, Andreas Meyer,Mathias Weske Hasso Plattner Institute at the University of Potsdam Prof.-Dr.-Helmert-Str.2–3, D-14482 Potsdam, Germany {Anne.Baumgrass,Nico.Herzberg,Andreas.Meyer, Mathias.Weske}@hpi.de Abstract: The execution of business processes generates alot of data representing happenings (also called events) that may be utilized for process monitoring and analysis. This, however, is not supported by typical Business Process Management Systems (BPMSs). Especially,inmanual executing business process environments, i.e., not drivenbyaBPMS,the correlation of events to processes for monitoring and analysis is not trivial. At design-time, ProcessEvent Monitoring Points are usedinprocess models to specify the locations, where particular events are expected. Therewith, occurring events can be assigned to aprocess during run-time. In this paper,weintroduce an extension to BPMN,which implements this connection between process models and events. We showapplicability of this extension by applying it to alogistics scenario taken from an EU project. 1Introduction Nowadays, the availability and creation of process-relevant information in terms of events1 increases substantially,e.g. through the Internet of Things, Big Data as well as newand faster in-memory and data streaming technologies. Furthermore, sensors such as GPS receivers, RFID chips, transponders, detectors, cameras, or satellites enable the depiction of the current status of processes. Although the amount of events would lead to afine-grained monitoring, mining, and decision support for business processes, alarge number of business processes controlled by a BPMS operate without them. Especially,monitoring events of business processes not controlled by asingle system and across enterprise boundaries is valuable for gaining insights about business process execution [Luc02, Luc11], e.g., to ensure abusiness process is executed as expected.
    [Show full text]
  • A COMPLEX EVENT PROCESSING SYSTEM for MONITORING of MANUFACTURING SYSTEMS Master of Science Thesis
    JORGE A. GARCIA IZAGUIRRE MONTEMAYOR A COMPLEX EVENT PROCESSING SYSTEM FOR MONITORING OF MANUFACTURING SYSTEMS Master of Science Thesis Examiner: Professor José L. M. Lastra Examiner and topic approved in the Automation, Mechanical and Materials Engineering Faculty Council meeting on 7 Mar 2012 I Abstract TAMPERE UNIVERSITY OF TECHNOLOGY Master’s Degree Programme in Information Technology GARCIA IZAGUIRRE MONTEMAYOR, JORGE A.: A Complex Event Processing system for monitoring of manufacturing systems Master of Science Thesis, 84 pages, 12 Appendix pages February 2012 Major: Factory Automation Examiner: Prof. José Luis Martínez Lastra Keywords: complex event processing, event-driven architecture, service oriented architecture, production engineering, web services, factory automation, OPC-UA, DPWS Future manufacturing systems will require to process large amounts of complex data due to a rising demand on visibility and vertical integration of factory floor devices with higher level systems. Systems contained in higher layers of the business model are rapidly moving towards a Service Oriented Architecture, inducing a tendency to push Web Technologies down to the factory floor level. Evidence of this trend is the addition of Web Services at the device level with Device Profile for Web Services and the transition of OPC based on COM/DCOM communication to OPC- UA based on Web Services. DPWS and OPC-UA are becoming nowadays the preferred options to provide on a device level, service-oriented solutions capable to extend with an Event Driven Architecture into manufacturing systems. This thesis provides an implementation of a factory shop floor monitor based on Complex Event Processing for event-driven manufacturing processes. Factory shop monitors are particularly used to inform the workshop personnel via alarms, notifications and, visual aids about the performance and status of a manufacturing process.
    [Show full text]
  • Integration and Exploitation of Sensor Data in Smart Cities Through Event-Driven Applications
    sensors Article Integration and Exploitation of Sensor Data in Smart Cities through Event-Driven Applications Manuel Garcia Alvarez * , Javier Morales and Menno-Jan Kraak Geo-information Processing Department, ITC Faculty, University of Twente, Hengelosestraat 99, 7514 AE Enschede, The Netherlands; [email protected] (J.M.); [email protected] (M.-J.K.) * Correspondence: [email protected]; Tel.: +31-53-487-4444 Received: 3 February 2019; Accepted: 14 March 2019; Published: 19 March 2019 Abstract: Smart cities are urban environments where Internet of Things (IoT) devices provide a continuous source of data about urban phenomena such as traffic and air pollution. The exploitation of the spatial properties of data enables situation and context awareness. However, the integration and analysis of data from IoT sensing devices remain a crucial challenge for the development of IoT applications in smart cities. Existing approaches provide no or limited ability to perform spatial data analysis, even when spatial information plays a significant role in decision making across many disciplines. This work proposes a generic approach to enabling spatiotemporal capabilities in information services for smart cities. We adopted a multidisciplinary approach to achieving data integration and real-time processing, and developed a reference architecture for the development of event-driven applications. This type of applications seamlessly integrates IoT sensing devices, complex event processing, and spatiotemporal analytics through a processing workflow for the detection of geographic events. Through the implementation and testing of a system prototype, built upon an existing sensor network, we demonstrated the feasibility, performance, and scalability of event-driven applications to achieve real-time processing capabilities and detect geographic events.
    [Show full text]
  • Join Query Optimization Techniques for Complex Event Processing Applications
    Join Query Optimization Techniques for Complex Event Processing Applications Ilya Kolchinsky Assaf Schuster Technion, Israel Institute of Technology Technion, Israel Institute of Technology Haifa 32000 Israel Haifa 32000 Israel [email protected] [email protected] ABSTRACT Keywords Complex event processing (CEP) is a prominent technology Stream Processing, Complex Event Processing, Lazy Eval- used in many modern applications for monitoring and track- uation, Query Optimization ing events of interest in massive data streams. CEP engines inspect real-time information ows and attempt to detect 1. INTRODUCTION combinations of occurrences matching predened patterns. Complex event processing has become increasingly im- This is done by combining basic data items, also called portant for applications in which arbitrarily complex pat- primitive events, according to a pattern detection plan, terns must be eciently detected over high-speed streams in a manner similar to the execution of multi-join queries of events. Online nance, security monitoring, and fraud in traditional data management systems. Despite this sim- detection are among the many examples. Pattern detection ilarity, little work has been done on utilizing existing join generally consists of collecting primitive events and combin- optimization methods to improve the performance of CEP- ing them into potential (partial) matches using some type based systems. of detection model. As more events are added to a partial In this paper, we provide the rst theoretical and experi- match, a full pattern match is eventually formed and re- mental study of the relationship between these two research ported. Popular CEP mechanisms include nondeterministic areas. We formally prove that the CEP Plan Generation nite automata (NFAs) [5, 18, 51], nite state machines [6, problem is equivalent to the Join Query Plan Generation 45], trees [36], and event processing networks [21, 42].
    [Show full text]
  • Complex Event Processing for Self-Optimizing Cellular Networks
    sensors Article Complex Event Processing for Self-Optimizing Cellular Networks Isabel de-la-Bandera 1,* , Matías Toril 1 , Salvador Luna-Ramírez 1 , Víctor Buenestado 1 and José María Ruiz-Avilés 2 1 Campus Teatinos, University of Málaga, 29071 Málaga, Spain; [email protected] (M.T.); [email protected] (S.L.-R.); [email protected] (V.B.) 2 Ericsson, Parque Tecnológico de Andalucía, 29590 Málaga, Spain; [email protected] * Correspondence: [email protected] Received: 25 February 2020; Accepted: 27 March 2020; Published: 30 March 2020 Abstract: In a cellular network, signaling and data messages exchanged between network elements are an extremely valuable information for network optimization. The consideration of different types of information allows to improve the optimization results. However, the huge amount of information has made it very difficult for operators to process all the available information. To cope with this issue, in this paper, a methodology for processing cell and user connection traces to optimize a live cellular network is presented. The aim is to generate new performance indicators different from those supplied by manufacturers, taking advantage of the ability of complex event processing tools to correlate events of different nature. For illustrative purposes, an example of how a new performance indicator is created from real traces by complex event processing is given. Keywords: big data; complex event processing; LTE; mobile network; trace 1. Introduction Mobile communications have experienced an unprecedented evolution that has led to a rapid increase in the number of mobile users and services. To cope with these changes, new radio access technologies and network architectures have been developed, which has increased the size and complexity of mobile networks.
    [Show full text]
  • Enhancing Stream Processing and Complex Event Processing Systems Abderrahmen Kammoun
    Enhancing Stream Processing and Complex Event Processing Systems Abderrahmen Kammoun To cite this version: Abderrahmen Kammoun. Enhancing Stream Processing and Complex Event Processing Systems. Net- working and Internet Architecture [cs.NI]. Université de Lyon, 2019. English. NNT : 2019LYSES012. tel-02468246 HAL Id: tel-02468246 https://tel.archives-ouvertes.fr/tel-02468246 Submitted on 5 Feb 2020 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. No d’ordre NNT: 2019LYSE012 THESE DE DOCTORAT DE L’UNIVERSITE DE LYON opérée au sein de L’UNIVERSITE JEAN MONNET Ecole Doctoral N o 488 Sciences, Ingénierie et Santé Spécialité de doctorat: Discipline : Informatique Soutenue publiquement le 08/07/2019, par: Abderrahmen Kammoun Enhancing Stream Processing and Complex Event Processing Systems Devant le jury composé de : Jean-Marc Petit, PR, Institut National des Sciences Appliquées de Lyon, Rapporteur Yann Busnel, PR, École Nationale Supérieure Mines-Télécom Atlantique Bretagne Pays de la Loire, Rapporteur Frédérique Laforest, PR, Institut National des Sciences Appliquées de Lyon, Examinatrice Jacques Fayolle, PR, Université Jean Monnet, Directeur de thèse Kamal Singh, MCF, Université Jean Monnet, Co-Directeur Je dédie cette thèse : À mon père Habib Kammoun.
    [Show full text]