Requirements and State of the Art Overview on Flexible Event Processing
Total Page:16
File Type:pdf, Size:1020Kb
ICT, STREP FERARI ICT-FP7-619491 Flexible Event pRocessing for big dAta aRchItectures Collaborative Project D4.1 Requirements and state of the art overview of flexible event processing 01.02.2013 – 31.01.2014(preparation period) Contractual Date of Delivery: 31.01.2015 Actual Date of Delivery: 31.01.2015 Author(s): Fabiana Fournier and Inna Skarbovsky Institution: IBM Workpackage: Flexible Event Processing Security: PU Nature: R Total number of pages: 48 Project funded by the European Community under the Information and Communication Technologies Programme Contract ICT-FP7-619491 Project coordinator name Michael Mock Revision: 1 Project coordinator organisation name Fraunhofer Institute for Intelligent Analysis and Information Systems (IAIS) Schloss Birlinghoven, 53754 Sankt Augustin, Germany URL: http://www.iais.fraunhofer.de Abstract The goal of the FERARI (Flexible Event pRocessing for big dAta aRchItectures) project is to pave the way for efficient real-time Big Data technologies of the future. The proposed framework aims at enabling business users to express complex analytics tasks through a high-level declarative language that supports distributed complex event processing as an integral part of the system architecture. Work package 4 “Flexible Event Processing” deals with all the developments around event processing technologies in order to achieve this goal. In order to be flexible, event processing engines need to tackle the two following requirements in a satisfactory way: • The easy adaptability to non-functional requirements, specially, the way the tool copes with scalability issues in a distributed environment. • The easy definition and maintenance of the event-driven logic. The task of work package 4 is to provide a model and methodology to cope with these limitations. The proposed approach addresses both the functional and non-functional properties of event processing applications by supporting non-technical users with a declarative language expressed in tabular forms. The outcome model can be then automatically translated into event driven definitions and eventually into a running application in the proposed FERARI architecture. D4.1 Requirements and state of the art overview on flexible event processing Revision history Administration Status Project acronym: FERARI ID: ICT-FP7-619491 Document identifier: D4.1 Requirements and state of the art overview of flexible event processing (01.02.2013 – 31.01.2014) Leading Partner: IBM Report version: 1 Report preparation date: 31.01.2014 Classification: PU Nature: REPORT Author(s) and contributors: Fabiana Fournier and Inna Skarbovsky Status: - Plan - Draft - Working - Final x Submitted Copyright This report is © FERARI Consortium 2014. Its duplication is restricted to the personal use within the consortium and the European Commission. www.ferari-project.eu D4.1 Requirements and state of the art overview on flexible event processing Document History Version Date Author Change Description 0.1 15/11/2014 Fabiana Fournier (IBM) First draft 0.2 1/12/2014 Fabiana Fournier (IBM) Second draft including sections 3 and 4 0.3 15/12/2014 Fabiana Fournier (IBM) First complete version 0.4 15/12/2014 Fabiana Fournier (IBM) Inclusion of abstract 0.5 15/12/2014 Fabiana Fournier (IBM) Updates per internal review 1.0 30/12/2014 Fabiana Fournier (IBM) Final fixes and cleanup D4.1 Requirements and state of the art overview on flexible event processing Table of Contents 1 Introduction .......................................................................................................................................... 1 1.1 Purpose and scope of the document ............................................................................................ 1 1.2 Relationship with other documents ............................................................................................. 1 2 Complex event processing – The motivation ........................................................................................ 1 3 Complex event processing – The business case ................................................................................... 4 4 State of the art in complex event processing tools .............................................................................. 6 4.1 Commercial tools .......................................................................................................................... 8 4.1.1 InfoSphere Streams (IBM) [18] [19] ....................................................................................... 9 4.1.2 Informatica Platform for streaming analytics (Informatica), ................................................ 9 4.1.3 Event Stream Processor (ESP) (SAP) [18] [19] ........................................................................ 9 4.1.4 Apama (Software AG) [18] [19] ........................................................................................... 10 4.1.5 StreamBase (Tibco) [18] [19] .............................................................................................. 10 4.2 Open source engines ................................................................................................................... 10 4.2.1 Esper (EsperTech Inc) .......................................................................................................... 11 4.2.2 IBM Proactive Technology Online (PROTON)...................................................................... 11 4.2.3 Open source event processing running on distributed stream computing platforms ....... 12 4.3 Research tools ............................................................................................................................. 13 4.4 Limitations of contemporary event processing tools ................................................................. 14 5 Complex event processing background .............................................................................................. 14 5.1 Event types .................................................................................................................................. 15 5.2 Event attributes .......................................................................................................................... 16 5.3 Context ........................................................................................................................................ 16 5.4 Event Processing Network (EPN) ................................................................................................ 17 5.5 Event Processing Agent (EPA) ..................................................................................................... 17 5.6 Pattern policies ........................................................................................................................... 18 5.7 Context initiator policies ............................................................................................................. 19 5.8 PROTON definitions .................................................................................................................... 20 6 Requirements for flexible event processing ....................................................................................... 21 D4.1 Requirements and state of the art overview on flexible event processing 6.1 Non-functional requirements of event processing applications ................................................ 21 6.1.1 Scalability ............................................................................................................................ 22 6.1.2 Availability ........................................................................................................................... 22 6.1.3 Security ............................................................................................................................... 23 6.1.4 Performance objectives ...................................................................................................... 23 6.1.5 Usability............................................................................................................................... 24 6.2 Requirements for the mobile fraud use case ............................................................................. 26 6.2.1 Description of the mobile fraud use case ........................................................................... 27 6.2.1 Event types .......................................................................................................................... 28 6.2.2 Event processing agents ...................................................................................................... 29 6.2.3 Mobile phone fraud use case functional requirements summary ..................................... 35 6.3 Introduction to the event model ................................................................................................ 35 6.4 Summary of the requirements for flexible event processing in FERARI ..................................... 36 7 Summary and future steps .................................................................................................................. 36 8 References .......................................................................................................................................... 38 List of Tables Table 1: Initial EPN for the mobile phone fraud use case ........................................................................... 28 D4.1 Requirements and state of the art overview on flexible event