PUBLIC SAP Event Stream Processor 5.1 SP08 Document Version: 1.0 - 2014-06-25 Programmers Guide Table of Contents 1 Introduction.................................................................. 7 1.1 Data-Flow Programming.......................................................... 7 1.2 Continuous Computation Language..................................................9 1.3 SPLASH......................................................................9 1.4 Authoring Methods..............................................................9 2 CCL Project Basics.............................................................11 2.1 Events.......................................................................11 2.2 Operation Codes...............................................................12 2.3 Streams.....................................................................13 2.4 Delta Streams.................................................................14 2.5 Keyed Streams................................................................ 15 2.6 Windows.....................................................................18 2.6.1 Retention............................................................. 19 2.6.2 Named Windows........................................................22 2.6.3 Unnamed Windows......................................................23 2.7 Comparing Streams, Windows, Delta Streams, and Keyed Streams...........................24 2.8 Bindings on Streams, Delta Streams, and Windows...................................... 27 2.9 Input/Output/Local.............................................................30 2.10 Implicit Columns...............................................................31 2.11 Event Block Lifecycle............................................................32 2.12 Schemas.................................................................... 32 2.13 Stores...................................................................... 33 2.14 CCL Continuous Queries......................................................... 34 2.15 Reference Table Queries......................................................... 35 2.16 Adapters.................................................................... 36 2.17 Order of Elements..............................................................36 3 CCL Language Components......................................................37 3.1 Datatypes....................................................................37 3.1.1 Intervals..............................................................40 3.1.2 Choice Between decimal and money Datatypes.................................. 41 3.2 Operators....................................................................41 3.3 Expressions.................................................................. 45 3.4 CCL Comments................................................................46 3.5 Case-Sensitivity............................................................... 47 3.6 Literals......................................................................48 3.6.1 Time Literals...........................................................48 3.6.2 Boolean Literals........................................................ 50 PUBLIC Programmers Guide 2 © 2014 SAP AG or an SAP affiliate company. All rights reserved. Table of Contents 3.6.3 String Literals..........................................................50 3.6.4 Numeric Literals........................................................ 51 4 Developing a Project in CCL..................................................... 53 5 CCL Query Construction........................................................54 5.1 Filtering.....................................................................54 5.2 Splitting Up Incoming Data........................................................54 5.3 Unions......................................................................56 5.3.1 Example: Merging Data from Streams or Windows................................56 5.4 Joins....................................................................... 57 5.4.1 Key Field Rules.........................................................59 5.4.2 Join Examples: ANSI Syntax................................................60 5.4.3 Join Example: Comma-Separated Syntax...................................... 62 5.5 Pattern Matching...............................................................63 5.6 Aggregation..................................................................64 5.7 Reference Table Queries......................................................... 65 5.7.1 Using a Reference Table Query in a Join........................................67 5.7.2 Using a Reference Table Query in a Module.....................................68 5.7.3 Using a Reference Table Query in SPLASH......................................70 5.7.4 Using a Reference Table Query in SPLASH to Get a Single Row from a Table..............72 6 Advanced CCL Programming Techniques........................................... 74 6.1 Declare Blocks................................................................ 74 6.1.1 Typedefs..............................................................75 6.1.2 Parameters............................................................76 6.1.3 Variables..............................................................77 6.1.4 Declaring Project Variables, Parameters, Datatypes, and Functions....................78 6.2 Flex Operators.................................................................79 6.3 Automatic Partitioning...........................................................80 6.3.1 Guidelines for Partitioning Flex Operators......................................83 6.3.2 Guidelines for Partitioning SPLASH...........................................86 6.3.3 Guidelines for Partitioning Elements with Retention Policies.........................88 6.3.4 Guidelines for Partitioning Aggregation........................................90 6.3.5 Guidelines for Partitioning Modules...........................................91 6.3.6 Guidelines for Partitioning Joins.............................................93 6.3.7 Guidelines for Partitioning Elements on Log Stores................................95 6.4 Modularity................................................................... 95 6.4.1 Module Creation and Usage................................................96 6.4.2 Example: Creating and Using Modules.........................................96 6.4.3 Example: Parameters in Modules............................................ 97 6.5 Data Recovery.................................................................98 Programmers Guide PUBLIC Table of Contents © 2014 SAP AG or an SAP affiliate company. All rights reserved. 3 6.5.1 Log Store Optimization Techniques..........................................100 6.6 Error Streams................................................................100 6.6.1 Monitoring Streams for Errors..............................................102 7 Zero Data Loss.............................................................. 104 7.1 Consistent Recovery........................................................... 104 7.2 Auto Checkpoint..............................................................105 7.3 Guaranteed Delivery............................................................107 7.4 Achieving Zero Data Loss........................................................109 8 Creating a Log Store.......................................................... 112 8.1 Log Store Guidelines............................................................113 8.2 Sizing a Log Store..............................................................114 8.3 Log Store Sizing Reference.......................................................118 9 Writing SPLASH Routines...................................................... 122 9.1 Internal Pulsing...............................................................122 9.2 Order Book..................................................................123 10 Integrating SPLASH into CCL................................................... 125 10.1 Access to the Event............................................................125 10.2 Access to Input Windows........................................................ 125 10.3 Output Statement............................................................. 127 10.4 Notes on Transactions..........................................................127 11 Using SPLASH in Projects......................................................128 12 ESP Datatypes in SQL Queries...................................................131 13 SAP PowerDesigner for Event Stream Processor.....................................133 13.1 Getting Started............................................................... 133 13.1.1 Data Modeling Scenarios................................................. 133 13.1.2 Sample PowerDesigner Project.............................................134 13.1.3 Learning More About PowerDesigner.........................................134 13.2 Data Model..................................................................135 13.2.1 ESP Schema Logical Data Model............................................135 13.2.2 Finding an Object in a Diagram............................................. 135 13.2.3 Data Model Tables......................................................136 13.2.4 Extensions............................................................137 13.3 Extended Model Setup..........................................................139