BB Installation Guide s6

SAP Event Stream Processing (SAP ESP)/ SAP HANA smart data streaming

June 2015 English

SAP Event Stream Processing (ESP)/Smart Data Streaming (SDS) Deployment for Signal Intelligence (EB8) Building Block Configuration Guide

SAP SE Dietmar-Hopp-Allee 16 69190 Walldorf Germany SAP Best Practices SAP ESP/SDS Deployment for Signal Intelligence (EB8): Configuration Guide

Copyright

© 2015 SAP SE or an SAP affiliate company. All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company. SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. Please see http://global.sap.com/corporate-en/legal/copyright/index.epx#trademark for additional trademark information and notices. Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors. National product specifications may vary. These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP SE or its affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty. In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.

Revision Change Date Description 0 June 6, 2015 Initial

Icon Meaning Caution

Example

Recommendation

Syntax

Typographic Conventions

Type Style Description Example text Words or characters that appear on the screen. These include field names, screen titles, pushbuttons as well as menu names, paths and options. Cross-references to other documentation. Example text Emphasized words or phrases in body text, titles of graphics and tables. EXAMPLE TEXT Names of elements in the system. These include report names, program names, transaction codes, table names, and individual key words of a programming language, when surrounded by body text, for example, SELECT and INCLUDE. Example text Screen output. This includes file and directory names and their paths, messages, source code, names of variables and parameters as well as names of installation, upgrade and database tools.

EXAMPLE TEXT Keys on the keyboard, for example, function keys (such as F2) or the ENTER key. Example text Exact user entry. These are words or characters that you enter in the system exactly as they appear in the documentation. Variable user entry. Pointed brackets indicate that you replace these words and characters with appropriate entries.

Content 1 Purpose...... 6 2 Preparation...... 6 2.1 Prerequisites...... 6 2.1.1 Nearline Storage Options...... 6 2.1.2 Initialize Environment Variables...... 6 3 Deploying Signal Solution...... 8 3.1 Configure Deployment Parameters...... 8 3.2 Deploy rds_bdi_signal Project...... 8 3.2.1 Confirming the Component Install...... 9 3.2.2 Verify Environment...... 9 3.3 Configure SAP IQ [Optional]...... 9 3.4 Configuration for Hadoop [Optional]...... 9 3.5 Bootstrap Solution to SAP ESP or SDS Cluster...... 9 3.6 Administration of Solution...... 11 3.6.1 Stopping and Removing Project...... 11 3.6.2 Restarting project...... 11 3.7 Extension to solution...... 11 4 Appendix...... 13 4.1 Additional Information...... 13 4.1.1 SAP smart data streaming (SAP SDS)...... 13 4.1.2 SAP Event Stream Processor (SAP ESP)...... 13 4.2 Set up UNIX ODBC...... 13 4.2.1 Installation of UNIX ODBC driver...... 13 4.3 Configuration for Hadoop [Optional]...... 14 4.4 Configuration for SAP IQ [Optional]...... 14

1 Purpose The purpose of this document is to describe the general configuration steps required to manually configure SAP® Event Stream Processing (SAP ESP) or SAP HANA® smart data streaming (SDS) for Signal Intelligence within the SAP HANA landscape that has already been installed using the corresponding installation or configuration guide for installation.

2 Preparation 2.1 Prerequisites Before installing this scenario, you must install the prerequisite building blocks. For more information, see the Latest Information and Configuration Guide: Getting Started on the SAP Service Marketplace for this solution, http://service.sap.com/rds-bdi.

SAP HANA smart data streaming or SAP Event Stream Processor version SP09 or higher must be installed before configuring this solution.

The configuration guide assumes that the ODBC driver is configured in the SAP ESP or SAP HANA smart data streaming deployment server. If this has not been done, contact your system administrator to install the ODBC libraries on the ESP/SDS installation server. The appendix section, Set up UNIX ODBC driver, describes the high-level steps to complete this activity. Ensure that the latest guidelines for the ODBC driver setup are followed, as required by the driver providers.

The SAP HANA ODBC client driver must be installed in the ESP/SDS server. The path of the SAP HANA ODBC driver install library path is used in the configuration setup. 2.1.1Nearline Storage Options SAP IQ (Optional) If SAP IQ is selected as nearline storage, then product prerequisites for SAP IQ must be completed. Ensure that you have access to the SAP IQ user and password for configuration. Additionally, ensure that the environment variable $SYBASE is set using the command echo $SYBASE and then the previous command should resolve to a path and not an empty string.

HADOOP (Optional) If HADOOP is chosen as nearline storage, then ensure that product prerequisites are fulfilled to enable effective operation of this solution with HADOOP using SAP ESP or SAP HANA smart data streaming. For more information, see the Appendix Configuration for Hadoop [Optional].

2.1.2Initialize Environment Variables The following environment variables must be initialized before proceeding further with solution configuration.

UNIX setup:

© SAP SE Page 6 of 14 SAP Best Practices SAP ESP/SDS Deployment for Signal Intelligence (EB8): Configuration Guide Ensure that the shell logon profile, which could be.profile or.bashrc, or the appropriate shell profile and this file are located in the home folder. The Python export path refers to standard Python libraries, which are delivered with the SAP HANA installation. Refer to your SAP HANA installation and adjust the path as needed. To update the profile file, use the following export:

Verify your local environment for python installation and adapt the following path: export PYTHONPATH=/usr/sap/hdbclient:/usr/sap/hdbclient/hdbcli:/usr/sap/hdbc lient/Python/lib:$PYTHONPATH export SAP_HANA_ODBC_PATH= .

When SAP ESP or SAP HANA smart data streaming are co-hosted on the same landscape, then ff usually resolves to /usr/sap//XX/exe/, where SID refers to the SAP HANA server system identifier and XX refers to SAP HANA instance number. For example, if SID is HDB and instance is 00, then the path typically is /usr/sap/HDB/HDB00/exe/. Confirm with your system administrator for the SAP HANA ODBC installation path.

Verify that $STREAMING_HOME is available on the installation server. $STREAMING_HOME can be verified by issuing echo $STREAMING_HOME command in the terminal session. The $STREAMING_HOME profile parameter resolves to the installation home folder of either SAP HANA smart data streaming or SAP ESP. Contact your system administrator when $STREAMING_HOME is not available in the installation server.

If PYTHONPATH and/or SAP_HANA_ODBC_PATH is added to the user profile, then the profile must be initialized by issuing the following command: $shell>source . Additionally, the initialization is effected by starting a new terminal session.

3 Deploying Signal Solution Setting up SAP HANA smart data streaming or SAP ESP technical components for the solution requires the following high-level processes: 1. Configure deployment parameters and optional Nearline Storage parameters 2. Deploy rds_bdi_signal project for either SAP EPS or SAP HANA smart data streaming installation 3. Bootstrap the solution to SAP ESP or SAP HANA smart data streaming 3.1 Configure Deployment Parameters Implementation-specific parameters are configured using the following URL: http(s)://SERVER:80XX/sap/rds-bdi/mfg/signal/config.html Where XX refers to your SAP HANA instance number and SERVER is your server IP or DNS address. Relevant configuration parameters are included in the following settings: 1. SAP HANA – Ensure that the SAP HANA Signal Admin User is assigned. 2. SAP ESP or SAP HANA smart data streaming -- This category includes server access URI a. SAP ESP installation can be either esp or esps b. SAP HANA smart data streaming must be specified as esps. This solution pre-delivers a default value of esps; Additionally, the SAP ESP or SAP HANA smart data streaming user must have an administration role access to perform start and stop for either SAP ESP or SAP HANA smart data streaming. 3. Signal Nearline Storage (Optional) – The solution provides the option to configure SAP IQ and HADOOP for nearline storage. This is optional and depends on requirements. The feature can be enabled any time after the initial configuration. 3.2 Deploy rds_bdi_signal Project The SAP HANA smart data streaming environment for the rapid-deployment solution consists of a group of files supporting the operation of the system. These files configure automatically, based on local settings for server IDs, logon credentials, and the SAP HANA server environment.

SAP ESP or SAP HANA smart data streaming have two logical workspace definitions: 1. SAP HANA studio, where source code is developed 2. The deployed server, where runtime code is staged to be loaded into the SAP ESP or SAP HANA smart data streaming cluster.

SAP ESP or SAP HANA smart data streaming use the same physical base directory for all of its processing. This includes the storage of source code, executable code, configuration, and adapter management.

The installation of the provided solution components requires extracting the contents of rds_bdi_signal.tar file to the SAP HANA smart data streaming projects base directory as follows: cd $STREAMING_HOME/cluster/projects

3.2.1 Confirming the Component Install Once the unzip process has completed, verify the directory structure with the following command: ls –l $STREAMING_HOME/cluster/projects

Verify that the directory listing includes the following folders:  rds_bdi_signal  repository-signal No further configuration activity is required for this step.

3.2.2 Verify Environment To verify the environment, from the folder location: $STREAMING_HOME/cluster/projects/repository-signal Execute following command $shell>./verifyEnvironment.sh If the environment is configured correctly; then the following result displays: Environment verified for install When the results indicate errors, either PYTHONPATH is not configured or SAP_HANA_ODBC_PATH is not configured for the environment 3.3 Configure SAP IQ [Optional] No additional steps are required for configuring SAP IQ when SAP IQ is installed and the credentials are available for configuring the parameters as described in Appendix Configuration for SAP IQ [Optional] 3.4 Configuration for Hadoop [Optional] SAP ESP and SAP HANA smart data streaming provide options to configure the destination of the Hadoop HDFS store. Refer to SAP ESP or SAP HANA smart data streaming configuration documentation, and configure the landscape with any required libraries, as described in the product installation guide. For more information about optional Hadoop-specific configuration, see the Appendix Configuration for Hadoop [Optional]

For more information about loading the data to Hadoop Hive table, see the SAP Note 1956985.

3.5 Bootstrap Solution to SAP ESP or SDS Cluster The following step is performed at initial configuration as well as when the solution configuration is modified. This section describes the process for adding the signal integration solution to either the SAP ESP or SAP HANA smart data streaming cluster (SDS).

From the following folder path: $STREAMING_HOME/streaming/STREAMING-1/cluster/projects/repository- signal

The user executing the step must have read access to the Signal solution.

./bootstrap.sh 3XX15 SOLUTION_VIEWER_USER Password

If the command is successful, then the following message displays: ***************************************************** * The Bootstrap process has completed ***************************************************** Next, execute the following on the command prompt: ps aux | grep –I rds_bdi_signal

If the previous command returns the process details, then the process was successful.

In addition, the following steps can be performed to verify successful deployment of the signal solution to ESP/SDS cluster. From the $STREAMING_HOME/streaming/STREAMING- 1/cluster/projects/repository-signal folder path, execute the following command and to quit the ESP/SDS process shell, execute quit followed by return. ./admin.sh > get projects ======Workspace: default Project: rds_bdi_signal Instance Count: 1 ------Instance Details ------Instance Name: 0 Controller Name: xxxxx Current Status: started-running Requested Status: started-running Failure Interval: 0 Failures Per Interval: 0 ======>quit

The previous steps can be executed as often as necessary, but we recommend to execute signal_stop.sh as described in section Stopping and removing project before repeating this section.

Any changes made to the configuration using the link in section Configure deployment parameters takes effect only when the solution project is restarted (as described in section Restarting project ).

© SAP SE Page 10 of 14 SAP Best Practices SAP ESP/SDS Deployment for Signal Intelligence (EB8): Configuration Guide 3.6 Administration of Solution The following scripts can be used for managing the solution.

3.6.1Stopping and Removing Project To stop the project and remove it from the ESP or SDS cluster, issue following command ./signal_stop.sh {SYS_USER} {Password} Where SYS_USER is ESP or SDS user with administration role and Password is that user’s password. When using SAP HANA smart data streaming deployment, ensure that the SYS_USER is granted the roles described for the Integration Admin User in the Signal Intelligence Configuration for Monitoring and Predictive Maintenance (EZ6) configuration guide.

When performing the bootstrap process, non-sensitive information is cached for future process management, which are named as .__solutionUri__ and .__solutionPort and are stored as hidden files. In case of accidental deletion of these files, the previous script fails. Your alternative to supply SOLUTION_URI and SOLUTION_PORT is provided in the error message.

3.6.2Restarting project To restart the project, issue following command: ./signal_restart.sh {SYS_USER} {Password} Where SYS_USER is ESP or SDS user with administration role. In the case of SAP HANA smart data streaming deployment, ensure that the SYS_USER has been granted the roles described for the Integration Admin User in the Signal Intelligence Configuration for Monitoring and Predictive Maintenance (EZ6) configuration guide.

When performing the bootstrap process, non-sensitive information is cached for future process management, which are named as .__solutionUri__ and .__solutionPort and are stored as hidden files. In case of accidental deletion of these files, the previous script fails. Your alternative to supply SOLUTION_URI and SOLUTION_PORT is provided in the error message.

3.7 Extension to solution If future extension is planned after the initial install, then the following steps are used as guidelines to load the delivered CCL files to the project.

This solution does not support custom extension to the delivered code line.

If you modify the SAP ESP source code for your specific requirements, the following rules must be followed, because the bootstrap script modifies the SAP ESP source files. We recommend making no changes on the server. Any changes to the actual source code file are lost whenever the bootstrap script is run. The source code file of record for the STREAM project is in the folder, along with the bootstrap script. The files are named:  rds_bdi_signal.ccl_tokenized – The SDS source code for the STREAM project  rds_bdi_signal.ccr_tokenized – The SDS resource file for the STREAM project

We recommend as the procedure for modifying the source code is to archive and compress the code into a ZIP file in the project directory from under <$STREAMING_HOME/cluster/projects>, and transfer it to the computer where SAP HANA studio is installed. After transferring to the development machine, rename the files with the extension “_tokenized”, removing this suffix.

To accommodate the configuration properties defined in the AT_BDI_CONFIG view, while also being syntactically correct for development and testing, a special coding convention has been defined.

The values in the AT_BDI_CONFIG view are represented in the files as tokens, with a format of “$$$token-name$$$”. A token may represent a server domain, a database table name, a timer setting, and so on.

The SAP SDS/ESP compiler interprets the lines of code containing tokens and then the following coding convention is applied. When a line of code requires a token, these lines are commented out using the ‘//’ character sequence. Immediately following this, the actual command line without the token is listed. This allows the code to run in SAP HANA studio without error. During the deployment process with the bootstrap program, the tokens are replaced and the comment characters are removed from the current line. The bootstrap program immediately applies comment characters to the following line.

When the completed SAP HANA smart data streaming project is ready to be deployed back to the server, perform the following steps:  Compress the directory into a ZIP file, and transfer the file to the server.  Uncompress the directory ZIP file and replace the existing source code directory under <$STREAMING_HOME/cluster/projects >  Copy the ccl and ccr files from the source code directory to the directory, with the suffix of “_tokenized”. For example, the files in the directory would be named: o rds_bdi_signal.ccl_tokenized o rds_bdi_signal.ccr_tokenized

4 Appendix 4.1 Additional Information 4.1.1SAP smart data streaming (SAP SDS) For information about installing, configuring, and optimizing SAP SDS, see the resource listed in the table: Topic Guide/Tool Quick Link SAP Smart SAP Help http://help.sap.com/hana_options_sds Data Portal Product documentation is based on version and SP level Streaming

4.1.2SAP Event Stream Processor (SAP ESP) For information about installing, configuring, and optimizing SAP ESP, see the resource listed in the table: Topic Guide/Tool Quick Link SAP Event SAP Help http://help.sap.com/esp Stream Processor Portal Product documentation is based on version and SP level

4.2 Set up UNIX ODBC 4.2.1 Installation of UNIX ODBC driver

The following section is provided for reference only. Always refer to the latest installation guides, provided by the provider of ODBC drivers. To install the UNIX ODBC drivers, the open source software “unixODBC” is approved by SAP for use in UNIX and Linux environments. The following steps provide the installation procedure for the unixODBC driver.

The file contains Data Source Names (DSNs), which are used to identify specific database connections. The unixODBC system supports two levels of the , a global one and a local one that is unique for each user on the system.

The global system file name is stored in: /usr/local/etc/odbc.ini. The local file name is stored in: ${HOME}/.odbc.ini.

To perform the installation, the following steps must be completed: 1. Create a temporary folder for the ODBC installation files. mkdir 2. Change current directory to this new temporary folder. cd 3. Download the unixODBC installer to the current directory. WGET FTP example wget ftp://ftp.unixodbc.org/pub/unixODBC/unixODBC-2.3.2.tar.gz

© SAP SE Page 13 of 14 SAP Best Practices SAP ESP/SDS Deployment for Signal Intelligence (EB8): Configuration Guide 4. Extract the contents of the unixODBC installer, which automatically creates a subdirectory named unixODBC-2.3.2.

tar -zxvf unixODBC-2.3.2.tar.gz

5. Change to the subdirectory “unixODBC-2.3.2”.

cd unixODBC-2.3.2

6. Perform the installation using the “configure” and “make” commands.

The following commands generate a large output. Check for error messages.

./configure make make install make clean

4.3 Configuration for Hadoop [Optional] If Hadoop is selected for nearline storage, to store the processed log information from SAP ESP or SAP HANA smart data streaming, ensure that parameter - HADOOP_ON is marked as TRUE using the link provided in Configure deployment parameters .

If you have marked HADOOP_ON as True in AT_BDI_CONFIG view of SAP HANA Schema, then ensure that you have configured SDS to work with Hadoop system.

Follow the SDS or SAP ESP (depending on the implementation) product guides to import the required Hadoop libraries to the SAP ESP or SDS server. 4.4 Configuration for SAP IQ [Optional] If SAP IQ is selected for nearline storage, then make sure that SAP IQ is installed and that the appropriate SAP IQ client libraries are configured in the SAP ESP or SDS server, as described in the product recommendations. Ensure IQ_ON is marked as TRUE using the link provided in Configure deployment parameters