DATABASE INTEGRATION GUIDE PIPELINE PILOT INTEGRATION COLLECTION 2017 Copyright Notice ©2016 Dassault Systèmes. All rights reserved. 3DEXPERIENCE, the Compass icon and the 3DS logo, CATIA, SOLIDWORKS, ENOVIA, DELMIA, SIMULIA, GEOVIA, EXALEAD, 3D VIA, BIOVIA and NETVIBES are commercial trademarks or registered trademarks of Dassault Systèmes or its subsidiaries in the U.S. and/or other countries. All other trademarks are owned by their respective owners. Use of any Dassault Systèmes or its subsidiaries trademarks is subject to their express written approval. Acknowledgments and References To print photographs or files of computational results (figures and/or data) obtained using BIOVIA software, acknowledge the source in an appropriate format. For example: "Computational results obtained using software programs from Dassault Systèmes BIOVIA. The ab initio calculations were performed with the DMol3 program, and graphical displays generated with Pipeline Pilot." BIOVIA may grant permission to republish or reprint its copyrighted materials. Requests should be submitted to BIOVIA Support, either through electronic mail to [email protected], or in writing to: BIOVIA Support 5005 Wateridge Vista Drive, San Diego, CA 92121 USA Contents Chapter 1: Introduction 1 Testing SQL Statements 22 Who Should Read this Guide 1 Error Handling 23 Requirements 1 Chapter 5: Customizing SQL Components 24 Supplied Database Drivers 1 Dynamic SQL Using String Replacement 24 Additional Information 2 Chapter 6: Building Protocols with Multiple Chapter 2: Configuring a Database 3 SQL Components 26 Overview 3 Chapter 7: Calling Database Stored Procedures 27 Pipeline Pilot Data Sources 3 Oracle Stored Procedures 27 Configuring Data Sources 4 Simple Stored Procedure 27 ODBC (PP) Data Sources 5 Simple Function 29 JDBC Data Sources 5 Stored Procedure with Result Sets 29 ODBC (DSN) Data Sources 5 Microsoft SQL Server Stored Procedures 32 MongoDB Data Sources 5 Simple Function 32 Connection Pooling 6 Stored Procedure with Result Sets 33 Configuring Data Source Access Rights 6 MySQL Stored Procedures 35 Authentication 6 Simple Stored Procedure 35 Initializing the Connection 7 Simple Function 37 Additional Information 7 Stored Procedure with Result Sets 37 Query Service Settings 7 Identifying Result Sets 39 Testing Data Source Connections 7 Appendices 41 Configuring an ODBC Data Source on the Server 7 Appendix A: ODBC Server Configuration 41 Support for Molecular Databases 8 Configuring an ODBC Data Source on Linux 44 Remote Access Files 8 Appendix B: Using dates with Oracle based components 45 Chapter 3: SQL Components Overview 9 Database Access 9 SQL Component Parameters 10 Parameter Mapping 10 Selecting a Data Source 12 Sharing SQL Component Connections 13 Batch Size 13 Handling of Date Values 14 Data Source Access Rights 14 Examples 14 View Data Source Rights 15 Use Data Source Rights 16 None Rights 17 Chapter 4: Building SQL Statements 19 Opening the SQL Builder 19 SQL Editing Modes 19 Graphically Building SQL Statements 20 Manually Constructing SQL Statements 21 SQL Editing Guidelines 21 Creating a Data Table 22 Chapter 1: Introduction The Pipeline Pilot Integration collection includes a set of components designed for use with Open Database Connectivity (ODBC) and Java Database Connectivity (JDBC) drivers. These drivers allow connections to a variety of compliant databases such as Oracle, SQL Server, and MS Access that reside anywhere on the network. Network databases can be accessed through the SQL components and configured on the server. Connections to these data sources are defined through the Pipeline Pilot Administration Portal. All information for the data source can be specified in the Administration Portal (including the necessary login and password) or this information can be provided from Data Service Names (DSNs) that are defined on the server using the ODBC Administrator tool. Who Should Read this Guide This guide provides information for integrating the SQL components with relational databases. This first chapter explains how to configure a Pipeline Pilot data source on your server (for administrators). The remaining chapters explain how to use SQL components to access data sources on your network (for client users). Requirements To configure an Pipeline Pilot data source, the Administration Portal role must be assigned to you. When using ODBC DSNs to specify the database information, you also need administrator privileges on the Pipeline Pilot server. To use the SQL components, you need some experience writing structured query statements. You also need a valid login and password to access the databases on your network. Supplied Database Drivers ODBC drivers from DataDirect are included with your Pipeline Pilot server installation. These drivers support connections to Oracle, Microsoft SQLServer, MySQL1 and DB2. These DataDirect drivers support faster access using a wire protocol and portability across platforms. No additional drivers or database software needs to be installed by administrators to let Pipeline Pilot communicate with databases using these drivers. In addition, Pipeline Pilot ships with a version of the Oracle JDBC driver for connecting to Oracle databases. To access other ODBC databases, install the appropriate driver on the server and create a DSN. To access other JDBC databases, install the appropriate driver jar file through the Administration Portal and configure the Data Source. Introduction | Page 1 Notes: Pipeline Pilot only officially supports the DataDirect drivers included with your product installation, although other drivers might continue to work. For full compatibility, use the DataDirect drivers for Pipeline Pilot. From the Pipeline Pilot Home Page, you can access documentation for Data Direct drivers that ship with Pipeline Pilot (select the "ODBC Driver Help" link in the Administrators section). Pipeline Pilot Clients cannot query a Microsoft Access database if your server runs on Linux. To use the installed drivers, you need to set up a new Data Source through the Administration Portal. Each Data Source maps a driver to a particular database on your network. Once a data source is defined, it is automatically displayed in Pipeline Pilot's Data Source dialog, allowing you to configure SQL components to read from that particular database. The Oracle ODBC driver from Data Direct does not support reading LOB data from tables which use SECUREFILE storage. If you have LOB data stored in tables that use SECUREFILE storage, you can read it using the supplied Oracle JDBC driver or you can install a different ODBC driver, such as Oracle's own ODBC driver. In the case of an updated ODBC driver on the Linux platform, the included driver replaces your existing version. On Windows you will have the option of continuing to use the previous version. See release notes for more details. In addition, you could use a custom ODBC DSN or a JDBC driver to connect to the database instead of using the latest Data Direct driver. 1The DataDirect drivers only work with the commercial version of MySQL. To connect to the community edition, use a MySQL-provided ODBC or JDBC driver. Additional Information For more information about the Pipeline Pilot Integration collection and other BIOVIA software products, visit https://community.3dsbiovia.com. Page 2 | Pipeline Pilot • Database Integration Guide Chapter 2: Configuring a Database Overview The Pipeline Pilot server can access databases that are ODBC or JDBC compliant (for example, Oracle and SQL Server). It can also read molecular data from ISIS and AEI databases. The server accesses these databases through a standard ODBC or JDBC layer. To communicate with the database, the ODBC layer uses an ODBC driver specific to the database type that is in use (such as, Oracle, DB2, etc.). Database configuration involves installing and configuring the appropriate driver on the server machine. The following diagram illustrates how the server communicates with an Oracle database using a JDBC driver: Pipeline Pilot communication with a database Protocols are edited on the client systems and protocol jobs run on the server. Therefore, the connection is made to a database source from the server. The server is the only machine that needs to be configured to communicate with the database. The client machines do not require configuration. To use the Pipeline Pilot ODBC or JDBC drivers, no additional installation is required. To use another ODBC driver, install the driver on the Pipeline Pilot server. To use a different JDBC driver, use the Administration Portal to upload the jar file to the server. Pipeline Pilot Data Sources To communicate with the database, a driver needs configuration information, such as server name, username, etc. This information is provided when the data sources are configured in the Administration Portal. If a data source has been configured by the Administrator, a user can select this data source on a SQL component in a protocol if the administrator has given the user access to the data source. When the protocol runs, the SQL component accesses the configuration of the data sources, passes this information to the driver software, and makes a connection to the database. The SQL component can then execute commands and retrieve information from this database. You can configure the following types of data source connections for use with Pipeline Pilot's SQL components: ODBC (DSN): ODBC-compliant databases that you configure to work with your own drivers that you install on the Pipeline Pilot server. For example, create a DSN to reference a Microsoft Access database. The data source includes all the information required to connect to the database, or it references the name of an ODBC DSN separately configured on the server. ODBC (PP): Data sources that use Pipeline Pilot's Data Direct drivers. JDBC: JDBC-compliant data sources that use driver JAR files that you install on the Pipeline Pilot server. Configuring a Database | Page 3 MongoDB: Mongo data sources defined in the Admin Portal. Tip: You can also store the username and password with the data source. This is convenient because this information does not need to be specified on each SQL component that uses the data source.