Connectivity and Knowledge Modules Guide for Oracle Data Integrator
Total Page:16
File Type:pdf, Size:1020Kb
Oracle® Fusion Middleware Connectivity and Knowledge Modules Guide for Oracle Data Integrator 12c (12.2.1.4.0) E95630-03 November 2020 Oracle Fusion Middleware Connectivity and Knowledge Modules Guide for Oracle Data Integrator, 12c (12.2.1.4.0) E95630-03 Copyright © 2010, 2020, Oracle and/or its affiliates. Primary Author: Oracle Corporation This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable: U.S. GOVERNMENT END USERS: Oracle programs (including any operating system, integrated software, any programs embedded, installed or activated on delivered hardware, and modifications of such programs) and Oracle computer documentation or other Oracle data delivered to or accessed by U.S. Government end users are "commercial computer software" or "commercial computer software documentation" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, reproduction, duplication, release, display, disclosure, modification, preparation of derivative works, and/or adaptation of i) Oracle programs (including any operating system, integrated software, any programs embedded, installed or activated on delivered hardware, and modifications of such programs), ii) Oracle computer documentation and/or iii) other Oracle data, is subject to the rights and limitations specified in the license contained in the applicable contract. The terms governing the U.S. Government’s use of Oracle cloud services are defined by the applicable contract for such services. No other rights are granted to the U.S. Government. This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. Intel and Intel Inside are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Epyc, and the AMD logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group. This software or hardware and documentation may provide access to or information about content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services unless otherwise set forth in an applicable agreement between you and Oracle. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services, except as set forth in an applicable agreement between you and Oracle. Contents Preface Audience xxvii Documentation Accessibility xxvii Related Documents xxvii Conventions xxviii 1 Introduction 1.1 Terminology 1-1 1.2 Using This Guide 1-2 1.3 Accessing Data in the Relational Structure 1-2 1.4 Accessing Data in the Schemas 1-2 Part I Databases, Files, and XML 2 Oracle Database 2.1 Introduction 2-1 2.1.1 Concepts 2-1 2.1.2 Knowledge Modules 2-1 2.2 Installation and Configuration 2-4 2.2.1 System Requirements and Certifications 2-4 2.2.2 Technology Specific Requirements 2-4 2.2.2.1 Using the SQL*Loader Utility 2-4 2.2.2.2 Using External Tables 2-4 2.2.2.3 Using Oracle Wallet 2-5 2.2.3 Connectivity Requirements 2-6 2.3 Setting up the Topology 2-7 2.3.1 Creating an Oracle Data Server 2-7 2.3.1.1 Creation of the Data Server 2-7 2.3.2 Creating an Oracle Physical Schema 2-9 2.4 Setting Up an Integration Project 2-9 iii 2.5 Creating and Reverse-Engineering an Oracle Model 2-10 2.5.1 Create an Oracle Model 2-10 2.5.2 Reverse-engineer an Oracle Model 2-10 2.6 Setting up Changed Data Capture 2-11 2.7 Setting up Data Quality 2-12 2.8 Designing a Mapping 2-12 2.8.1 Loading Data from and to Oracle 2-13 2.8.1.1 Loading Data from Oracle 2-13 2.8.1.2 Loading Data to Oracle 2-13 2.8.2 Integrating Data in Oracle 2-14 2.8.3 Designing an ETL-Style Mapping 2-15 2.9 Troubleshooting 2-18 2.9.1 Troubleshooting Oracle Database Errors 2-18 2.9.2 Common Problems and Solutions 2-19 3 Oracle Autonomous Data Warehouse Cloud 3.1 Introduction 3-1 3.1.1 Concepts 3-1 3.1.2 Knowledge Modules 3-1 3.2 Prerequisites 3-3 3.3 Setting up the Topology 3-5 3.3.1 Creating an Oracle Data Server 3-5 3.3.2 Creating an Oracle Physical Schema 3-7 3.4 Creating and Reverse-Engineering an Oracle Model 3-8 3.4.1 Create an Oracle Model 3-8 3.4.2 Reverse Engineer an Oracle Model 3-8 3.5 Designing a Mapping 3-8 3.5.1 Loading data 3-9 3.5.1.1 Loading Data using Oracle KMs 3-9 3.5.1.2 Loading Data using SQL* Loader KMs 3-9 3.5.1.3 Loading Data directly into ADWC 3-10 3.5.1.4 Loading Oracle Object Storage files into ADWC 3-33 3.5.2 Extracting data 3-40 3.6 Best Practices for Working with ADWC 3-40 3.6.1 Caching Oracle Sequences in ADWC 3-40 4 Files 4.1 Introduction 4-1 4.1.1 Concepts 4-1 iv 4.1.2 Knowledge Modules 4-1 4.2 Installation and Configuration 4-2 4.2.1 System Requirements and Certifications 4-2 4.2.2 Technology Specific Requirements 4-2 4.2.3 Connectivity Requirements 4-5 4.3 Setting up the Topology 4-5 4.3.1 Creating a File Data Server 4-5 4.3.1.1 Creation of the Data Server 4-6 4.3.2 Creating a File Physical Schema 4-8 4.4 Setting Up an Integration Project 4-8 4.5 Creating and Reverse-Engineering a File Model 4-9 4.5.1 Create a File Model 4-9 4.5.2 Reverse-engineer a File Model 4-9 4.5.2.1 Delimited Files Reverse-Engineering 4-9 4.5.2.2 Fixed Files Reverse-engineering using the Wizard 4-10 4.5.2.3 COBOL Copybook reverse-engineering 4-11 4.5.2.4 Customized Reverse-Engineering 4-12 4.6 Designing a Mapping 4-14 4.6.1 Loading Data From Files 4-14 4.6.2 Integrating Data in Files 4-16 4.6.2.1 IKM SQL to File Append 4-16 4.6.2.2 IKM File to File (Java) 4-17 5 Generic SQL 5.1 Introduction 5-1 5.1.1 Concepts 5-1 5.1.2 Knowledge Modules 5-2 5.2 Installation and Configuration 5-4 5.2.1 System Requirements and Certifications 5-5 5.2.2 Technology-Specific Requirements 5-5 5.2.3 Connectivity Requirements 5-5 5.3 Setting up the Topology 5-5 5.3.1 Creating a Data Server 5-5 5.3.2 Creating a Physical Schema 5-5 5.4 Setting up an Integration Project 5-6 5.5 Creating and Reverse-Engineering a Model 5-6 5.5.1 Create a Data Model 5-6 5.5.2 Reverse-engineer a Data Model 5-6 5.6 Setting up Changed Data Capture 5-7 5.7 Setting up Data Quality 5-7 v 5.8 Designing a Mapping 5-7 5.8.1 Loading Data From and to an ANSI SQL-92 Compliant Technology 5-7 5.8.1.1 Loading Data from an ANSI SQL-92 Compliant Technology 5-7 5.8.1.2 Loading Data to an ANSI SQL-92 Compliant Technology 5-8 5.8.2 Integrating Data in an ANSI SQL-92 Compliant Technology 5-8 5.8.3 Designing an ETL-Style Mapping 5-9 6 XML Files 6.1 Introduction 6-1 6.1.1 Concepts 6-1 6.1.2 Pre/Post Processing Support for XML Driver 6-1 6.1.3 Knowledge Modules 6-2 6.2 Installation and Configuration 6-2 6.2.1 System Requirements 6-2 6.2.2 Technologic Specific Requirements 6-2 6.2.3 Connectivity Requirements 6-5 6.3 Setting up the Topology 6-5 6.3.1 Creating an XML Data Server 6-6 6.3.1.1 Creation of the Data Server 6-6 6.3.2 Creating a Physical Schema for XML 6-7 6.4 Setting Up an Integration Project 6-7 6.5 Creating and Reverse-Engineering a XML File 6-7 6.5.1 Create an XML Model 6-8 6.5.2 Reverse-Engineering an XML Model 6-8 6.6 Designing a Mapping 6-8 6.6.1 Notes about XML Mappings 6-8 6.6.1.1 Targeting an XML Structure 6-9 6.6.1.2 Synchronizing XML File and Schema 6-9 6.6.1.3 Handling Large XML Files 6-9 6.6.2 Loading Data from and to XML 6-10 6.6.2.1 Loading Data from an XML Schema 6-10 6.6.2.2 Loading Data to an XML Schema 6-10 6.6.3 Integrating Data in XML 6-11 6.7 Troubleshooting 6-11 6.7.1 Detect the Errors Coming from XML 6-11 6.7.2 Common Errors 6-12 7 Complex Files 7.1 Introduction 7-1 vi 7.1.1 Concepts 7-1 7.1.2 Pre/Post Processing Support for Complex File Driver 7-2 7.1.3 Knowledge Modules 7-2 7.2 Installation and Configuration 7-2 7.2.1 System Requirements 7-3 7.2.2 Technology Specific Requirements 7-3 7.2.3 Connectivity Requirements 7-3 7.3 Building a Native Schema Description File Using the Native Format Builder 7-3 7.4 Setting up the Topology 7-4