Deltek Connector Design Studio 2.0 Getting Started Guide

February 22, 2013

While Deltek has attempted to verify that the information in this document is accurate and complete, some typographical or technical errors may exist. The recipient of this document is solely responsible for all decisions relating to or use of the information provided herein. The information contained in this publication is effective as of the publication date below and is subject to change without notice. This publication contains proprietary information that is protected by copyright. All rights are reserved. No part of this document may be reproduced or transmitted in any form or by any means, electronic or mechanical, or translated into another language, without the prior written consent of Deltek, Inc. This edition published February 2013. © 2013 Deltek, Inc. Deltek’s software is also protected by copyright law and constitutes valuable confidential and proprietary information of Deltek, Inc. and its licensors. The Deltek software, and all related documentation, is provided for use only in accordance with the terms of the license agreement. Unauthorized reproduction or distribution of the program or any portion thereof could result in severe civil or criminal penalties. All trademarks are the property of their respective owners.

Getting Started Guide

Data Integrator Getting Started Designing and Running Integration Solutions While Deltek has attempted to verify that the information in this document is accurate and complete, some typographical or technical errors may exist. The recipient of this document is solely responsible for all decisions relating to or use of the information provided herein. The information contained in this publication is effective as of the publication date below and is subject to change without notice. This publication contains proprietary information that is protected by copyright. All rights are reserved. No part of this document may be reproduced or transmitted in any form or by any means, electronic or mechanical, or translated into another language, without the prior written consent of Deltek, Inc. This edition published February 2011. © 2011 Deltek, Inc. Deltek’s software is also protected by copyright law and constitutes valuable confidential and proprietary information of Deltek, Inc. and its licensors. The Deltek software, and all related documentation, is provided for use only in accordance with the terms of the license agreement. Unauthorized reproduction or distribution of the program or any portion thereof could result in severe civil or criminal penalties. All trademarks are the property of their respective owners.

Contents

About This Manual ...... v Who Should Read This Manual ...... vi New Users ...... vi Users of Previous Versions ...... vi Software Developers ...... vi

1 Getting Started with the Integration Products ...... 1-1 Integration Tool Set...... 1-2 Repository ...... 1-3 Source and Target Connections...... 1-3 Map Files ...... 1-3 Process Files ...... 1-3 Portability...... 1-3 File Extensions and Descriptions ...... 1-4 Documentation Overview ...... 1-7 Online Help Content ...... 1-7 Cosmos.ini Settings ...... 1-10 Streaming Data through Pipes ...... 1-14 What Do You Want to Do First?...... 1-17 More Information ...... 1-18

2 Installing on Windows ...... 2-1 How to Set Up and Install the Products on Windows Minimum System Requirements ...... 2-2 Before Installation on Windows ...... 2-3 Installation Tips ...... 2-4 Installation Instructions ...... 2-5 Data Integrator (32-bit Installation) ...... 2-5 Engine Only (64-bit Installation) ...... 2-6 Installation Locations...... 2-7 Managing License Files on Windows ...... 2-17 Launching License Manager...... 2-17 Obtaining a License ...... 2-17 Repository Manager and Document Schema Designer ...... 2-18 Choosing the Current License...... 2-18 Setting the Active License Directory ...... 2-18 Viewing Enabled Products and Features ...... 2-18 Upgrading from Previous Versions ...... 2-19 Upgrading with Additional Connectors or Components Installed...... 2-19 Upgrading from 9.x to 9.2.5...... 2-19 Upgrading from 8.x to 9.x ...... 2-20 Upgrading from 7.x to 8.x ...... 2-22

i Contents

Sharing Transformation Files ...... 2-25 Solutions for Installation and Startup Issues...... 2-26 Common Issues ...... 2-26 Interpreting Initialization Error Messages...... 2-28 Remove and Reinstall Instructions ...... 2-29

3 Installing on Unix...... 3-1 How to Set Up and Install the Products on Unix Minimum System Requirements on Unix ...... 3-2 Before Installation ...... 3-3 Permissions ...... 3-3 License File...... 3-3 See Checklist for Your Unix System ...... 3-3 Installation Utility ...... 3-4 Installation Notes for Unix ...... 3-5 Additional Notes on SDKs for Unix ...... 3-5 Multiuser License Machine Installation ...... 3-5 Managing License Files on Unix ...... 3-6 Verifying Unix Setup: The Acid Test ...... 3-9 Basic Setup Testing ...... 3-9 Advanced Setup Testing ...... 3-10 Reverting to Previous Unix Permissions ...... 3-12 Moving Files from Windows to Unix...... 3-13 ASCII Data Files ...... 3-13 Other ASCII Files ...... 3-13 Option Files ...... 3-13 Other Considerations ...... 3-13 Using File Transfer Protocol ...... 3-16 Using FTP ...... 3-16 Before You Run FTP on Unix ...... 3-16 Running FTP ...... 3-16

4 Installing on AIX ...... 4-1 How to Set Up and Install the Products on AIX Checklist Before Installing on AIX ...... 4-2 To Install on AIX from a CD ...... 4-3 To Install on AIX from a Downloaded Archive ...... 4-4 Steps After Installing on AIX ...... 4-5

5 Installing on HP-UX ...... 5-1 How to Set Up and Install the Products on HP-UX Checklist Before Installing on HP-UX ...... 5-2 To Install on HP-UX from a CD ...... 5-4 Installation Tips ...... 5-4 ii Contents

To Install on HP-UX from a Downloaded Archive ...... 5-6 Steps After Installing on HP-UX...... 5-7

6 Installing on Linux ...... 6-1 How to Set Up and Install the Products on Linux Checklist Before Installing on Linux...... 6-2 To Install on Linux from a CD...... 6-3 To Install on Linux from a Downloaded Archive ...... 6-4 Steps After Installing on Linux...... 6-5

7 Installing on Solaris ...... 7-1 How to Set Up and Install the Products on Solaris Checklist Before Installing on Solaris ...... 7-2 To Install on Solaris from a CD ...... 7-3 To Install on Solaris from a Downloaded Archive ...... 7-4 Steps After Installing on Solaris ...... 7-5

8 Repositories, Workspaces, and Macros ...... 8-1 Their Use in the Integration Environment Introducing Workspaces, Repositories, and Collections...... 8-2 What Is a Workspace? ...... 8-2 What Is a Repository? ...... 8-3 What is a Collection?...... 8-4 Using Workspace Manager...... 8-5 Workspace Manager Interface ...... 8-5 Managing Workspaces...... 8-5 Managing Repositories ...... 8-9 Macro Manager ...... 8-13 Where Can Macros Be Used? ...... 8-13 Where Should Macros Not Be Used? ...... 8-14 Creating, Editing and Deleting Macros...... 8-14 Using a Macro in Connect Information ...... 8-14 Using a Macro to Store a Password ...... 8-15

9 Help and Support...... 9-1 Online Documentation and Technical Support Using the Online Product Documentation ...... 9-2 Getting Help from Technical Support Engineering ...... 9-3

10 Solution Development Overview ...... 10-1 Building Integration Solutions Introduction to Integration Software Development ...... 10-2 Prerequisites ...... 10-2

iii Contents

Engine Overview ...... 10-3 Why Use an Engine? ...... 10-3 How Does the Engine Work with the Designers? ...... 10-3 Running the Engine ...... 10-4 Command Line Execution...... 10-5 Command Line Syntax ...... 10-5 Introduction to the Development Toolset ...... 10-8 Engine Engine SDK ...... 10-8 Content Extraction Language (CXL) SDK ...... 10-8 Other Tools ...... 10-9 Resources Needed to Construct a Solution ...... 10-10 Requirements ...... 10-10 The Solution ...... 10-10 The Integration Platform Solution...... 10-11 Integration Platform Programming Resources ...... 10-12 Application Development Basic Steps ...... 10-13 Creating a Java Application...... 10-13 Creating a COM Application...... 10-13 The Next Step ...... 10-14

A Solution Development Reference ...... A-1 More Information on Developing Solutions Debugging Processes and Troubleshooting Problems ...... A-2 Testing for Errors ...... A-2 Solution Development Checklist ...... A-3 Product and Feature Limitations ...... A-4 Map Designer ...... A-4 Process Designer...... A-5 Repository Explorer ...... A-7 Workspace Manager ...... A-7 Structured Schema Designer ...... A-7 RIFL ...... A-8

iv About This Manual

This manual gets you started with the integration products and includes important installation information. This guide includes the following:

Lists documentation manuals Introduces the design- and run-time products Details installation and setup for Windows and Unix platforms Explains the basic use of repositories, workspaces, and macros Introduces the integration platform development environment The tutorials are a good starting point before you start working on your own projects. See Tutorials Reference. To find out which product you should start using first, see “What Do You Want to Do First?”

v Who Should Read This Manual

New Users This manual provides a reference for administrators and users who are installing and running the integration products for the first time.

Users of This documentation is useful for individuals who have experience Previous with previous versions of the integration products and want to Versions become familiar with the current version.

Software IT professionals who will support the integration products or Developers develop applications to interact with them will find this a good starting point for understanding the integration platform.

vi chapter Getting Started with the Integration Products 1

Introduction to the Integration Products

The following topics are designed to help familiarize you with the integration products and documentation:

“Integration Tool Set” on page 1-2 “File Extensions and Descriptions” on page 1-4 “Documentation Overview” on page 1-7 “Cosmos.ini Settings” on page 1-10 “What Do You Want to Do First?” on page 1-17

1-1 Getting Started with the Integration Products

Integration Tool Set The integration platform is a set of tools that perform mapping, transformation, process, and file management tasks.

Designers connect and manipulate data from various sources, mapping and transforming them to target data types. This includes Map Designer for mapping and transforming data, schema designers to define data structures, and Process Designer to group related tasks into automated data integration processes. Repository Explorer manages workspaces, repositories, collections, and files that are used by the other tools. The Engine runs the transformations and processes from the designers to generate results.

1-2 Integration Tool Set

The integration platform offers a design- and a run-time environment for the creation of data integration transformations and processes. These data integration designs are made up of source and target connectors, structured schemas, and maps that exist as reusable, portable elements. Each transformation also includes event actions, event handlers, and error-management settings.

Repository A repository stores transformations, processes, schemas, and other information used for data integration. See Chapter 8, “Repositories, Workspaces, and Macros” for more information on working with repositories.

Source and The repository stores connection information in reusable source Target (.sc.xml) and target (.tc.xml) files. In a simple example, if your Connections source files in ASCII-delimited format always have a header row, in Map Designer set the Header source property to True, save the .sc.xml file, and then reuse it for another ASCII-delimited source file.

Map Files Map Designer creates map files with the extension .map.xml. Event handlers, expressions, error handling, record validation, and other features of a transformation are saved in the map file. Each map file can also refer to particular .sc.xml and .tc.xml files.

Process Files The repository stores integration processes as .ip.xml files. Processes automate data integration jobs by defining and linking steps for aggregating, iterating, and queuing data, invoking programs, running transformation map files,

Portability A completed transformation design can be moved to run-time production in either a Windows or Unix environment. Absolute paths, database and table names, user names, and passwords can all be stored in the transformation.

1-3 Getting Started with the Integration Products

File Extensions and Descriptions The following table lists files used by the integration platform and describes the relationship of each file to its function. The product name generally refers to the application that creates the file. Some of the files are used by more than one application.

Table 1-1 File Extensions and Their Descriptions

File Extension, File Name, and Name Function – What This File Contains or Does

File Extension: .tf.xml Names of the .map.xml, .sc.xml, .tc.xml, and to use in a File Name: Transformation transformation

Application Name: Map Designer

File Extension: .map.xml Mapping details (the options you configured to extract, transform, File Name: Map and load)

Application Name: Map Designer

File Extension: .sc.xml Source connector and properties

File Name: Source connection

Application Name: Map Designer

File Extension: .tc.xml Target connector and properties

File Name: Target connection

Application Name: Map Designer

File Extension: .ip.xml Process file; holds all process metadata File Name: Integration process

Application Name: Process Designer

File Extension: .cxl Record rules, definitions, all design configuration and properties File Name: Extract script

Application Name: Extract Schema Designer

1-4 File Extensions and Descriptions

Table 1-1 File Extensions and Their Descriptions

File Extension, File Name, and Name Function – What This File Contains or Does

File Extension: .mdb Repository for Extract Schema Designer script metadata File Name: Microsoft Access

Application Name: Extract Schema Designer

File Extension: .xmldb Repository where the XML metadata associated with File Name: XML database structured schemas, maps, and processes is stored Application Name: Map Designer

File Extension: .ds.xml Document schemas; user-modified schemas File Name: Document schema

Application Name: Document Schema Designer

File Extension: .ss.xml Record-specific field order, definition, configuration File Name: Structured schema

Application Name: Map Designer

File Extension: .slc Holds digitally encrypted license information for installation of File Name: License integration tools and components

Application Name: (all products)

File Extension: workspace.xml Root workspace location and one entry for every workspace off of this File Name: Workspace root

File Extension: specification_file ASCII file of global options such as command line parameters, File Name: @specification_file extending command line, offering portability, and re-use Application Name: Integration Engine

File Extension: .rifl Reusable, user-defined, portable functions File Name: Code module (also “RIFL file”)

Application Name: (user-defined)

1-5 Getting Started with the Integration Products

Table 1-1 File Extensions and Their Descriptions

File Extension, File Name, and Name Function – What This File Contains or Does

File Extension: .join.xml See the Join Designer User’s Guide File Name: Join

Application Name: Join Designer

File Extension: .djar Package with all files needed for transformation or process File Name: .djar

Application Name: Integration Engine

File Extension: .tpl Text file containing expressions saved as “expression templates” File Name: Template File

Application Name: Map Designer

File Extension: .tar Unix compression format (“tape archive” file) File Name: Tar

Application Name: Unix

File Extension: .jar Java compression format

File Name: Jar

Application Name: Java; Document Schema Designer

File Extension: .pdf Adobe format for printable documentation File Name: Portable Document Format

Application Name: Adobe Reader

File Extension: .chm Online help documentation

File Name: Compiled HTML

Application Name: Online Help

1-6 Documentation Overview

Documentation Overview The online product documentation contains information on all integration products, including the manuals listed in the following table. See DataIntegrator_readme.htm for important release information. This file is available in InstallDir\Common, where InstallDir is the installation directory for the integration platform. For information about user and developer documentation, please see “Using the Online Product Documentation” on page 9-2. PDF files are available for printing in the InstallDir\Common\Help\PDF folder.

Note The compressed HTML (CHM) format is heavily linked with hypertext. In the PDF files, certain links are not active, such as those between books, because these files are designed only for printing.

Online Help The following table lists the documentation set that appears in the Content online documentation

Table 1-2 List of Product Documentation

Document Description

Getting Started Guide The guide you are currently reading. Covers system requirements, setup, installation, an introduction to the integration components, and a section for integration solution developers.

Repository Explorer User’s Launch any of the designers from this application, or open any map, process, or Guide schema files in your repository list. Also enables you to access files under version control in CVS or Microsoft Visual SourceSafe.

Integration Engine User’s Describes the command line interface for executing Map Designer (.tf.xml) or Guide Process Designer (.ip.xml) specification files with various options.

Integration Server and SDK Describes the SDK that includes client implementations of Message Driven Guide Beans, web services, and the Java Connector Architecture Resource Adapter.

Repository Manager User’s Helps manage files in multiple repositories in one or more workspaces. You can Guide access design documents, view their contents, make simple revisions, bundle them in a package, and generate reports.

1-7 Getting Started with the Integration Products

Table 1-2 List of Product Documentation

Document Description

Map Designer User’s Guide Introduces the use of Map Designer to design the transformation of data from source to target. Covers basic concepts, common transformations, and simple examples of filtering, sorting, and trimming fields.

Intermediate and Advanced Continues mapping concepts and procedures beyond the basic level. Mapping Guide

Easy Loader User’s Guide Presents a one-to-one record flat file mapper that creates data load files. Target connectors, schemas, and validation logic are predefined to speed up the integration process. Mapping wizards, fuzzy matching, and automatic event handling logic further simplify mapping tasks.

Process Designer User’s Describes the visual tool that defines integration steps as a flowchart so that you Guide can run an entire workflow sequence as a single unit. Process designs can range in complexity from simple integration flows to parallel processing of large data loads or pushing messages back and forth across message queues.

Structured Schema Designer Describes the use of this tool to set up the data structure of your source or target User’s Guide file. This includes arranging field order, assigning field names, and specifying the data type and size of each field. The resulting metadata is stored as XML-based files that include schema, record recognition rules, and record validation rule information.

Document Schema Designer Documents a Java-based tool that allows you to build templates for e-document User’s Guide files. You can customize schema subsets for specific EDI Trading Partner and TranType scenarios. In addition to EDI and EDIFACT e-doc types, This designer is useful to those working with HL7, HIPAA, and SAP (IDoc) data files.

Extract Schema Designer Offers an alternative to manual coding of scripts for integrating data sources such User’s Guide as e-mail, report data, HTML, print data, or any other raw text. Extract desired data fields from various lines in a text file and assemble those fields into a flat record of data. Once the extract file is saved, it can be used as a source file in Map Designer.

Join Designer User’s Guide Describes how you can join two or more single-record data sources prior to running a Map Designer transformation on them. These sources do not have to be of the same type.

Data Browser User’s Guide Documents a standalone utility that allows you to take a peek at virtually any application file or URL in record and field format. If your file has multiple record types, you can choose to view all the record types or just view specific records.

Engine Profiler User’s Guide Provides graphical views of how much processor time (in milliseconds) each event or function call uses in your transformation and process designs. It provides performance details on each event, displays function and code modules in histogram, and calls graph formats.

Extract Schema Transfer Allows you to update existing extract scripts from previous versions of Content Utility User’s Guide Extractor to the current version of Extract Schema Designer.

1-8 Documentation Overview

Table 1-2 List of Product Documentation

Document Description

Rapid Integration Flow Documents a scripting language that includes functions, operators, statements, Language (RIFL) variables, object variables, and error objects. Programmer’s Reference Manual

Source and Target The master reference for source and target connectors available in the Map Connectors User’s Guide Designer user interface.

Integration Components Serves as a starting point for working with components. This manual lists Getting Started Guide component documentation, introduces the design time and run time products, and provides the location of the tutorials and example files. See this guide for a list of available components and related documentation.

Tutorials Reference Includes tutorials designed for Map Designer, Structured Schema Designer, and Extract Schema Designer. The tutorials are most useful to users who are new to the designers.

Samples Handbook Describes what was done to produce the sample files provided with the integration platform package. By reading about these samples and working with the files, you will see examples of how to perform numerous transformations and processes.

The handbook and sample files are available for download from the Pervasive Software web site.

Glossary Defines integration terms relating to the integration products. Most useful to beginning users, but intermediate and advanced users may find it informative.

Error Codes and Messages Provides documentation about error codes and messages for all integration Reference Manual components.

Content Extraction Documents a programming language for recognizing and extracting structured Language (CXL) fields of data from text files and assembling them into flat records for a following Programmer’s Reference application, such as Map Designer. Manual

Easy Loader Developer’s Explains how the developer or integration platform administrator defines Guide customized targets for use with Easy Loader.

Software Development Kit Shows how to build custom applications using the engine APIs to override Developer’s Guide transformations and processes and customize other settings at run time.

Message Component Documents a Java development environment in which you can write your own Framework Programmer’s components that can be accessed in Process Designer. It includes a family of Reference Manual components that share the same support classes that can be bundled and deployed together as a single package.

1-9 Getting Started with the Integration Products

Cosmos.ini Settings The cosmos.ini file contains the startup information needed to launch the integration products. For the location of this file, see “Installation Locations” on page 2-7. The cosmos.ini file has several sections:

[UserInfo] – Information pertaining to users and their system. Some entries can be customized or added to this section. [ProductInfo] – Product information that must not be changed. [Preferences] – User preference information, including optional customization. [Internal] – Profiling preferences that can be customized. [NamedPipeSchemeWriteHandlers] – Customizable settings for named pipes. [AnonymousPipeSchemeWriteHandlers] – Customizable settings for anonymous pipes.

[UserInfo] You can edit some of the information in this section.

LicenseFile – Full path to license file. This setting is required. If this entry is not set, the GUI products prompt you to locate the file so that its path can be written to cosmos.ini. Version – Version of the integration platform installed on your system. This information is also in the license file and should not be changed. InstallPath – Path to the integration platform installation. If set, the core engine code uses InstallPath as a default Java class path if the optional ClassPath entry is not set. The GUI uses InstallPath for several functions, including locating directories off of the root. Do not enclose the path name in quotation marks. LibraryPath – Path to library files. This option is used to find configuration files, including .djx files (such as for configuring metadata browsers and validators) and encoding DLLs. The LibraryPath option must be set and generally has the same value as InstallPath. Do not enclose the path name in quotation marks.

1-10 Cosmos.ini Settings

NetworkTimeout – Gives the number of seconds the integration products will wait for various network operations to finish, such as establishing connections and receiving packets. The default value is 5 seconds.

Optional [UserInfo] Values

ClassPath – Points the Java Virtual Machine to the location of user-defined packages and classes, down to the file level. Set the class path if you are using any user-defined Java classes or packages which are not located in the Data Integrator engine working directory or, if the Java objects are Process Designer components, not located in the Plug-ins directory. Do not enclose the path name in quotation marks. JarClassPath – Points to directories that contain Java .jar files. If set, this value is treated as a list of directories separated by the standard class path separator – semicolon for Windows and colon for Unix. Do not enclose the path name in quotation marks. JavaVirtualMachine – If set, JavaVirtualMachine points to the jvm.dll file for the Java virtual machine to be run. If not set, the integration platform will attempt to load the default for the system. Do not enclose the path name in quotation marks. JvmArgs – A set of command line arguments for an instance of a Java Virtual Machine. One use of this property is to define the memory allocated to a JVM when starting integration platform applications. In addition to the JvmArgs property, you can define separate JVM memory allocations for instances of selected applications. These settings allow more precise memory management for JVM instances competing for system resources. The following tool-specific options can be used:

MapDesignerJvmArgs ProcessDesignerJvmArgs JoinDesignerJvmArgs StructuredSchemaDesignerJvmArgs RepositoryExplorerJvmArgs In the following example, all tools are allocated 512 MB of memory except Map Designer, which is allocated 128 MB.

1-11 Getting Started with the Integration Products

JvmArgs=-Xmx512m MapDesignerJvmArgs=-Xmx128m

Note Setting the memory allocation for an individual application to higher than 128 MB is not recommended.

LookupIndexDir – If set, points to a directory used for storing lookup file indexes. If not set, the default is the lookup subdirectory in the above LibraryPath entry. NamePartAdd or NamePartReplace – When using the NamePart function, you may need to add or replace existing entities. For instructions on adding or replacing entities, see “NamePart Function Customization” in RIFL Programmer’s Reference Manual. For a complete list of entities, see “NamePart Function” in RIFL Programmer’s Reference Manual.

OptimizationLevel – Used by RIFL expression processing RIFL to determine how much to optimize expression compilation. This setting is a bit mask. The 0x01 bit turns optimizations on and off. The 0x02 bit controls bytecode optimizations. The default value is 3 (both bits on), with optimizations and byte code optimizations turned on. OptionExplicit – A flag to indicate whether the RIFL compiler should consider option explicit the default. If set to 0, then option implicit is the default. ShellAllowed – Controls whether execution of external shell commands is allowed. If this entry is absent, then execution is allowed. If set to no value or a value of true, t, 1, or yes, then execution is allowed. If set to any other value, execution is not allowed. The engine checks for a value when an attempt is made to execute an external command. If execution is not allowed, a shell function causes a type mismatch run-time exception, and an EXE step in a process fails. In both cases, the log message is "external program execution disabled."

1-12 Cosmos.ini Settings

V7compat – A special flag to indicate whether the integration platform uses the older EBCDIC code pages or the newer, corrected versions. The corrected versions are used by default. If V7compat is set to 1, then the integration platform uses the old code pages. If V7compat is not set or uses another value, the integration platform uses the new code pages.

[ProductInfo] Do not change these settings.

CompanyName – Owner of the installed product. ProdTitleShort – Short name for the installed product. ProductAcronym – Acronym for the installed product. PackageName – Installed package, usually Data Integrator.

[Preferences] You can customize these settings for your environment.

DefaultFontName – Default font set to MS San Serif. DefaultFontSize – Default set to 8. LocaleID – Default set to 0. MaxConcurrentThreads – Default number of threads set to 32. SingleRecordSchemaCheck – Used with delimited ASCII and Unicode source connectors when a map defines only one record type. Default of 0 means the engine does not check the source file against the schema at run time. Set to 1 to check the schema and follow schema mismatch settings if needed.

[Internal] You can edit the profiling preferences in this section.

Debug – Sends more detail to the error log about steps executed by the engine. Default is -1, or True. Set to 0 to turn off. HTTPPutMethod – By default, Map Designer uses a POST to a URL. To verify or change these settings, go to the [Internal] section of your cosmos.ini file. For more details, see “Messaging with HTTP POST” in Intermediate and Advanced Mapping Guide.

1-13 Getting Started with the Integration Products

HTTPMimeType – Specifies any HTTP MIME type you wish. The default is xml. For more details, see “Messaging with HTTP POST” in Intermediate and Advanced Mapping Guide. Profile – Turns RIFL profiling on and off. Default is off with a value of 0.

Optional [Internal] Values

ProfileOutput – If set, this option names the file to which the profile information is written. Default is djprof.out.

Streaming Data Data Integrator supports pipes for streaming data through named or through Pipes anonymous pipes. For more information, see the Streaming Data through Pipes topic in Source and Target Connectors User Guide. You can set the pipe scheme sections in the cosmos.ini file as described under the following topics:

[NamedPipeSchemeWriteHandlers] [AnonymousPipeSchemeWriteHandlers] Currently only write handlers are supported.

[NamedPipeSchemeWriteHandlers] The format for pipe handler schemes in the cosmos.ini file is scheme=. For example, a named pipe handler called npwritetest would be specified in this NamedPipeSchemeWriteHandlers section like so:

npwritetest=pipetest -r n %pipe The pipe handler scheme would appear in the map as Target File/URI npwritetest://. The command line run would be:

pipetest -r -n and the actual file system name of the named pipe would appear in place of . You can provide various parameters in the command line. The substitution is made when the command is executed. The %pipe is replaced at run time with the actual name of the pipe. When parts of a URI are mentioned, they refer to the URI entered in the Target tab as the file name. If a needed part of the URI is missing, then the substitution is an empty string.

1-14 Cosmos.ini Settings

Note Read handlers for pipes are not currently supported.

The following table gives URI parameters for pipe handlers and their replacement values.

URI Parameter Replacement Value

%uri Entire URI provided.

%puri Entire URI provided, escaped using the standard URI escaping mechanism (i.e., a space appears as %20).

%scheme Scheme part of the URI (i.e., the URI handler’s name).

%auth Entire authority part of the URI (username, password, server, and port).

%host Host or server part of the URI.

%user User name part of the URI.

%passwd Password part of the URI

%port Port part of the URI.

%path Path part of the URI.

%query Query part of the URI.

%pipe Pipe name, generated when the pipe is created at run time.

%% A single % character.

Note Do not use the same scheme name for both named and anonymous pipe handlers. The named pipe handlers are searched first, and if the scheme is found there, then the anonymous handlers section is not used.

[AnonymousPipeSchemeWriteHandlers] Read the [NamedPipeSchemeWriteHandlers] topic before you read this topic. In this cosmos.ini section, enter the names of anonymous pipe handlers. For example:

1-15 Getting Started with the Integration Products

apwritetest=pipetest -r specifies an anonymous pipe handler called apwritetest, which can then be entered in the map Target File/URI as apwritetest://. For this pipe handler, the command line run is:

pipetest -r See the table of parameters and replacement values in the NamedPipeSchemeWriteHandlers topic. Other than the pipe name (%pipe), all parameters work for both named and anonymous pipes.

1-16 What Do You Want to Do First?

What Do You Want to Do First? Before you start using any of the integration products, it may be helpful to evaluate the kind of work you will be doing and how best to use this documentation. Especially consider the scope of your data integration requirements. Use the following table to help you choose which sections of the documentation to read.

Table 1-3 What Do You Want to Do?

If you want to... Go to...

Install the integration platform products “Installing on Windows” on page 1-1 or “Installing on Unix” on page 3-1

As a new user, learn how to use the integration Tutorials Reference to do the exercises provided products

Understand what is different from a previous “New Features and Enhancements” in the version of the integration products InstallDir\Common\DataIntegrator_readme.htm

Transform a specific type of source data Source and Target Connectors User’s Guide

Understand the basics of the data “Introducing Structured Schema Designer” on page 1-2 and transformation process and structured “Map Designer Tutorial 1 - The Basics” on page 1-2 in Tutorials schemas in general Reference

• Find the source type of a flat file such as “Defining Source Record Length” on page 5-25 and “Using the fixed ASCII, COBOL or Binary Data Parser” on page 5-2 in Structured Schema Designer • Determine if the source type is a record User’s Guide manager such as Btrieve, C-tree, Also see “Map Designer Tutorial 2 - Applying a Structured MicroFocus COBOL or Acucobol Schema to a Fixed ASCII File” on page 1-6 and “Structured • Parse source data Schema Designer Tutorial 4 - Using Data Parser to Build a • Determine record length, field separators, Schema” on page 1-15 in Tutorials Reference and delimiters

Change the field order, field names, field sizes, “Defining Target Field Properties” on page 4-4 and “Mapping data types, or data properties in the target file Data” on page 6-2 in Intermediate and Advanced Mapping Guide

Change the relationships between source and “Commonly Used Functions” on page -42 in RIFL target fields (merge fields, parse names or Programmer’s Reference Manual addresses, and so forth)

Include only a subset of source records in the “Setting Transformation Filters” on page 8-2 in Intermediate and target file by filtering source records that meet a Advanced Mapping Guide and “Writing Expressions” on page certain condition or fall in a certain range -2 and “Commonly Used Functions” on page -42 in RIFL Programmer’s Reference Manual

1-17 Getting Started with the Integration Products

Table 1-3 What Do You Want to Do?

If you want to... Go to...

Use a single command to run multiple Process Designer User’s Guide to learn how to include transformations in order. transformation steps in a process design

Run processes and transformations from a “Command Line Syntax” on page 5-2 in Integration Engine command line. User’s Guide

Join two or more source types. Join Designer User’s Guide

Look at sample transformations and processes Download the Samples Handbook and sample files from the to tailor them for specific needs. Pervasive Software web site.

More The Glossary gives definitions of terms used in this documentation. Information

1-18 chapter Installing on Windows 2

How to Set Up and Install the Products on Windows

This chapter includes the following sections:

“Minimum System Requirements” on page 2-2 “Before Installation on Windows” on page 2-3 “Installation Tips” on page 2-4 “Installation Instructions” on page 2-5 “Installation Locations” on page 2-7 “Managing License Files on Windows” on page 2-17 “Upgrading from Previous Versions” on page 2-19 “Sharing Transformation Files” on page 2-25 “Solutions for Installation and Startup Issues” on page 2-26 “Remove and Reinstall Instructions” on page 2-29

2-1 Installing on Windows

Minimum System Requirements Before beginning installation, verify that your computer meets the system requirements. The following resources are the minimum recommended for a Windows operating system:

An IBM PC or compatible PC with 1 GHz or faster processor Microsoft Windows XP, Server 2008 R2, or 7 1.25 GB free hard drive space in the installation location, depending on the installation options selected RAM 32-bit installation: 1 GB 64-bit installation: 4 GB CD-ROM drive unless installing from downloaded files These system requirements are the minimum for satisfactory operation. For higher performance, you may need more physical memory to support your operating system and its applications. When you open a process or transformation, all components are loaded into memory. A large, complex process may consume so much memory that it cannot execute. Where business processes permit, redesigning the process may resolve this issue. Execute large, complex processes only on machines that have sufficient memory to load all components of the process while supporting the operating system and all other open applications.

Note Windows Vista/2008 SP2/2003/2000/95/98/ME/NT are not supported in this release. No technical support is provided for these operating systems.

See Also The release notes provide additional information on supported environments.

2-2 Before Installation on Windows

Before Installation on Windows Read and consider the following precautions before beginning the installation process.

Installing Multiple Versions on the Same Computer Data Integrator is compatible and will run with versions of the integration platform before 9.x, such as Integration Architect (8.x). Its current installation location is different from these earlier releases. See also “Upgrading from Previous Versions” on page 2-19.

Windows – Logging On as Local Administrator When installing on a Windows system, you may be required to log on as local administrator for the installation to succeed.

Closing Down All Applications Exit all programs before running the setup. This includes antivirus software, browsers, screen savers, and all applications and processes that run in background. To exit all active programs: 1 Press Ctrl+Alt+Delete, select Task Manager, and click the Applications tab. 2 Select an application name in the list and click End Task. 3 Repeat until all applications are closed. 4 Restart the computer after closing all applications. After restarting, verify that all applications are still closed.

2-3 Installing on Windows

Installation Tips

To upgrade to a 9.x release, see “Upgrading from Previous Versions” on page 2-19. To install version 9.x side-by-side on the same machine with version 8.x, see the upgrade scenarios under “Upgrading from 8.x to 9.x” on page 2-20. For problems running the integration applications after installing, go to “Solutions for Installation and Startup Issues” on page 2-26. To remove or reinstall the integration platform, see “Remove and Reinstall Instructions” on page 2-29.

2-4 Installation Instructions

Installation Instructions After satisfying all the prerequisites described in “Before Installation on Windows,” you are ready to begin the installation steps. In order to perform the installation, you must have administrator rights on the computer where the program will be installed. If you do not currently have those rights, contact your network administrator for assistance.

Data Integrator ³ To install Data Integrator (32-bit 1 Log on to the Windows machine where you wish to install. Installation) 2 Do one of the following:

Design Studio (default) – To install the designers, browsers, utilities, and engine, accept this setting. Integration Engine – To install only the engine and its utilities, select this option. Note that your memory and storage requirements may be somewhat less than for a full installation of Design Studio. 4 At the end of the installation, click Finish. For a Design Studio installation, a new group of program shortcuts appear under the Start menu. 5 Verify successful setup of Design Studio or Integration Engine by opening a prompt in the directory InstallDir\Cosmos9\Common and running the following command to display version and licensing information:

djengine -V

2-5 Installing on Windows

Note Installation generates detailed log files in the directory InstallDir\Cosmos9\InstallLogs. Log files under 15K generally indicate a problem with the installation.

Engine Only You can install the 64-bit engine stand-alone on a server or (64-bit side-by-side with 9.x Data Integrator installations. For side-by-side Installation) installations, use your existing license file to enable the engine.

³ To install the 64-bit engine 1 Log on to the Windows machine where you wish to install. 2 Do one of the following:

If you are installing from a CD, insert the installation CD. If it does not automatically play, then select Start > Run, enter the path name to DataIntegrator9.exe in the CD root directory, and click OK. If you downloaded the installation files from the web site, navigate to the installation files and double-click DataIntegrator9.exe. 3 Follow the instructions in the installation wizard. 4 At the end of the installation, click Finish. The installation creates detailed log files for each major portion of the install, such as connectors and documentation. The log files are located in the InstallDir\Cosmos9 (64-bit)\InstallLogs directory. Log files under 15 KB generally indicate a problem with the installation. 5 Verify successful setup of Integration Engine by opening a prompt in the directory InstallDir\Cosmos9 (64-bit)\Common and running the following command to display version and licensing information:

djengine -V

Note The COM interface supports only the 32-bit edition of Data Integrator.

2-6 Installation Locations

Installation Locations The following tables show default locations of key files in version 9.x compared to their former locations in version 8.x.

Common Folder Cosmos.ini Cosmos9_Work Documentation Extractor900.mdb License Folder Logs Plug-Ins Release Notes Samples Settings Folder Embedded or rebranded installations may use custom locations. Some platforms and configurations described here may not apply to your particular use. Note that 8.x installations are not supported on 64-bit architectures.

Note If you accept the default installation directory, cosmos.ini, plug-ins, and connectors are installed as described in these topics. If you specify an alternate directory, these files are installed under C:\Documents and Settings\All Users\Application Data\InstallDir\Common\ on Windows XP or C:\ProgramData\InstallDir\Cosmos9\Common\ on Server 2008 and Windows 7.

Common Folder In 8.x, the Common800 folder held both application- and user-specific files. In 9.x, files are divided between a Common folder for the application and an individual Common folder for each user. Common800 is the default folder for target file output in 8.x. For 9.x on XP, it is C:\Documents and Settings\username. The output folder on Server 2008 and Windows 7 is C:\Users\username.

2-7 Installing on Windows

32-bit Architecture

Common Folder Location

XP Application-specific: C:\Program Files\Pervasive\Cosmos9\

User-specific: C:\Documents and Settings\username\InstallDir\Cosmos9\

Server 2008 Application-specific: C:\Program Files\Pervasive\Cosmos9\ Windows 7 User-specific: C:\Users\username\AppData\Roaming\Pervasive\Cosmos9\

8.14 (Common800) C:\Program Files\Pervasive\Cosmos\

64-bit Architecture

Common Folder Location

XP 32-bit installation

Application-specific: C:\Program Files (x86)\Pervasive\Cosmos9\

User-specific: C:\Documents and Settings\username\Application Data\Pervasive\Cosmos9\

64-bit installation

Application-specific: C:\Program Files\Pervasive\Cosmos9 (64-bit)\

User-specific: N\A

Server 2008 32-bit installation Windows 7 Application-specific: C:\Program Files (x86)\Pervasive\Cosmos9\

User-specific: C:\Users\username\AppData\Roaming\Pervasive\Cosmos9\

64-bit installation

Application-specific: C:\Program Files\Pervasive\Cosmos9 (64-bit)\

User-specific: N\A

Cosmos.ini

32-bit Architecture

Cosmos.ini Location

XP C:\Documents and Settings\All Users\Application Data\Pervasive\Cosmos9\Common\

2-8 Installation Locations

32-bit Architecture

Cosmos.ini Location

Server 2008 C:\ProgramData\Pervasive\Cosmos9\Common\ Windows 7

8.14 (dj800.ini) C:\Program Files\Pervasive\Cosmos\Common800\

64-bit Architecture

Cosmos.ini Location

XP 32-bit installation

C:\Documents and Settings\All Users\Application Data\Pervasive\Cosmos9\Common\

64-bit installation

C:\Documents and Settings\All Users\Application Data\Pervasive\Cosmos9 (64-bit)\Common\

Server 2008 32-bit installation Windows 7 C:\ProgramData\Pervasive\Cosmos9\Common\

64-bit installation

C:\ProgramData\Pervasive\Cosmos9 (64-bit)\Common\

Cosmos9_Work The default location for data integration workspaces. Users may choose another location.

32-bit and 64-bit Architectures

Cosmos9_Work Location

XP C:\Documents and Settings\username\

Server 2008 C:\Users\username\ Windows 7

8.14 (Cosmos_Work) C:\

Documentation The design-time graphical user interfaces use compiled HTML help (CHM). The Help directory also has a PDF folder of printable document files. Engine-only installations do not include CHM files.

2-9 Installing on Windows

32-bit Architecture

Documentation Location

XP C:\Program Files\Pervasive\Cosmos9\Common\Help\

Server 2008 C:\Program Files\Pervasive\Cosmos9\Common\Help\ Windows 7

8.14 C:\Program Files\Pervasive\Cosmos\Common800\Help\

64-bit Architecture

Documentation Location

XP 32-bit installation

C:\Program Files (x86)\Pervasive\Cosmos9\Common\Help\

64-bit installation

C:\Program Files\Pervasive\Cosmos9 (64-bit)\Common\Help\

Server 2008 32-bit installation Windows 7 C:\Program Files (x86)\Pervasive\Cosmos9\Common\Help\

64-bit installation

C:\Program Files\Pervasive\Cosmos9 (64-bit)\Common\Help\

Extractor900.mdb As of version 9.0, when a user first opens the Microsoft Access .mdb file that stores extract scripts, it is automatically copied from its original location to a user-specific location. Applications such as Extract Schema Transfer Utility then use that personal copy for that user.

32-bit Architecture

Extractor900.mdb Location

XP Original: C:\Program Files\Pervasive\Cosmos9\Common\

User-specific: C:\Documents and Settings\username\Application Data\Pervasive\Cosmos9\Common\

2-10 Installation Locations

32-bit Architecture

Extractor900.mdb Location

Server 2008 Original: C:\Program Files\Pervasive\Cosmos9\Common\ Windows 7 User-specific: C:\Users\username\AppData\Roaming\Pervasive\Cosmos9\Common\

8.14 (extractor800) C:\Program Files\Pervasive\Cosmos\Common800\

64-bit Architecture

Extractor900.mdb Location

XP 32-bit installation

Original: C:\Program Files (x86)\Pervasive\Cosmos9\Common\

User-specific: C:\Documents and Settings\username\Application Data\Pervasive\Cosmos9\Common\

64-bit installation

N/A

Server 2008 32-bit installation Windows 7 Original: C:\Program Files (x86)\Pervasive\Cosmos9\Common\ (original)

User-specific: C:\Users\username\AppData\Roaming\Pervasive\Cosmos9\Common\

64-bit installation

N/A

License Folder License Manager offers the following paths as the default location for the License folder used as the Active License Directory. Users may choose another location.

32-bit Architecture

License Folder Location

XP C:\Documents and Settings\All Users\Application Data\Pervasive\Cosmos9\

Server 2008 C:\ProgramData\Pervasive\Cosmos9\ Windows 7

8.14 C:\Program Files\Pervasive\Cosmos\Common800\

2-11 Installing on Windows

64-bit Architecture

License Folder Location

XP 32-bit installation

C:\Documents and Settings\All Users\Application Data\Pervasive\Cosmos9\Common\

64-bit installation

C:\Documents and Settings\All Users\Application Data\Pervasive\Cosmos9 (64-bit)\Common\

Server 2008 32-bit installation Windows 7 C:\ProgramData\Pervasive\Cosmos9\Common\

64-bit installation

C:\ProgramData\Pervasive\Cosmos9 (64-bit)\Common\

Logs The first time that a user starts an integration application that logs run-time information, the application creates a folder for log files. The folder is specific to that user and is named after the application, such as \Logs\MapDesigner.

32-bit Architecture

Log Files Location

XP C:\Documents and Settings\username\Pervasive\Logs\

Server 2008 C:\Users\username\Pervasive\Logs\ Windows 7

8.14 C:\Program Files\Pervasive\Cosmos\Common800\

2-12 Installation Locations

64-bit Architecture

Log Files Location

XP 32-bit installation

C:\Documents and Settings\username\Pervasive\Logs\

64-bit installation

N/A

Server 2008 32-bit installation Windows 7 C:\Users\username\Pervasive\Logs\

64-bit installation

N/A

Plug-Ins Copy a new component to these locations so that it can be unpacked and installed by Process Designer.

32-bit Architecture

Plug-Ins Location

XP C:\Documents and Settings\All Users\Application Data\Pervasive\Cosmos9\Common\Plug-Ins\

Server 2008 C:\ProgramData\Pervasive\Cosmos9\Common\Plug-Ins\ Windows 7

8.14 C:\Program Files\Pervasive\Cosmos\Common800\

2-13 Installing on Windows

64-bit Architecture

Plug-Ins Location

XP 32-bit installation

C:\Documents and Settings\All Users\Application Data\Pervasive\Cosmos9\Common\Plug-Ins\

64-bit installation

C:\Documents and Settings\All Users\Application Data\Pervasive\Cosmos9 (64-bit)\Common\Plug-Ins\

Server 2008 32-bit installation Windows 7 C:\ProgramData\Pervasive\Cosmos9\Common\Plug-Ins\

64-bit installation

C:\ProgramData\Pervasive\Cosmos9 (64-bit)\Common\Plug-Ins\

Samples Sample files are available for download from the Pervasive Software samples home page.

Release Notes The release notes are found in the file DataIntegrator_readme.htm. Embedded or rebranded releases may rename or customize this file.

32-bit Architecture

Release Notes Location

XP C:\Program Files\Pervasive\Cosmos9\Common\

Server 2008 C:\Program Files\Pervasive\Cosmos9\Common\ Windows 7

8.14 C:\Program Files\Pervasive\Cosmos\Common800\

2-14 Installation Locations

64-bit Architecture

Release Notes Location

XP 32-bit installation

C:\Program Files (x86)\Pervasive\Cosmos9\Common\

64-bit installation

C:\Program Files\Pervasive\Cosmos9 (64-bit)\Common\

Server 2008 32-bit installation Windows 7 C:\Program Files (x86)\Pervasive\Cosmos9\Common\

64-bit installation

C:\Program Files\Pervasive\Cosmos9 (64-bit)\Common\

Settings Folder Settings are unique to each user who logs on to the Windows system where the integration platform is installed.

32-bit Architecture

Settings Folder Location

XP C:\Documents and Settings\username\Application Data\Pervasive\Cosmos9\Common\Settings\

Server 2008 C:\Users\username\AppData\Roaming\Pervasive\Cosmos9\Common\Settings\ Windows 7

8.14 C:\Documents and Settings\username\Application Data\Pervasive\Cosmos\Common800\Settings\

2-15 Installing on Windows

64-bit Architecture

Settings Folder Location

XP 32-bit installation

C:\Documents and Settings\username\Application Data\Pervasive\Cosmos9\Common\

64-bit installation

N/A

Server 2008 32-bit installation Windows 7 C:\Users\username\AppData\Roaming\Pervasive\Cosmos9\Common\

64-bit installation

N/A

2-16 Managing License Files on Windows

Managing License Files on Windows License Manager allows you to locate, add, and refresh license files for the integration platform. You can search for license files on your local machine. This utility enables you to handle licensing separately from product installation. This section covers the following topics:

Launching License Manager Obtaining a License Choosing the Current License Setting the Active License Directory Viewing Enabled Products and Features

Launching ³ To launch License Manager License 1 Select Start > Programs > Pervasive > Data Integrator 9 > Manager Managers > License Manager. 2 The first time License Manager opens, the License Web Service Login window appears. Click OK without logging in.

Obtaining a If you require a license before you can run the integration platform, License you must select a license file.

³ To select a license file 1 In License Manager, do one of the following:

Select the Active Licenses tab and click Find. Select File > Find License File. 2 Enter a directory location and select Search. 3 In the list of licenses found under the directory, select the check box for each needed license. 4 Select Add. The licenses are added to License Manager according to expiration date under the Active Licenses or Inactive Licenses tabs. License files are also copied to the folder selected using Preferences > Active License Directory.

2-17 Installing on Windows

Repository Obtaining a license file with License Manager does not automatically Manager and enable Repository Manager and Document Schema Designer. Document When you open these applications for the first time, each prompts Schema you to select a license file. The prompt offers as a default the active Designer license directory listed in the cosmos.ini file. Because License Manager writes the active license directory path to the cosmos.ini file, it is helpful to first open another application and use License Manager to obtain the license file, then to second open Repository Manager or Document Schema Designer. In this way, the license file is readily available to these two applications.

Choosing the ³ To choose the current license file Current 1 With all other integration applications closed, in License License Manager, select the Active Licenses tab. 2 Click Current to set a license file to use. The current license is marked with two greater-than symbols. When you reopen integration applications, they now point to this license and features authorized by the license are enabled.

Note License Manager notifies you when a license is to expire in 14 days or less. It sorts active and inactive licenses to help you determine which license is expiring.

Setting the The active license directory contains the license files that activate Active License integration products and features. Directory ³ To set the active license directory 1 Select Preferences > Active License Directory. A dialog box displays the current folder where license files are stored. 2 If needed, enter a new license directory and click OK.

Viewing ³ To view a license Enabled 1 Select the Active Licenses or Inactive Licenses tab. Products and Features 2 Select a license and click View to display the connectors, features, and products that it enables.

2-18 Upgrading from Previous Versions

Upgrading from Previous Versions You can upgrade to the current release without losing existing files. All maps and processes can run in the new version.

Table 2-1

From To See the Topic

9.x 9.2.5 Either a stand-alone or a side-by-side installation. In the side-by-side case, use the 32-bit Design Studio to edit maps and processes that you want to run on the 64-bit engine. See “Installation Instructions” on page 2-5.

8.x 9.x See “Upgrading from 8.x to 9.x”.

7.x 9.x You must first upgrade from 7.x to 8.x, and then from 8.x to 9.x. To upgrade from 7.x, you must temporarily install 8.14.

7.x 8.x See “Upgrading from 7.x to 8.x”.

Use these procedures to prepare to install, follow an upgrade scenario, and bring any extract scripts forward to the upgrade.

Upgrading with Connectivity Pack Already Installed Additional If you installed a Pervasive Connectivity Pack and then install Data Connectors or Integrator, the connectivity pack features are overwritten by the new Components Data Integrator version of the connectors. Previously set property Installed options are not reset by the installation. The only exception is the Siebel EAI Invoker component, which is in the connectivity pack but not in Data Integrator. This component is not overwritten by the Data Integrator installer.

Manually Installed Connectors or Components If you manually install a connector or component not included in a connectivity pack, the connector or component installation is separate from the Data Integrator installation and is not removed or overwritten by the Data Integrator installer.

Upgrading from It is best practice to back up your repositories before upgrading. 9.x to 9.2.5

2-19 Installing on Windows

To upgrade from 9.0.3 or later, install Data Integrator 9.2.5 using the steps under “Installation Instructions” on page 2-5. You do not need to remove Data Integrator 9.0.3. To upgrade from 9.0.0, 9.0.1, or 9.0.2, remove Data Integrator 9.x using Windows Add and Remove Programs. Next, install Data Integrator 9.2.5 using the steps under “Installation Instructions” on page 2-5.

Upgrading from You can upgrade from 8.x to 9.x in one of two ways: 8.x to 9.x Replace 8.x with 9.x. Add 9.x to run side-by-side with your existing 8.x system. Side-by-side installation works for both Design Studio and engine-only environments for 8.x and 9.x.

Note Installing the 9.x engine side-by-side with an existing 8.x system where the COM SDK is in use is not recommended because of possible Windows Registry conflicts.

³ To prepare to install 9.x 1 In the 8.x installation, back up Cosmos_Work. 2 Back up all repositories outside of Cosmos_Work. 3 Back up the file InstallDir\Common800\extractor800.mdb. 4 Choose a location for installing 9.x. It cannot be the same location as 8.x. You may install 9.x in another folder on the same machine or on a new one.

³ To choose an upgrade scenario Before you upgrade from 8.x to 9.x, choose scenario A, B, C, or D:

(A) Install and run 9.x on a new machine while continuing to run 8.x on its original machine. Install 9.x side-by-side on the 8.x machine and do one of the following: (B) Enable 9.x to run 8.x files directly and no longer use 8.x. (C) Copy files from 8.x to 9.x and continue running them in both versions.

2-20 Upgrading from Previous Versions

(D) Run both 8.x and 9.x without copying 8.x files to 9.x. Each scenario requires different steps for workspaces and repositories. The first 9.x designer application you open prompts you for a workspace location. See the following table for choices.

Table 2-2 Scenarios for Upgrading from 8.x to 9.x

Workspaces Repositories At Run Time

A 1. Install 9.x on a new machine. 1. Copy repositories in Continue running 8.x maps and 2. Select Cosmos9_Work as the Cosmos_Work from the original processes in 8.x. machine to Cosmos9_Work on root directory and copy all Run the copied maps and Cosmos_Work files to it from the the new machine and use processes in 9.x. 8.x machine. Repository Explorer to add them as repositories in 9.x. Note: Do not attempt to move 2. Copy repositories outside files saved in 9.x to 8.x. They Cosmos_Work from the original to cannot be run in 8.x. the new machine and add them in 9.x. See also “More Considerations When Upgrading from 8.x.”

B 1. Install 9.x side-by-side on the 8.x Use Repository Explorer in 9.x to add Stop running 8.x. machine. the 8.x repositories. Run maps and processes in 9.x. 2. Do one of the following: • Select Cosmos_Work as the root Note: Once you have opened directory instead of and saved files in 9.x, you can Cosmos9_Work. reverse this upgrade scenario only by using a backup of the 8.x • Select Cosmos9_Work as the files. root directory and move over (do not copy) all Cosmos_Work files from 8.x.

C 1. Install 9.x side-by-side on the 8.x 1. Use Repository Explorer in 9.x to Continue running 8.x maps and machine. confirm that 8.x files are in the 9.x processes in 8.x. 2. Copy all Cosmos_Work files to repository. Run the copied maps and Cosmos9_Work. For any existing 2. Copy repositories outside processes in 9.x. files, choose to replace them. Cosmos_Work to new folders and 3. Change location of add them in 9.x. Note: Do not attempt to move Cosmos_Work to reduce risk of 3. Change locations of all 8.x files saved in 9.x to 8.x. They run-time conflicts. repositories to reduce risk of cannot be run in 8.x. run-time conflicts. See also “More Considerations When Upgrading from 8.x.”

D 1. Install 9.x side-by-side on the 8.x 1. Leave 8.x repositories as they Continue running 8.x maps and machine. are. processes in 8.x.

2. Leave Cosmos_Work as it is. 2. Create 9.x repositories but copy Create and run new 9.x maps no files to them. 3. Create Cosmos9_Work but copy and processes in 9.x. no files to it. See also “More Considerations When Upgrading from 8.x.”

2-21 Installing on Windows

More Considerations When Upgrading from 8.x

For side-by-side installations on a single machine, if you added the 8.x engine location to the path environment variable, change it to refer to the 9.x engine. Engine locations in 9.x copies of 8.x batch files must be changed. Engine locations in 9.x copies of 8.x .rifl code modules must be changed. If you want to open and modify an 8.x map or process in 9.x and save the changes, first make a backup of the original 8.x version for future reference. It is good practice to make similar copies for the batch and .rifl files mentioned above.

³ To bring 8.x extract scripts forward to 9.x If you used extract scripts in 8.x, do the following: 1 On the 8.x machine, copy InstallDir\Common800\extractor800.mdb to another location and rename it extractor900.mdb. 2 On the 9.x machine, use this renamed file to replace InstallDir\Common\extractor900.mdb.

Upgrading from Before you can upgrade to 9.x, you must upgrade from 7.x to 8.x. 7.x to 8.x In a single-user 7.x system, all processes, transformations, and extract schemas are stored in a Microsoft Access file djwin7xx.mdb, where 7xx is the version. Extract scripts, if you use them, are stored in extractor7xx.mdb. Follow these steps to bring your 7.x files forward to the 8.x release: 1 Back up both djwin7xx.mdb and extractor7xx.mdb. 2 If you have not already done so, install 8.x on the same machine in a new folder or on a different machine. 3 Convert your djwin7xx.mdb file to 8.x repository files. See “To use the Upgrade Utility” in this topic. 4 If you have exported processes or transformations to .djs files that no longer exist in djwin7xx.mdb, you can import them into your new 8.x system. See “To import a .djs file”in this topic. 5 If extractor7xx.mdb has extract scripts, do one of the following:

2-22 Upgrading from Previous Versions

Run Extract Schema Designer, select File > Change Database, in the dialog box select and open extractor7xx.mdb, and then follow the steps given. Use Extract Schema Transfer Utility copy schemas from extractor7xx.mdb to the newly installed extractor800.mdb. See “Copying Extract Schemas” on page 1-1 in Extract Schema Transfer Utility User’s Guide.

³ To use the Upgrade Utility 1 In the 8.x installation, open the Upgrade Utility from the Start menu and use it to select your djwin7xx.mdb file. This file can be in its old location or a new one where you have copied it. 2 Select the 8.x workspace and repository where you want to store the upgraded maps, processes, and schemas. 3 Click Start Upgrade. You can view the upgrade progress in the Upgrade Status box. 4 When the upgrade is complete, click Done. When you open Repository Explorer, transformations appear as .map.xml and .tf.xml files, processes as .ip.xml files, and structured schemas as .ss.xml files.

³ To import a .djs file 1 Back up a copy of the .djs file to a safe location. 2 In Map Designer, click File > Open old spec (.djs) file. 3 Browse to the .djs file and open it. 4 In Map Designer, save the transformation to a repository. 5 In both the Source and Target tabs, refresh the connection by clicking the Refresh and then save the transformation again. 6 See the release notes for changed features that may affect source and target connection properties, schemas, and RIFL scripts. For example, in previous versions, the AfterFirstRecord source event fired before OnDataChange events, but now it fires afterward. You may need to make changes accordingly. The imported transformation is now ready to validate and run.

2-23 Installing on Windows

Encrypted Passwords in Maps New encryption in Microsoft Jet Engine may cause loss of encrypted passwords in maps. Choose one of two solutions:

³ If you have numerous previously generated maps 1 Back up djwin7xx.mdb, the file with the encrypted passwords. 2 In 7.x, replace the djwin7xx.mdb passwords with plain text ones. 3 Repeat the steps of “To use the Upgrade Utility.”

³ If you have only a few previously generated maps Create new maps in 8.x and copy and paste the mapping expressions to save them in the new encoding.

2-24 Sharing Transformation Files

Sharing Transformation Files Your transformations, structured schemas, and processes are stored in XML format. If you want to share these files, save them to a directory that is accessible to everyone who needs them. You also can use source control software, such as Microsoft Visual SourceSafe or CVS. Be careful of path differences between machines. Paths to source, target, structured schema, and reject files are stored in .map.xml files. If one user creates a map using a source file path that exists only locally on his or her machine and then saves the map to a shared network location, an error will occur when another user tries to open the saved map. Consider using macros to define a common path. See “Macro Manager” on page 8-13 for information on defining and using macros.

2-25 Installing on Windows

Solutions for Installation and Startup Issues

Common Issue: Installing Multiple Versions of Integration Products Issues Can I install version 9.x on the same computer with version 8.x? I want to test Map Designer 9.x before putting it into production. Solution Yes, version 9.x and 8.x can be installed side-by-side and run on the same machine. For instructions on side-by-side installation, see “Upgrading from Previous Versions” on page 2-19.

Issue: File Length or Path Location Errors During installation, you receive an error dialog pertaining to file length or path location. Examples

Component Transfer error Component: EngineSDK1 File Group: File: Error: The file name or extension is too long OR File Error The following error occurred on file, C:\dir_of_length_244_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaa\apidd473.rra The system cannot find the path specified Solution The path or file name is too long and you must use a shorter install root. When using either NTFS or FAT32 file systems, the combined limit for drive volume, directory, path, and file name (including separators) is 255 characters. Use an install root path of 100 characters or less so the directories and file names installed will not reach the limit of 255 characters.

2-26 Solutions for Installation and Startup Issues

Issue: Other Applications Are Active The most common installation problems are caused by installing the integration platform while other applications are open. Solution Shared system files are modified at installation and the Windows registry is updated. If one or more of the system files are in use, conflicts may occur that can affect the proper operation of any applications you have installed. Solution Use the Windows Task Manager (Ctrl+Alt+Del) to close all applications, except Explorer (especially any virus scanners, E-mail applications, and Web browsers) prior to installing Map Designer or any other application (including Extract Schema Designer, Integration Engine, and so forth). You must first uninstall and then reinstall the products. See “Remove and Reinstall Instructions” on page 2-29.

Issue: Running djengine.exe in a Side-by-Side Installation After I installed version 9.x on the same computer with 8.x, running djengine -V at a command prompt shows only the 8.x version. How can I run the 9.x version at the command prompt? Solution Create a batch file that includes a command like the following: set path=%path%;"C:\Program Files\Pervasive\Cosmos9\Common\" Then create a shortcut file with a target string like this:

%windir%\System32\cmd.exe /k ".bat" Use the shortcut to open a command prompt in which the batch file sets the path needed for that session.

Issue: Local Rights – Domain Local Group and Mixed Mode The correct user rights are not being granted. If you add a Domain Local Group to a local user right (for example, Log on Locally) on a member server or workstation and the user

2-27 Installing on Windows

right assignment settings in the Local Security Policy are correct, the user may not actually be granted that right. This behavior can occur in a Windows 2000 domain that is running in Mixed Mode. It may also be an issue when domain control is handled by Novell NetWare rather than by a Microsoft controller. A user that is part of a Mixed Mode domain and is a member of the domain local group Administrators (whether or not they have been added to the local Administrators group), will not be granted the necessary rights to install or write to the registry.

Note Local groups created in a Mixed Mode domain behave the same way in Windows XP. The only exception is domain local groups created on a domain controller.

Solution To install correctly on a Windows XP platform running in Mixed Mode, log on to the local computer as local administrator and have rights to write to the local registry.

Interpreting Issue: Unhandled Exception Error Initialization “Unhandled exception: java.lang.OutOfMemoryError” Error Messages Solution 1 This error occurs when insufficient memory is assigned to Java. To change the JVM value from the default, enter a single line in the cosmos.ini file under the [User Info] section. Depending on how much memory you allocate, use one of the following settings:

JvmArgs=-Xmx64m, JvmArgs=-Xmx128m JvmArgs=-Xmx256m The numerical value of the argument indicates the megabytes of memory. You can use values of 64, 128, 256, and 1024 to set the value based on the capabilities of your computer. Solution 2 Use a tool-specific option such as MapDesignerJvmArgs to set a smaller memory value for use by individual tools. For more information on using the JvmArg options, see “Optional [UserInfo] Values” on page 1-11.

2-28 Remove and Reinstall Instructions

Remove and Reinstall Instructions When working to resolve issues, a technical support engineer may ask you to remove and reinstall the integration platform. Use the steps given here.

³ To remove and reinstall the integration platform 1 Back up your workspace directory from the following default location, where username is the account that you use for logon, or from any other place that you have chosen:

XP—C:\Documents and Settings\username\Cosmos9_Work Server 2008 and Windows 7— C:\Users\username\Cosmos9_Work Repeat this step for each user on the machine. 2 If you use extract scripts, back them up. If you have saved extract scripts as .cxl files in your workspace, their backup is covered by the previous step. Scripts may also be stored in extractorxxx.mdb, where xxx is the integration platform version, in the following default locations:

XP—C:\Documents and Settings\username\Application Data\Pervasive\Cosmos9\Common Server 2008 and Windows 7— C:\Users\username\AppData\Pervasive\Cosmos9\Common Back up the entire file or use Extract Schema Transfer Utility to copy extract scripts to another extractorxxx.mdb. Repeat this step for each user on the machine. 3 Close all applications as follows:

Close antivirus software, e-mail, and web browsers. Use Task Manager to close any background applications. Empty the Recycle Bin. 4 Use Add or Remove Programs to remove the integration platform. Workspaces, repositories, and related files are left for use by the reinstallation. Backups you made are for safekeeping.

2-29 Installing on Windows

5 Reinstall using the steps on page 2-5 under “Installation Instructions.”

2-30 chapter Installing on Unix 3

How to Set Up and Install the Products on Unix

This chapter includes the following sections:

“Minimum System Requirements on Unix” on page 3-2 “Before Installation” on page 3-3 “Installation Utility” on page 3-4 “Installation Notes for Unix” on page 3-5 “Managing License Files on Unix” on page 3-6 “Verifying Unix Setup: The Acid Test” on page 3-9 “Reverting to Previous Unix Permissions” on page 3-12 “Moving Files from Windows to Unix” on page 3-13 “Using File Transfer Protocol” on page 3-16

3-1 Installing on Unix

Minimum System Requirements on Unix Before beginning installation, you must verify that your computer meets the resource requirements. The following resources are recommended for a Unix operating system.

A server running a supported Unix operating system At least 400 MB free hard drive space in the installation location, depending upon the installation options selected At least 4 GB RAM One CD-ROM drive unless installing from downloaded files These system requirements are the minimum for satisfactory operation in the configurations shown in the release notes file DataIntegrator_readme.htm. For higher performance, be certain that physical memory meets the requirements of the local system and all applications. When you open a process or transformation, all components are loaded into memory. A large, complex process may consume so much memory that it will not execute. Where business processes permit, redesign of the process may resolve this issue. Execute large, complex processes only on machines that have sufficient memory to load all components of the process while supporting the operating system and all other open applications.

3-2 Before Installation

Before Installation Before beginning installation, verify that the following resources are available.

Permissions You need write permissions to the installation directory.

License File Obtain a license file from your sales representative. Also see “Managing License Files on Unix” on page 3-6.

See Checklist “Checklist Before Installing on AIX” on page 4-2 for Your Unix “Checklist Before Installing on HP-UX” on page 5-2 System “Checklist Before Installing on Linux” on page 6-2 “Checklist Before Installing on Solaris” on page 7-2

3-3 Installing on Unix

Installation Utility The installation creates a Groupid called pvsw, under which djengine runs. The pvsw Groupid permissions are 644. Transformations and resulting files run by a user are owned by the user who created them.

3-4 Installation Notes for Unix

Installation Notes for Unix Unix installation instructions vary according to the brand of Unix operating system. For an installation checklist for your Unix type, see the following topics:

“Checklist Before Installing on AIX” on page 4-2 “Checklist Before Installing on HP-UX” on page 5-2 “Checklist Before Installing on Linux” on page 6-2 “Checklist Before Installing on Solaris” on page 7-2

Additional See your installation CD for complete instructions on installing the Notes on SDKs Data Integrator SDKs on your Unix platform. Installation for Unix instructions vary among the different Unix systems.

Multiuser If you are installing the engine on a server with a multiuser license, License there are a few special steps you must take into consideration for Machine installing, running, and using the engine. Installation 1 Follow all the recommendations provided under “Before Installation” on page 3-3 to ensure a proper installation. 2 Ensure there is sufficient storage and processing memory on the install machine to support multiple users. 3 Consider using Integration Server to manage integration processing on the server. This product is provided with the integration platform. Before using its features, you must configure it and set up user access. See Integration Server and SDK Guide for instructions. 4 When performing the installation, you must be logged on the install machine with Administrator authority. You cannot perform a remote installation from a different machine. 5 After installing the engine on the server, provide instructions to users on how to access the Unix server, their login information (if needed), and directions to other tools that may also be available on that machine.

3-5 Installing on Unix

Managing License Files on Unix License Manager allows you to locate and add license files associated with the features you install. You can search for license files on your local machine. License Manager notifies you when a license is to expire in 14 days or less. It sorts active and inactive licenses to help you determine which license is expiring. The Unix License Manager uses the following files:

LicenseManagerCLI.jar LicenseManagerCLI.sh You must run License Manager as root. Use the following syntax:

LicenseManagerCLI [-i inifile] [-lo outputpath]

Option Parameter

-i Fully qualified path to cosmos.ini file where license location is written

-lo Path name to license file active directory where file is downloaded

-h Help. Ignores other parameters.

To apply the license file for your Unix type, see the following instructions.

³ To apply the license file on AIX Use these steps to apply the default license file. If you do not have one, see “Managing License Files on Unix” on page 3-6. 1 After you have installed the engine, copy the file cosmos.slc to the following default location:

/usr/PervasiveSoftware/IntegrationEngine/license 2 Change the file access permissions as follows:

chmod 644 /usr/PervasiveSoftware/IntegrationEngine/license/ cosmos.slc

3-6 Managing License Files on Unix

3 Next, you may want to verify the installation. See “Steps After Installing on AIX” on page 4-5.

³ To apply the license file on HP-UX Use these steps to apply the default license file. 1 After you have installed the engine, copy the file cosmos.slc to the following location:

/opt/PervasiveSoftware/IntegrationEngine/license 2 Change the file access permissions as follows:

chmod 644 /opt/PervasiveSoftware/IntegrationEngine/license/ cosmos.slc 3 Next, you may want to verify the installation. See “Steps After Installing on HP-UX” on page 5-7.

³ To apply the license file on Linux Use these steps to apply the default license file. 1 After you have installed the engine, copy the file cosmos.slc to the following location:

/opt/PervasiveSoftware/IntegrationEngine/license 2 Change the file access permissions as follows:

chmod 644 /opt/PervasiveSoftware/IntegrationEngine/license/ cosmos.slc Next, you may want to verify the installation. See “Steps After Installing on Linux” on page 6-5.

³ To apply the license file on Solaris Use these steps to apply the default license file. 1 After you have installed the engine, copy the file cosmos.slc to the following location:

/opt/PervasiveSoftware/IntegrationEngine/license 2 Change the file access permissions as follows:

3-7 Installing on Unix

chmod 644 /opt/PervasiveSoftware/IntegrationEngine/license/ cosmos.slc 3 Next, you may want to verify the installation. See “Steps After Installing on Solaris” on page 7-5.

3-8 Verifying Unix Setup: The Acid Test

Verifying Unix Setup: The Acid Test The following steps can verify that the engine is working correctly.

Basic Setup Basic Test 1 Testing 1 Transfer a copy of the file named tutor1.asc (found in the Windows default installation directory) to your Unix system (FTP transfer of tutor1.asc must be done in ANSI mode). 2 Run this command from the engine directory:

DJEngine -Source_T "ASCII (Delimited)" -Source_C tutor1.asc -Target_T "ASCII (Delimited)" -Target_C tutor1out.asc This test creates a default transformation where the source type, source file, target type, and target file are all declared directly from the command line. If the transformation does not produce a Return Code 0, the test has failed and the engine is not properly configured. You will need to review the installation directions and try again.

Basic Test 2 After running the first test successfully, run a transformation using transformation and map files created with Map Designer. Create your transformation (on a Windows machine) using the steps below, save the files (.tf.xml and .map.xml), and then copy them to your Unix system. The transformation itself should use a relative path when declaring both source and target files.

³ To create a new transformation Open Map Designer and use these steps to create a new transformation with the following specifications: 1 On the Source Connection tab, select the ASCII (Delimited) source connector. 2 Click the Source File/URL arrow, then browse to and select InstallDir\Common\tutor1.asc . Click Open. 3 Click Connect.

3-9 Installing on Unix

4 On the Target Connection tab, select the ASCII (Delimited) target connector. 5 Type tutor1out.asc in the Target File field and click Connect. 6 On the Map tab, drag the asterisk (*) to map all fields from source to target. 7 Run the transformation once so that the default Source R1 AfterEveryRecord event handler ClearMapPut action is added. Or, if you prefer, manually add that action for that event. 8 Save the transformation, naming it test, and close the file. 9 From your workspace, open test.tf.xml with a text editor. 10 Change any Windows paths to the Unix engine 9.x path, appending file names. 11 Using binary FTP mode, copy the transformation file (test.tf.xml) and map file (test.map.xml) to the engine 9.x directory on the Unix system. 12 Run the following on the command line from the engine directory:

djengine test.tf.xml If the transformation runs successfully, you will see Return Code 0. If not, then one of the previous steps was not done correctly. You will need to review and redo the transformation.

Advanced Because the following examples use a relational database Setup Testing management system (RDBMS), a RDBMS client must be on the Unix platform as well as the Windows platform before you can design for Unix in a Windows environment. The settings for each RDBMS client must match or the engine will be unable to connect to the database.

Advanced Test 1 If your process has a connection to a database, such as Oracle, you should go through this test to ensure proper connectivity between the engine and the database. The test is almost the same as “Basic Test 1” on page 3-9, but the target connection should use your specific database connection information. Here is an example of the test against an Oracle 8.04 database on a Solaris platform:

3-10 Verifying Unix Setup: The Acid Test

DJEngine -Source_T "ASCII (Delimited)" -Source_C tutor1.asc -Target_T "Oracle 8.x" -Target_C "Database=test;UserId=tester;Password=ora804;Table=TEST" Make sure you pass the proper connection information pertinent to your environment. If the transformation is successful, you will see Return Code 0. If you receive error messages such as Unable to load library[25547], you do not have the client portion of your database system installed or it is not properly configured. In order to connect to a database such as Oracle or Sybase, the full client portion must reside on the Unix system. Refer to your specific database installation documentation for configuration requirements of the database to which you are attempting to connect. You may test the source connection process by reversing the previous test, using your database connection as the source.

Advanced Test 2 This is the same as “Advanced Test 1” on page 3-10, but it uses Map Designer to set up the transformation that will be transferred to the Unix system.

After Testing Once these tests are successful, all Unix configuration requirements have been met.

3-11 Installing on Unix

Reverting to Previous Unix Permissions If the security settings applied during installation are not appropriate for your environment, you can revert to your previous permissions by executing the following commands as root: chmod 755 /usr/PervasiveSoftware/IntegrationEngineversion/djengine chmod 777 /usr/PervasiveSoftware/IntegrationEngineversion/Plug-Ins chmod 777 /usr/PervasiveSoftware/IntegrationEngineversion/lookup chmod 777 /usr/PervasiveSoftware/IntegrationEngineversion/eaa chmod 777 /usr/PervasiveSoftware/IntegrationEngineversion/jars chmod 777 /usr/PervasiveSoftware/IntegrationEngineversion/license The Unix security feature for the integration platform is now disabled.

3-12 Moving Files from Windows to Unix

Moving Files from Windows to Unix When executing transformations and processes with a Unix engine, you should be aware of certain issues. This topic contains tips on file formats and cross-platform portability.

ASCII Data ASCII data files (fixed, tagged, and delimited) contain specific record Files separators and field separators. When you use one of the design tools to create an extract or a transformation, those separator specifics are saved as part of the extract or transformation. If the data file is moved from Windows to Unix using FTP, it is important to use binary mode during the upload to preserve the same separators in the data file. Otherwise, the separators may change (usually resulting in loss of the carriage return) and may cause incorrect reading of the data by the extract or transformation on Unix.

Other ASCII There are other ASCII text files that may be used in your extracts and Files transformations, such as extract rules, reject files, code modules, error logs, lookup tables, and others. Again, in order to preserve the exact format of each of these files when moving them from Windows to Unix, it is important to use binary mode when the files are transmitted with FTP.

Option Files When creating an option (OPT) file or a batch file, it must either be created on the Unix system using a Unix text editor, or it must be transferred from a Windows system using ANSI mode through FTP. This will ensure that there are no CR-LFs (the Windows record separator) in the option or batch file that will make it invalid on the Unix system, which uses LFs as record separators. To check the Unix compatibility of an option or batch file, print the file from the console using the Unix cat command.

cat filename.opt

Other If you want to run your transformations on Unix, there are several Considerations items to consider before starting your transformation design session. Planning ahead will simplify the process of porting, configuring, and executing on Unix.

3-13 Installing on Unix

ActiveX and External DLLs Do not use an ActiveX component in your Windows-designed transformations if that ActiveX component is not supported on Unix. Do not use an external DLL in your Windows-designed transformations if that external DLL is not supported on Unix.

Database Connections For SQL database connections, command line overrides may not be required if the database environment is identical on the Unix and Windows systems. When the database environments are different, because you are moving from a test environment to a production environment, or if you are moving from a development environment to a customer’s site, you should use command line overrides to connect within the target system.

ODBC Your data source name (DSN) must be the same on the Windows system and the Unix system. Because you do not need driver options in Unix, before you save your file, go to your source or target properties to change ModifyDriverOptions to True and delete any value for DriverOptions.

Option Files If you do not want to read source data or write target data to the directory where you are designing your transformation, set up an option file that contains override connect information. For more information, see “Option Files” on page 3-9 in Integration Engine User’s Guide. Reject connections, code modules, error logs, and Extract Schema Designer scripts can be passed to Unix in .map.xml and .tf.xml files, and on the engine command line. When creating an option (OPT) file or a batch file, do one of the following.

Use a Unix text editor.

3-14 Moving Files from Windows to Unix

Use FTP to send the option file from a Windows system using ANSI transfer protocol. This ensures that no CR-LFs exist in the option or batch file that make it invalid on the Unix system. Unix uses LFs as record separators instead of CR-LFs, the Windows record separator. To check the Unix compatibility of an option or batch file, print the file from the console using the Unix cat command. cat filename.opt

Relative Paths If your file connection contains a path, use a path relative to your installation directory, usually ..\Common.

Note If you use relative paths, your data must be in that directory on both the Unix and Windows systems. This includes source files, reject files, code modules, error logs, and lookup tables.

Dynamic SQL Lookups Because you cannot override field expressions or transformation and map properties from an option file or a command line, you must put your dynamic SQL declarations in a code module within a .RIFL file.

SQL Your source and target connectors must be the same in your design environment and your production environment. For example, if you are connecting to Oracle 7.x on your Windows design system, you must use Oracle 7.x on your Unix production system as well. Do not assume that you can design a transformation for Oracle 7.x and have it work on the Oracle 8.x connector.

Limitations SQL Server databases cannot be used within a Linux environment. SQL Server is based on Microsoft libraries that do not run on the Linux operating system.

3-15 Installing on Unix

Using File Transfer Protocol File Transfer Protocol (FTP) uses a connection over a Transmission Control Protocol/Internet Protocol (TCP/IP) network to transfer files between two computer systems, a local host and a remote host. While you are logged on to a local host and running its FTP program, you can transfer files to and from a remote host by means of a server program that resides on the remote host.

Using FTP If you are using a Windows operating system, your FTP program may provide a graphical user interface (GUI) for file transfers, such as WS_FTP. If so, you may not need to use the FTP commands described in this document. If you are using a non-GUI FTP interface (such as the interface you get when you use the Windows Start and Run menu options to open FTP on a Windows system), some of the commands listed in this document will be useful to you.

Before You Run You must connect to the Unix host machine and log in. Before FTP on Unix running the FTP program, use the Unix command cd, if necessary, to change your working directory to the one you want to send files from (or, conversely, change to the directory where you want to receive remote files when you get them). This approach makes it easy for you to specify file names. If you do not change directories before you run FTP, there are two ways for you to specify a different local directory even though you have already begun to run FTP. You can use the lcd command within FTP to change your local directory (see Table 3-1 “FTP Commands”) or you can write out the relative path name of the files as in the following example:

send dir/file myremotefile

Running FTP To run FTP, type the following command at the Unix system prompt:

ftp For complete FTP help, type the following command at the Unix system prompt:

man ftp

3-16 Using File Transfer Protocol

The following table describes some FTP commands you may need.

Table 3-1 FTP Commands

FTP Command Description

open remotehost Use this command to connect to another computer host. When prompted for your name, type in your remote login name. You will be prompted for your remote machine password.

user username If you are not prompted for your login name after you open the remote host, type in user followed by your username.

pwd Prints out the name of the working directory on the remote machine.

ls or dir Lists the files in your remote directory.

cd directorypath To change to another directory on the remote machine, type cd followed by the path to the directory to which you want to change.

lcd directory Change your local working directory to another one on the local machine.

send localfile remotefile Use this command to send a local file to the remote machine. You can keep the same file name or specify another file name to save as on your remote machine.

mput localfilespec Use this command with wildcard characters to put multiple local files onto the remote machine.

get remotefile localfile Use this command to get (copy) a file from the remote machine to your local machine. You can keep the same file name or specify another file name to “save as” on your local machine.

mget remotefilespec Use this command with wildcard characters to get multiple local files from the remote machine onto your local machine.

!command Use the exclamation followed by a Unix command to run a Unix command from within FTP.

quit To exit from FTP, type quit.

Ctrl+C To abort an FTP process without exiting from FTP, hold down the Ctrl key and press the C key. See Note below.

help or ? To get a list of FTP commands, type help or ? at the FTP prompt.

3-17 Installing on Unix

Note On Unix systems, if you use a Ctrl+C key combination to abort an FTP process, such as a file transfer or directory listing, you may get an incomplete termination. In that case, your workstation will not respond to any FTP commands or Unix commands you type. If this happens, press Ctrl+Z to stop the hung FTP process. Then, use the Unix command ps to list all your Unix processes. The list will show the stopped FTP job, along with its PID (process ID) number. To get rid of the stopped FTP job, type the Unix command kill -HUP pidnumber.

Case Distinction When you type a Unix file name, it makes a difference when you use uppercase or lowercase letters, because Unix is case-sensitive. The file name you type must exactly match the file name on the Unix system. For example, when you tell Unix to get a file whose name you type as cosmosfile, Unix will not find files named CosmosFile or COSMOSFILE.

Default FTP Mode By default, when using FTP, a file is copied in ASCII mode. In case you need to copy a file in binary mode, type binary at the FTP prompt, before executing a get or send command. To know which mode you are in currently, enter type at the FTP prompt to display the mode.

3-18 chapter Installing on AIX 4

How to Set Up and Install the Products on AIX

This section includes the following topics:

“Checklist Before Installing on AIX” on page 4-2 “To Install on AIX from a CD” on page 4-3 “To Install on AIX from a Downloaded Archive” on page 4-4 “Steps After Installing on AIX” on page 4-5

4-1 Installing on AIX

Checklist Before Installing on AIX Before installing the engine:

Make sure you have a root user ID and password. If desired, remove any previous version of the engine by determining which versions you have installed and then running the installp command: lslpp -la | grep IntegrationEngine

installp -u IntegrationEngineversion

Note When running the installp command, leave off the final digit of the version number.

Run the following command to ensure you have at least AIX 5.3.0.50 installed:

oslevel -g

The slibclean command unloads object files that have load and use counts of 0, those that are no longer used in the shared library region, and those that are no longer required in the shared library and kernel text regions. Before and after you install the engine on AIX, run /usr/sbin/slibclean as root. Once you have met these requirements, you are ready to install. See one of the following topics:

“To Install on AIX from a CD” on page 4-3 “To Install on AIX from a Downloaded Archive” on page 4-4

4-2 To Install on AIX from a CD

To Install on AIX from a CD

1 Log on to the machine using as a root user. 2 Mount the CD.

mount -r -v cdrfs /dev/cddevice /cdrom For /dev/cddevice, use your CD device name. 3 Create a temporary directory. This example uses mytmp.

mkdir /mytmp 4 Copy the archive to the directory.

cp dje900-aix-version.tar.gz /mytmp For version, use the values from the .tar.gz file name. 5 Change to the directory.

cd /mytmp 6 Uncompress the archive.

gunzip dje900-aix-version.tar.gz 7 Untar the archive.

tar xvf dje900-aix-version.tar 8 Run the installer.

dje900-aix-version.sh 9 Read the license agreement and type Yes to accept. The software is installed in the following location:

/usr/PervasiveSoftware/IntegrationEngineversion After installing, apply the license. See “Managing License Files on Unix” on page 3-6.

4-3 Installing on AIX

To Install on AIX from a Downloaded Archive

1 Log on to the machine as a root user. 2 Create a temporary directory. This example uses mytmp.

mkdir /mytmp 3 Copy the archive to the directory.

cp dje900-aix-version.tar.gz /mytmp For version, use the values from the .tar.gz file name. 4 Change to the directory.

cd /mytmp 5 Uncompress the archive.

gunzip dje900-aix-version.tar.gz 6 Untar the archive.

tar xvf dje900-aix-version.tar 7 Run the installer.

dje900-aix-version.sh 8 Read the license agreement and type Yes to accept. The software is installed in the following location:

/usr/PervasiveSoftware/IntegrationEngineversion After installing, apply the license. See “Managing License Files on Unix” on page 3-6.

4-4 Steps After Installing on AIX

Steps After Installing on AIX To ensure that the engine is set up correctly, follow these steps after installation: 1 Verify installed engine versions. You can determine the engine versions installed on your AIX system with the following command:

djengine -V 2 Set the environment variables as shown for all engine users. Make sure the path to the engine is at the beginning of each environment variable.

export PATH=$PATH:/usr/PervasiveSoftware/IntegrationEngine version

export LIBPATH=$LIBPATH:/usr/PervasiveSoftware/Integration Engineversion 3 For any Java application that uses the engine SDK for Java or the IS SDK, make sure to include djcosmos_core.jar in the CLASSPATH environment variable. Also reference any .jar files needed by the specific application. The following example references djcosmos_core.jar and the engine SDK .jar file, djec.jar:

java -cp /usr/PervasiveSoftware/IntegrationEngine921/djcosmo s_core.jar:/usr/PervasiveSoftware/IntegrationEngine 921/djec.jar MyECJavaApp

4 The slibclean command unloads object files that have load and use counts of 0, those that are no longer used in the shared library region, and those that are no longer required in the shared library and kernel text regions. Before and after you install the engine on AIX, run /usr/sbin/slibclean as root. See Also “Verifying Unix Setup: The Acid Test” on page 3-9

4-5 Installing on AIX

4-6 chapter Installing on HP-UX 5

How to Set Up and Install the Products on HP-UX

This section includes the following topics:

“Checklist Before Installing on HP-UX” on page 5-2 “To Install on HP-UX from a CD” on page 5-4 “To Install on HP-UX from a Downloaded Archive” on page 5-6 “Steps After Installing on HP-UX” on page 5-7

5-1 Installing on HP-UX

Checklist Before Installing on HP-UX Before installing the engine:

Install Java Development Kit. For supported versions, see the general release notes. To verify your JDK version, use the following command:

java -version See the following web site to download the Java SDK:

http://www.hp.com/products1/unix/java/

Make sure the Standard C++ run time libraries for -AA support (.depot) are installed. For information, see www.hp.com. Have a root user ID and password. If desired, remove any previous version of the engine by running the following command as root:

swremove IntegrationEngineversion

Note When running the swremove command, leave off the final digit of the version number.

Upgrade the libjvm and libfontmanager libraries. If you do not upgrade the SDK and libraries, an error appears when you attempt to run the engine.

³ To upgrade the libraries 1 Verify that Java is at the beginning of the path environment variable:

export PATH=/opt/javaversion/bin:$PATH For version, use your Java version number.

5-2 Checklist Before Installing on HP-UX

2 Change SHLIB_PATH to include the following:

/opt/javaversion/jre/lib/PA_RISCversion/hotspot:/opt/j avaversion/jre/lib/PA_RISCversion Once you have met these requirements, you are ready to install. See one of the following topics:

“To Install on HP-UX from a CD” on page 5-4 “To Install on HP-UX from a Downloaded Archive” on page 5-6

5-3 Installing on HP-UX

To Install on HP-UX from a CD

1 Log on to the machine as a root user. 2 Mount the CD. 3 Create a temporary directory. This example uses mytmp.

mkdir /mytmp 4 Copy the archive to the directory.

cp dje900-hpux-version.tar.gz /mytmp 5 Change to the directory.

cd /mytmp 6 Uncompress the archive.

gunzip dje900-hpux-version.tar.gz 7 Untar the archive.

tar xvf dje900-hpux-version.tar 8 Run the installation.

dje900-hpux-version.sh 9 Read the license agreement and type Yes to accept. The software is installed in the following location:

/opt/PervasiveSoftware/IntegrationEngineversion

Installation If you have difficulty installing using the above steps, try the Tips following options. In all examples, for /dev/cddevice, use your CD device name.

HP-UX does not support the Rock Ridge CD format used to make the installation CD. To view files in lower case with long file names, mount the CD with the following command:

mount /dev/cddevice /cdrom

If you have problems viewing the CD contents, mount the CD with the following arguments:

mount -F cdfs -o ro /dev/cddevice /cdrom

5-4 To Install on HP-UX from a CD

If the CD still does not have the proper file names, try to install the Portable File System (PFS) file set available from HP. Once the PFS package daemon is installed, do the following as root user: 1 Edit /etc/pfs_fstab and add the following entry if it does not already exist:

/dev/cddevice /cdrom pfs-rrip xlat=unix 0 0 2 Run the following as the root user:

pfs_mountd & pfsd & pfs_mount /cdrom After installing, apply the license. See “Managing License Files on Unix” on page 3-6.

5-5 Installing on HP-UX

To Install on HP-UX from a Downloaded Archive

1 Log on to the machine as a root user. 2 Create a temporary directory. This example uses mytmp.

mkdir /mytmp 3 Copy the archive to the directory.

cp dje900-hpux-version.tar.gz /mytmp 4 Change to the directory.

cd /mytmp 5 Uncompress the archive.

gunzip dje900-hpux-version.tar.gz 6 Untar the archive.

tar xvf dje900-hpux-version.tar 7 Run the installation.

dje900-hpux-version.sh 8 Read the license agreement and type Yes to accept. The software is installed in the following location:

/opt/PervasiveSoftware/IntegrationEngineversion After installing, apply the license. See “Managing License Files on Unix” on page 3-6.

5-6 Steps After Installing on HP-UX

Steps After Installing on HP-UX To ensure that the engine is set up correctly, follow these steps after installation: 1 Set the PATH environment variable for all engine users as shown. Make sure the path to the engine is at the beginning of the environment variable.

export PATH=$PATH:/opt/PervasiveSoftware/IntegrationEngine version 2 Source the /etc/SHLIB_PATH file:

export SHLIB_PATH="$(cat /etc/SHLIB_PATH):$SHLIB_PATH" 3 For any Java application that uses the engine SDK for Java or the IS SDK, make sure to include djcosmos_core.jar in the CLASSPATH environment variable. Also reference any .jar files needed by the specific application. The following example references djcosmos_core.jar and the engine SDK .jar file, djec.jar:

java -cp /opt/PervasiveSoftware/IntegrationEngine921/djcosmo s_core.jar:/opt/PervasiveSoftware/IntegrationEngine 921/djec.jar MyECJavaApp 4 Verify installed versions by using the following command:

swlist -v -l product IntegrationEngine See Also “Verifying Unix Setup: The Acid Test” on page 3-9

5-7 Installing on HP-UX

5-8 chapter Installing on Linux 6

How to Set Up and Install the Products on Linux

This section includes the following topics:

“Checklist Before Installing on Linux” on page 6-2 “To Install on Linux from a CD” on page 6-3 “To Install on Linux from a Downloaded Archive” on page 6-4 “Steps After Installing on Linux” on page 6-5

6-1 Installing on Linux

Checklist Before Installing on Linux Before installing the engine:

Make sure you have a root user ID and password. Ensure you are running a 64-bit version of Linux by running the following command. The output for a 64-bit version is x86_64.

uname -p

If desired, remove any previous versions of the engine by running the following command as root:

rpm -e IntegrationEngine-version-date

Obtain a license file. The license file to use for your solution is provided by your sales representative. This license .slc file determines which integration products can be used. If you need to update features for your existing installation, contact your sales representative. For information on how to use License Manager to access your new license file, see “Managing License Files on Unix” on page 3-6. Once you have met these requirements, you are ready to install. See one of the following topics:

“To Install on Linux from a CD” on page 6-3 “To Install on Linux from a Downloaded Archive” on page 6-4

6-2 To Install on Linux from a CD

To Install on Linux from a CD Installing the engine on either the Red Hat or SUSE Linux operating systems is the same procedure. 1 Log on to the machine as a root user. 2 Mount the CD.

cd /media/cdrom For /media/cdrom, use your CD device name. 3 Create a temporary directory. This example uses mytmp.

mkdir /mytmp 4 Copy the archive to the directory.

cp dje900-linux64-version.tar.gz /mytmp For version, use the value from the .tar.gz file name. 5 Change directories to the temporary directory.

cd /mytmp 6 Uncompress the archive.

gunzip dje900-linux64-version.tar.gz 7 Untar the archive.

tar xvf dje900-linux64-version.tar 8 Run the installation utility.

dje900-linux64-version.sh 9 Read the license agreement and type Yes to accept. The software is installed in the following location:

/opt/PervasiveSoftware/IntegrationEngineversion After installing, you must apply the license file. See “Managing License Files on Unix” on page 3-6.

6-3 Installing on Linux

To Install on Linux from a Downloaded Archive Installing the engine on either the Red Hat or SUSE Linux operating systems is the same procedure. 1 Log on to the machine as a root user. 2 Create a temporary directory. This example uses mytmp.

mkdir /mytmp 3 Copy the archive to the directory.

cp dje900-linux64-version.tar.gz /mytmp For version, use the value from the .tar.gz file name. 4 Change directories to the temporary directory.

cd /mytmp 5 Uncompress the archive.

gunzip dje900-linux64-version.tar.gz 6 Untar the archive.

tar xvf dje900-linux64-version.tar 7 Run the installation utility.

dje900-linux64-version.sh 8 Read the license agreement and type Yes to accept. The software is installed in the following location:

/opt/PervasiveSoftware/IntegrationEngineversion After installing, apply the license. See “Managing License Files on Unix” on page 3-6.

6-4 Steps After Installing on Linux

Steps After Installing on Linux To ensure that the engine is set up correctly, follow these steps after installation: 1 Set the PATH environment variable as shown for all engine users. Make sure the path to the engine is at the beginning of the environment variable.

export PATH=$PATH:/opt/PervasiveSoftware/IntegrationEngine version 2 Run the following command to verify the installation:

rpm -qi -a | grep IntegrationEngine 3 For any Java application that uses the engine SDK for Java or the IS SDK, make sure to include djcosmos_core.jar in the CLASSPATH environment variable. Also reference any .jar files needed by the specific application. The following example references djcosmos_core.jar and the engine SDK .jar file, djec.jar:

java -cp /opt/PervasiveSoftware/IntegrationEngine921/djcosmo s_core.jar:/opt/PervasiveSoftware/IntegrationEngine 921/djec.jar MyECJavaApp See Also “Verifying Unix Setup: The Acid Test” on page 3-9

6-5 Installing on Linux

6-6 chapter Installing on Solaris 7

How to Set Up and Install the Products on Solaris

This section includes the following topics:

“Checklist Before Installing on Solaris” on page 7-2 “To Install on Solaris from a CD” on page 7-3 “To Install on Solaris from a Downloaded Archive” on page 7-4 “Steps After Installing on Solaris” on page 7-5

7-1 Installing on Solaris

Checklist Before Installing on Solaris Before installing the engine:

Make sure you have a root user ID and password. Uninstall any previous version of the engine by running the following command as root:

pkgrm Engineversion

7-2 To Install on Solaris from a CD

To Install on Solaris from a CD

1 Log on to the machine as a root user. 2 Mount the CD.

mount -F cdfs /dev/cddevice /cdrom For /dev/cddevice, use your CD device name. 3 Uncompress the archive.

gunzip dje900-solaris-version.tar.gz 4 Untar the archive.

tar xvf dje900-solaris-version.tar 5 Run the installation utility.

dje900-solaris-version.sh 6 Read the license agreement and type Yes to accept. The software is installed in the following location:

/opt/PervasiveSoftware/IntegrationEngineversion After installing, apply the license. See “Managing License Files on Unix” on page 3-6.

7-3 Installing on Solaris

To Install on Solaris from a Downloaded Archive

1 Log on to the machine as a root user. 2 Create a temporary directory. This example uses mytmp.

mkdir /mytmp 3 Copy the archive to the directory.

cp dje900-solaris-version.tar.gz /mytmp For version, use the values from the .tar.gz file name. 4 Change to the directory.

cd /mytmp 5 Uncompress the archive.

gunzip dje900-solaris-version.tar.gz 6 Untar the archive.

tar xvf dje900-solaris-version.tar 7 Run the installation utility.

dje900-solaris-version.sh 8 Read the license agreement and type Yes to accept. The software is installed in the following location:

/opt/PervasiveSoftware/IntegrationEngineversion After installing, apply the license. See “Managing License Files on Unix” on page 3-6.

7-4 Steps After Installing on Solaris

Steps After Installing on Solaris To ensure that the engine is set up correctly, follow these steps after installation: 1 Set the PATH environment variable for all engine users as shown. Make sure the path to the engine is at the beginning of the environment variable.

export PATH=$PATH:/opt/PervasiveSoftware/IntegrationEngine version 2 Check for installed versions. You can determine the engine versions installed on your machine with the following command:

pkginfo -l Engine 3 For any Java application that uses the engine SDK for Java or the IS SDK, make sure to include djcosmos_core.jar in the CLASSPATH environment variable. Also reference any .jar files needed by the specific application. The following example references djcosmos_core.jar and the engine SDK .jar file, djec.jar:

7-5 Installing on Solaris

7-6 chapter Repositories, Workspaces, and Macros 8

Their Use in the Integration Environment

This chapter includes the following sections:

“Introducing Workspaces, Repositories, and Collections” on page 8-2 “Using Workspace Manager” on page 8-5 “Macro Manager” on page 8-13

8-1 Repositories, Workspaces, and Macros

Introducing Workspaces, Repositories, and Collections This product version differs from previous versions in the way that it stores transformation and process files. In practical terms, the basic difference is that you now have better control and more options regarding how your files are stored. Previous versions stored transformation and process metadata in Microsoft Access. This proprietary format imposed its own unique limitations and restricted cross-platform compatibility and portability. In newer versions, metadata is stored in compatible, XML format. The following figure illustrates just one potential configuration of a workspace containing repositories and collections. Notice that the repositories are all within one workspace, and that each repository is has numerous collections associated with it.

What Is a Workspaces provide portable, user-defined storage repositories. Workspace? They help you organize and separate your transformation and process design work by project. They can also be useful for packaging your processes and maps for transport to another machine with a different directory structure. This packaging simplifies migrating from development to test to production environments.

8-2 Introducing Workspaces, Repositories, and Collections

The workspace location is a directory that contains workspaces. The default workspace location on Windows XP is C:\Documents and Settings\username\Cosmos9_Work, where username is your logon name. On Windows 7 and 2008, the location is C:\Users\username\Cosmos9_Work. Each workspace is a subdirectory under the default workspace location Cosmos9_Work\Workspace1 and contains a macrodef.xml file and a repositories.xml file. The macrodef.xml file contains all macro definitions for the workspace (see “Macro Manager” on page 8-13), and repositories.xml defines the repositories for the workspace.

Workspace Scenario In a product development environment, workspaces are set up to reflect each major phase of the life cycle for the Widgets workspace location: Planning, Development, Testing, and Production. The resulting structure looks like this:

What Is a A repository is a user-defined area where transformation and process Repository? files are stored. Repositories reside in workspaces and contain collections of related files. An initial default repository is created in the default workspace under the first default workspace location, Cosmos9_Work\Workspace1\xmldb. Repositories, however, can be located anywhere you choose, on any local hard drive, shared network drives, or even on content management systems such as Visual SourceSafe or CVS.

Repository Scenario Using the same Development workspace described in “Workspace Scenario,” repositories are set up to contain each top-level project— Project Bluebonnet, Project Bootup, and Project Bronco. These

8-3 Repositories, Workspaces, and Macros

repositories serve as a holding place for project elements as they move through the lifecycle. The resulting structure looks like this:

What is a A collection is simply a subdirectory within a repository. It contains Collection? the next level of project elements. For example, a collection may contain files that are used to perform a specific task in the project.

Collection Scenario Finally, using the Project Bluebonnet repository structure described in “Repository Scenario,” collections are set up to contain each lower-level task—Customer Accounts, Branch Office Deliverables, and so forth. These collections serve as a holding place for task elements as they move through the lifecycle. The resulting structure looks like this:

8-4 Using Workspace Manager

Using Workspace Manager Integration designers with a graphical user interface can display Workspace Manager. It enables you to organize and separate your integration activities into workspaces by project or other criteria. Workspaces are also useful for packaging processes and maps to move to a machine with a different directory structure or from one environment to another, such as development to test to production. The workspaces.xml file contains the root workspace location and one entry for every workspace under this root. Each workspace subdirectory stores its own files:

A repositories.xml file that lists repositories for each workspace A macrodef.xml file for each workspace that defines its macros An xmldb subdirectory for .ss.xml and .map.xml files Java class files and RIFL files

Workspace Use File > Manage Workspaces to display the Workspace Manager Manager window with its two tabs:

Interface Workspaces: Add, modify, or delete a workspace on any drive where you have administrator privileges. For details, see “Managing Workspaces.” Repositories: Add, modify, or delete a repository in the workspace defined on the Workspaces tab. For details, see “Managing Repositories.”

Managing The Workspaces tab lists available workspaces. If you have added Workspaces none, it lists only Workspace1, which was created at installation. Within this section, the following tasks are documented:

To define the workspace root directory To rename workspaces To add workspaces To copy a workspace To remove a workspace

8-5 Repositories, Workspaces, and Macros

Defining the Workspace Root Directory Before defining any workspaces, you must define their location.

³ To define the workspace root directory 1 In any integration designer, select File > Manage Workspaces. 2 Click the arrow at the top of the window to open the Choose Workspace Root Location window. 3 Do one of the following:

To use an existing location, navigate to a folder and double-click it. In the field at the top of the root directory window, type the full path to an existing location. If the location does not yet exist, click Add Workspace Locations(s), type the folder name to use, and click OK. Type the full path name in the Workspaces Root Directory field at the top of the Choose Workspace Root Location window.

Tip To navigate to a network Workspaces Root Directory using the Drive list, the network drive must be mapped on your machine or it will not appear. If it is not mapped, you can still navigate to it. Just type the full path to the drive in the Workspaces Root Directory field and click the box below to reveal the expanded path tree. Select your destination Workspace Root Directory and click OK. To create a new directory from this window, click Add Workspace Location(s).

4 Click OK. The new workspace location appears in the Workspace Root Directory field at the top of the screen.

Note You can change your workspace root directory from the main Repository Explorer window or from any designer as a standalone process. If you access an application through Repository Explorer, that access is read-only. You can view the default directory but not make a change to it.

8-6 Using Workspace Manager

These steps should result in defining the workspaces to use a specific folder, creating the folder if it did not already exist, and creating the default Workspace1 folder. Using Windows Explorer to look at the path you defined in the workspace, you should see a folder with the workspace root directory name containing a secondary folder named Workspace1.

Renaming Workspaces At this point we can do any of the following:

Use the default Workspace1 Rename Workspace1 to something more appropriate Add new workspaces to meet our needs

³ To rename workspaces 1 To rename the default workspace to something more meaningful, on the Workspaces tab of the Workspace Manager screen, select Workspace1 in the grid. 2 Click Rename in the right panel. 3 Type the new name in the dialog box that displays and click OK. 4 When the Workspace dialog box opens, asking you to confirm the renaming task, click OK. The workspace name in the grid changes to your selected name. After renaming the default workspace, you can also see that name change reflected in Windows Explorer.

Adding Workspaces After using or renaming the default workspace, you may find you need additional workspaces to properly manage your work.

³ To add workspaces 1 To add a new workspace at the workspace location, click Add in the right panel of the Workspaces tab. The Add Workspace Location(s) window opens. 2 Click Create New Workspace , enter the name to use for the new workspace, and click OK.

8-7 Repositories, Workspaces, and Macros

3 Click OK again to save the new workspace which should now appear in the grid on the Workspaces tab.

Copying Workspaces You can copy workspaces to create a duplicate workspace that uses the same structure as another workspace, but under different names.

³ To copy a workspace 1 To copy a workspace in the same workspace root location on the Workspaces tab, select the workspace to be copied and click Copy in the right panel. 2 When the next window opens, type the name to use for the workspace copy. 3 Click OK to save the copied workspace which should now appear in the grid on the Workspaces tab. The copied workspace will include the same repository definitions as the original. To make them usable, you will need to modify the repository definitions as described in “Managing Repositories.”

Removing Workspaces After setting up your workspaces, you may discover that you have extraneous workspaces that are not being used. Note that removing a workspace through Workspace Manager does not delete the folder or contents on your operating system. You are simply removing the workspace definition from the integration platform, not the actual folders or files.

³ To remove a workspace 1 To remove a workspace from the workspace root location, select any workspace except the workspace to be removed and click Set Current on the right panel. 2 Click Remove in the right panel. 3 When the Remove Workspace(s) window opens, select the check box next to the workspace to be removed. 4 Click OK to remove the selected workspace and return to the Workspaces tab. The selected workspace should be gone from the grid.

8-8 Using Workspace Manager

Note Some workspace buttons are unavailable if you open Workspace Manager from an application launched from within Repository Explorer. For example, if you launch Map Designer from Repository Explorer, the Add, Set Current, Remove, Copy and Rename buttons are disabled. These buttons are available only from Repository Explorer or from the designers when they are run stand-alone.

Managing The Repositories tab provides a method of adding, modifying, or Repositories deleting repositories in a workspace. Within this section, the following tasks are documented:

To select a workspace for repository management To rename repositories To add repositories To remove a repository

³ To select a workspace for repository management 1 On the Workspaces tab, click the name of the workspace for which you want to define repositories. 2 Click Set Current in the right panel. The workspace row is highlighted.

3 Click Repositories on the right to display the repositories available for the selected workspace. You may have a single repository in each workspace, or you may have as many repositories as you deem necessary. The physical location for any given repository can be anywhere on the network. Defining a repository location is similar to setting up a shortcut on a Windows desktop.

8-9 Repositories, Workspaces, and Macros

With a new workspace, the only repository listed is the default repository named xmldb01. It is displayed in the dialog box that opens when you select the workspace on the Workspaces tab:

In this dialog box you can perform the same set of tasks as for workspaces, only for repositories. You can rename the default repository (xmldb01), you can add new repositories, or you can remove existing repositories.

Tip Unlike workspaces, if you want meaningful names to appear in Windows Explorer for your repositories, create or modify those folders in Windows Explorer before setting up repositories with Workspace Manager.

Renaming Repositories By default, a repository named xmldb01 is created whenever a new workspace is created. You can use that repository without any changes or you can rename it to something more meaningful if you prefer.

³ To rename repositories 1 To rename the default repository to something more meaningful, open Windows Explorer, locate the xmldb01 folder in the selected workspace folder, and rename it to your preferred name. 2 Back on the Select Repository screen, select xmldb01 in the grid. 3 Click Modify.

8-10 Using Workspace Manager

4 Type the new name in the Name field in the window that displays. 5 Click Find and navigate to the repository folder you renamed in step 1 and click OK to save that location. 6 Click OK to save the repository with the new name and location. When the Select Repository window reappears, the old repository name and location should have been replaced with the new ones:

Adding Repositories To add new repositories to a workspace, you must first create the folder in the operating system. After creating the repositories, you can use Workspace Manager to define the repositories in the selected workspace.

Tip You can also use CVS or Visual SourceSafe commands within Repository Explorer to manage the folders and files in your repositories. For details, see those topics in Repository Explorer User’s Guide.

³ To add repositories 1 To add a new repository in the workspace, click Add. The Add Repository window opens. 2 Type the repository name in the Name field in the window that displays. 3 Click Find and navigate to the location of the repository folder and click OK to save that location. 4 Click OK to save the new repository definition.

8-11 Repositories, Workspaces, and Macros

Again, looking at the Windows Explorer screen shows that the new workspace has been added:

Removing Repositories You can remove a repository definition from your workspace when it is no longer needed. Removal of the repository within Workspace Manager does not remove the actual folder or files. It only removes the repository definition from the workspace.

³ To remove a repository 1 On the Select Repository screen, select the name of the repository to be removed. 2 Click Remove, then click OK when prompted to confirm the removal of the repository.

Note If you only have one repository in the workspace, you can not remove the repository because each workspace must be associated with at least one repository.

8-12 Macro Manager

Macro Manager Macros use symbolic names for text strings, usually file paths. They are defined in Macro Manager and used in transformations. Assigning a meaningful macro name like TARGET to a file path, and using it when creating connections, saves time and reduces errors. However, the primary purpose of macros is to enable you to redefine the text string that a macro name refers to without the need to change your original transformation. You can redefine macros from the engine command line or create RIFL expressions that are triggered by events to redefine a macro dynamically during a transformation. Macros reside in your workspace in an XML file called macrodef.xml. Once you have created a macro, it can be used by any transformation or process that has access to the macrodef.xml file. Macros created in Map Designer can be used in Process Designer and vice versa.

Where Can You can use macros in the following locations in Map Designer: Macros Be Source connector information Used? Target connector information User name connection information Password connection information RIFL functions (MacroExpand, MacroValue, IsMacroDefined, and DefineMacro) Schema connection information DJImport object (MacroExpand and MacroValue) DJExport object (MacroExpand and MacroValue) QueryStatement Source View Designer Code modules (MacroValue, DefineMacro, and MacroExpand functions) QueryStatementFile parameter in command line override strings Create multiple macro definition files and specify them in the command line to configure your source and target dynamically

8-13 Repositories, Workspaces, and Macros

Where Should Macros cannot be used in the following locations: Macros Not Be Error logs Used? Multimode connector information Structured Schema connection information

Creating, ³ To create macros Editing and 1 From a designer Tools menu or the Repository Explorer Options Deleting menu, select Define Macros. The Macro Manager window Macros opens. 2 Click New. Complete the macro definition with the following parameters:

Macro Name—A unique name for the new macro. This name can be up to 198 characters in length and can include spaces. Macro Value—If you are assigning a macro to a file or directory, you can click the down arrow to navigate to the required directory. If you are using the macro to hold a text string, you can type it in this field. Description—(optional) Descriptive or explanatory notes about the macro. 3 Click OK to save the macro and then Close. After you have created a macro, you have three additional options in Macro Manager—Edit, Delete, and Set as Root.

³ To modify macros

Edit—In Macro Manager, select the macro you want to modify and click Edit. Modify the parameter values and click OK to save any changes. Delete—Select the macro and click Delete to remove it. Set as Root—Select the macro to use as the default macro when setting designer preferences and click Set as Root.

Using a Macro To use a macro, you can type it directly into the source and target in Connect fields, preceded by the $( characters and followed by a ) character. Information For example, if your macro is named TARGET, you will refer to it in a connection path as:

$(TARGET)

8-14 Macro Manager

An easier method is to position the cursor where you want the macro to be inserted, select Tools > Paste Macro String and click the required macro.

Note Some connectors do not expand macros. Their connections fail if macros are used in file paths.

Using a Macro When you use a macro to store a password, the password is entered to Store a in plain text and stored within the macro as plain text. In other Password words, it is not encrypted. For example, you can create a macro, name it PasswordMacro, and then within it store the string thisisapassword. Macros are stored and referenced in macrodef.xml. Within the macrodef.xml file, the value for the password is clearly visible.

Caution Security issues may arise from using this method.

For advanced tips on using macros, see the Best Practices Reference.

8-15 Repositories, Workspaces, and Macros

8-16 chapter Help and Support 9

Online Documentation and Technical Support

This chapter includes the following sections:

“Using the Online Product Documentation” on page 9-2 “Getting Help from Technical Support Engineering” on page 9-3

9-1 Help and Support

Using the Online Product Documentation The most current online documentation for the integration products is posted on the Internet. Your browser displays the following tabs and icons for finding topics in the documentation: Contents tab—Display a list of topics organized by category. A book icon next to a topic represents a category or group of related topics, and a page icon represents a single topic. Index tab—View topics organized by keywords. Select a letter from the list to see all keywords beginning with that letter. Select a keyword to display the topic for that keyword. Search tab—Search for a word or phrase. Type a string and click Go. Narrow the results by selecting a particular book. Click a result to display the topic. The search text is highlighted in the topic. To remove the highlights, right-click the topic and select Refresh. Favorites tab—Create a list of favorite topics. Display the topic, then click the Favorites tab. Click Add. To display added topics, click their names in the list. Show in Contents icon—Locate the currently displayed topic in the Contents. Previous icon—Move to the previous topic in the Contents. Next icon—Move to the next topic in the Contents. Print icon—Print the current topic.

9-2 Getting Help from Technical Support Engineering

Getting Help from Technical Support Engineering In addition to online help, technical support engineers can provide assistance. To report a problem, please provide the following information:

Product serial number (found by selecting About ProductName from the Help menu) Product version number (also found by selecting About ProductName from the Help menu) Your name, company name, e-mail address, and telephone number Operating system, version, and service pack Source and target names Exact wording of any error messages that appeared Exactly what you were doing when the problem or error occurred How you tried to solve the problem

Note Windows 2000/95/98/ME/NT are not supported in this release. No technical support is provided for these operating systems.

Troubleshooting Problems To help troubleshoot your data transformation problem, you may be asked for the following items.

Sample Data The sample may be a single file or it could be an entire database.

Note In the case of confidential data, Support may be able to use an altered subset or sign a nondisclosure agreement.

The Application Software If the application is a PC-based, nonserver database, is listed in the Source Connection or Target Connection menus, and is available

9-3 Help and Support

through general distribution, Technical Support owns or will purchase the application for troubleshooting. If the application is not PC-based, is not available through general distribution, or is a vertical market application, then Support may ask you to provide the application.

The System Configuration Your system configuration includes names and versions of your hardware platform or network and communication software and hardware.

9-4 chapter Solution Development Overview 10

Building Integration Solutions

This chapter describes how to get started with developing APIs, components, and interfaces to work with the integration platform. The topics in this chapter include:

“Introduction to Integration Software Development” on page 10-2 “Engine Overview” on page 10-3 “Command Line Execution” on page 10-5 “Introduction to the Development Toolset” on page 10-8 “Resources Needed to Construct a Solution” on page 10-10 “Application Development Basic Steps” on page 10-13

10-1 Solution Development Overview

Introduction to Integration Software Development If you are an IT specialist or developer who will be using the integration products in your development area to interface with your applications or to build solutions that include an integration component, then you have come to the right place. In this chapter we provide an introduction to the integration platform development environment. While it would be impossible to provide extensive, detailed information on creating and using applications and solutions in this introductory guide, we do provide product overviews and point you in the proper direction for more information.

Prerequisites Before going any further in this chapter, you do need to have at least a basic understanding of what the integration platform tools do, including their input requirements and output results. If you are already experienced with the integration platform functionality, please continue with “Engine Overview” on page 10-3. If you are new to the entire platform, please return to “Getting Started with the Integration Products” on page 1-1 and read about the tools and applications provided with this product set. Return to this chapter after you have read about the design tools and have examined a few sample transformations and processes.

10-2 Engine Overview

Engine Overview The Data Integrator engine is the centerpiece for integration execution and application development. It is an embeddable data transformation engine used to deploy run-time data replication, migration and transformation jobs on Windows or Unix-based platforms. The engine is written in C++. It contains the core data driver modules that are the foundation for the transformation architecture. Because the engine has no user interface components, it can automate run-time data transformations quickly and easily. This makes it ideal for environments where regular data transformations need to be scheduled and launched on Windows or Unix-based systems.

Why Use an You must use the graphical user interfaces in the interactive design Engine? tools to specify all the parameters that govern a particular source-to-target data transformation. When the design work is complete, a transformation XML file (.tf.xml) is created. The transformation file contains information about the sources, target, error logs, and run-time options. A map XML file (.map.xml), which contains the transformation rules, is also created. Using these two XML files, the entire set of transformation specifications can be transported to another platform for execution by the engine. This allows you to schedule the program execution through the command line. Or, if you prefer, you can also invoke the execution with a callable API. The callable API handling is available with the engine SDK.

How Does the Using Map Designer and Process Designer as front-end design tools, Engine Work you can fully exploit the power and ease-of-use that comes with a with the graphical, drag-and-drop interface. Invoking the engine as a separate Designers? transformation engine liberates the data transformation user to automate solutions on whatever platform makes the most sense. With the engine, you can deploy powerful and flexible run-time data transformations for almost all scenarios. With a separate, full-blown, execution engine to perform transformations, you can unleash a scalable, enterprise-wide tool to assist in all data transformation projects.

10-3 Solution Development Overview

Most transformations are designed using the Map Designer (the transformation tool) and Structured Schema Designer (the schema creation tool). These transformations can be combined with logic and other tasks in Process Designer. These user interfaces, along with the various utilities and an underlying engine, make up the integration product suite of tools.

Running the The engine can run on a Windows or Unix operating system. It can Engine run as a service on Windows using the Windows Services option. It can also run as a daemon on Unix. Using the engine, you have the option of processing multiple transformations or processes at the same time. You also have the option of batching work for scheduled processing using Cron on Unix and Cron-It! on Windows operating systems. All of those options are described in detail in Integration Engine User’s Guide. For instructions on installing the integration package that includes the engine, see Chapter 2, “Installing on Windows,” and Chapter 3, “Installing on Unix.” For instructions on using the engine to run transformations and processes, see Integration Engine User’s Guide.

10-4 Command Line Execution

Command Line Execution The engine can be invoked without a graphical interface through the command line of either a Windows or Unix operating system. The engine command line is a simple program that takes options and a transformation XML file (.tf.xml) or process XML file (.ip.xml) as input. The options are parsed and evaluated to determine the requested changes to the transformation or process. The engine command line utility loads the specified transformation or process (or creates a new one if the transformation XML file is not entered), makes the changes requested by any command line options, and runs the transformation or process. The main command line input is the name of the transformation or process XML file. The XML file contains all the stored parameters governing a specific end-to-end transformation or process. Overrides are available for both the source and target connection information in a transformation. This can range from simply pointing to a different flat file, to enumerating all the options necessary to connect to a SQL database.

Command Line Running a transformation or process is as simple as entering the Syntax djengine command followed by the full name of the transformation or process file.

Basic Command Execution To run a transformation from the command line, use the following syntax:

djengine transformation_file.tf.xml This runs the transformation using the settings exactly as they appear in the transformation_file.tf.xml transformation file. You can run a process in the same way:

djengine -pe process_file.ip.xml

Note The -pe option lets the engine know that it is about to run a process, not a transformation.

10-5 Solution Development Overview

Overriding Transformation or Process Options During Execution If you want to change or add settings, you can use global options to override the options defined in the transformation file. Global options are prefixed with a hyphen to distinguish them from file names. Enter the global options before the transformation or process file name on the command line as follows:

djengine [global options] [transformation/process specifications] So you lead with the djengine command, then enter any global options, and finally enter the name of the transformation or process file containing the specifications, as follows:

djengine -option1 –option2 abcd.tf.xml

Running Multiple Transformations or Processes You can run more than one transformation or process from a single command line. They are executed sequentially from left to right. Processes with options must be preceded by -pe. Options applying to each transformation or process must be specified immediately before that transformation or process. For example, the following command line tells the engine to run a transformation named transformation_1 and then a process named process_1:

djengine transformation_1.tf.xml –pe process_1.ip.xml In this example, no options were specified before the transformation_1 transformation file and the -pe specified after that file name indicates the next item (process_1) is a process. The following example illustrates what the command looks like if options are entered for two transformations and one process in a single, continuous command line entry:

djengine –option1 –option2 transformation_1.tf.xml –pe -option3 process_1.ip.xml -option4 transformation_2.tf.xml Items -option1 and -option2 apply to transformation_1, -option3 applies to process_1, and -option4 applies to transformation_2.

Using Complex Options in Specification Files If you want to repeatedly run transformations or processes from the command line and they require complex option entry, you can save

10-6 Command Line Execution yourself some time and reduce errors by creating and running specification files containing those commands. A specification file is an ASCII text file containing as many global options as required, plus the transformation or process file names. In earlier product versions, specification files were known as optfiles. The following example illustrates the contents of a sample specification file named sample_trans_spec_file:

-target_connect_info c:\apps\data\datafile.dbf -source_sample_logic type=1;start=2;count=100 -max_errors 1 -logfile C:...\Logs\MapDesigner\tf1error.log -target_filter_expr C:...\filter.txt

-reject_connect_info "File=d:\\dj5libs\\tutor1.rej;RecordSeparator=CR-LF; RecordFieldCount=0;FieldSeparator=,; FieldStartDelimiter=’"’;FieldEndDelimiter=’"’; Header=False;AlternateFieldSeparator=None; StartOffset=0;AutomaticStyling=False; StyleSampleSize=1000;StripLeadingBlanks=True; StripTrailingBlanks=True"

InstallDir\Common\tf1.tf.xml To run a specification file, you must prefix the file path and name with an at symbol (@), as follows: djengine @sample_trans_spec_file If duplicate options are found in a specification, the last option encountered is used.

Note The -Source_Filter_Expr and -Target_Filter_Expr options are the only exception to this handling, where each additional filter is added to the transformation.

Additional global options can be used with a specification file. Global options must appear before the name of the specification file in the following format: djengine -option1 -option2 @specification_file See Integration Engine User’s Guide for information on using the command line with the engine.

10-7 Solution Development Overview

Introduction to the Development Toolset As indicated in “Engine Overview” on page 10-3, the engine is the centerpiece of the development environment. The ability to access the engine functionality from a command line or API lays the groundwork for you to develop and use your own applications or interfaces with the integration platform. Three main SDKs are currently distributed as part of the integration platform. All three SDKs are installed as part of the integration platform and are found in the InstallDir\Common\SDKs folder.

Engine Engine The engine SDK provides a method of overriding scheduled SDK transformation and process handling. Applications built with the engine interfaces still use the same .tf.xml, .map.xml, and .ip.xml files output by Map Designer and Process Designer. The SDK simply provides the ability for you to use your own applications to interact with those files. Engine SDK users must be familiar with writing code using a Java Development Kit (JDK) or a COM environment supported by Visual Basic, VB Script, Java Script, Perl, or Delphi. C# and VB .Net are currently supported with a .Net interoperability layer. Users should be experienced with defining parameters for data transformations; setting the source and target locations; accessing SQL sessions; setting macros, message objects, and variables; using a repository; and storing code in a source control system. For detailed information on using the engine SDK, see the Software Development Kit Getting Started Guide.

Content The Content Extraction Language is an AWK-like line-oriented Extraction programming language. Its purpose is to recognize and extract Language structured fields of data from specific lines of incoming text files, and (CXL) SDK then assemble those fields into a flat record of data that it passes on to a subsequent process (such as Map Designer). To do this, CXL provides a wide variety of pattern recognition, computing, and data manipulation capabilities. This robust and expressive scripting language gives users a valuable tool to tackle the problem of extracting useful field and record structures from the huge and evergrowing volume of complex text files.

10-8 Introduction to the Development Toolset

For information on using CXL, see the Extract Schema Designer Content Extraction Language (CXL) Programmer’s Reference Manual.

Other Tools In addition to the SDKs that are strictly developer tools, the integration platform includes:

Connectors The set of source and target connectors that recognize the format of existing source data and convert it to a format acceptable for the target file. See Source and Target Connectors User’s Guide for descriptions of available connectors. Components These tools perform specific process tasks in Process Designer to further extend that tool functionality. These components are classified as aggregators, invokers, iterators, queues, and transformers. Each component has its own user’s guide. For an overview of components, see Integration Components Getting Started Guide. Integration Server Integration Server is an SDK installed by default when the integration platform is installed. The core components of the Integration Server SDK are the Engine Controller, Engine Instances (Managed Pool), and the Client API that accesses the Engine Controller through a proxy. Server stability is maintained, scalability enhanced, and resources are spared through the use of a control-managed pool of EngineExe objects. For instructions on installing, configuring, and using Integration Server, see Integration Server and SDK Guide.

10-9 Solution Development Overview

Resources Needed to Construct a Solution Solutions are a fairly modern concept in software development. A single program or tool no longer meets many requirements. Instead, various programs and interfaces are grouped together to address complex business needs, often spanning departments or even organizations. The integration platform is a set of products that can play a role in solution development, with its extensive number of connections to data types, databases, and interfaces.

Requirements There are as many potential solution scenarios as there are unique requirements. Here is a very basic one. The Brandolf Company is a fictional multi regional real estate investment and property management corporation. They have regional offices throughout the United States and each regional office supports multiple local offices that have been acquired through corporate acquisitions, mergers, and new office establishment. Each local office must provide weekly reports to the regional office outlining their real estate holdings, payroll processing, and general financial records. Each regional office must compile the information received from the local offices and generate monthly reports to headquarters. At headquarters, the regional data will be merged to produce monthly, quarterly, and annual reports to management, government agencies, and stockholders. The complexity occurs because these entities have all evolved independently in most cases, developing their own record keeping and reporting methods and tools. One local office tracks their data in simple spreadsheets while another office uses an MS Access database. One regional office has a fairly sophisticated IT organization so they maintain their records in an Oracle database. Another regional office uses a third-party bookkeeping program.

The Solution The long-term plan of the company is to select software that most closely meets their record keeping and reporting requirements, install that software in the appropriate locations, train the employees to use it, and convert all records and reporting to that solution. However, there are two roadblocks to this plan.

10-10 Resources Needed to Construct a Solution

First, with a large corporation where resources are widespread both geographically and technically, conversion of a major part of the business is both time consuming and expensive. It must be done without interfering significantly with day-to-day business activities. Secondly, even when a corporate software solution is selected, that solution is rarely represented by a single product or tool. In most cases it is a combination of software that has been grouped together with interfaces developed to make the components work together to produce the desired results. That is where the software development work comes in for most organizations.

The Integration Interestingly enough, the integration platform falls into both options Platform of the above solution scenario. When a major software conversion Solution project is undertaken, a large element of the project is the conversion of data from various sources into the desired target data type. This may be a one-time process where the resulting target is used for all future data processing. Or it may be a ongoing conversion task that is performed because no single tool meets all the requirements. One tool may be used for record keeping but may not meet the needs for reporting. Another tool may generate great reports, but does not meet the record keeping requirements. Using the two tools may be the answer, but the solution is not complete until you can move the data between the two programs. That is where the integration platform comes in. Going back to our original scenario, the fictional Brandolf Company may create their solution as follows: 1 The source comes from various data types, such as ASCII, Excel spreadsheets, or databases. 2 Map Designer transforms the data from the source to a target data type acceptable for the corporate record keeping and reporting. 3 Process Designer executes the transformation specification, runs internal tools to perform other data manipulation, passes the target data to another program, and produces the final output in report format using the reporting tools. 4 The process is run in batch mode on the engine to complete the process. Using the above steps, you can see that various resources come into play to build and use this solution.

10-11 Solution Development Overview

In step 1, the source comes from numerous existing programs in different data formats. In step 2, Map Designer transforms the data from the various data types to the target data type needed for processing by other programs. In step 3, Process Designer pulls the tools together to work as a single process. These tools include the ones in the integration platform, other off-the-shelf products, and tools developed by the user and third-party software developers. In step 4, the solution is used to produce the desired results.

Integration At any of these stages you or your vendor may develop software to Platform interact with one or more parts. Using the integration platform Programming SDKs, you can develop the APIs to complete your solution. The Resources following resources will assist you in that development: API Function Calls Users can launch transformation processes directly from a calling program or application. Java API Java classes that are supported by the engine SDK. COM API COM compounds that are supported by the engine SDK. Programming Examples Code examples for both Java and COM are provided to illustrate the interfaces. You can find out more about these resources in the Software Development Kit Getting Started Guide. This guide describes the use of Java and COM for developing applications.

10-12 Application Development Basic Steps

Application Development Basic Steps The specific steps performed in application development vary widely based on the programming language used, the complexity of the application, the interfaces being supported, and many other factors. With the use of Map Designer or Process Designer to take care of the main integration activity with the engine, we can identify the following as the main steps in developing an application in this environment: 1 Create the transformation with Map Designer or a process with Process Designer. 2 Write the additional code. 3 Compile the new code. 4 Run the application. Obviously that is a very simplified development process. To provide a little better idea of the steps involved in creating an integration application in Java or COM for use with the engine, we outline those steps below.

Creating a Java The following is a very simple Java application that starts the engine, Application and then loads and runs a transformation or process. 1 Create the “main” class to begin the program. 2 Create the engine object because you are interfacing with the engine. 3 Set the initialization file to start the engine and check the license. 4 Create and load the transformation or process to be run. 5 Define run-time overrides for the transformation or process if they will be used. This is similar to the way they are overridden when being issued from the engine command line. 6 Compile and run the application.

Creating a COM The following is the same basic process for creating a COM Application application to perform the same tasks as the Java application. 1 Specify that you are creating a form or module.

10-13 Solution Development Overview

2 Create the engine object so the application can interface with the engine, then explicitly name the members. 3 Set the initialization file so the program can start the engine. 4 Load the transformation or process specification file. 5 Define any run-time overrides for the transformation or process if they will be used. 6 Compile and run the application.

The Next Step To take the next step in developing your integration application, continue reviewing the development process as described in the Software Development Kit Getting Started Guide.

10-14 appendix Solution Development Reference A

More Information on Developing Solutions

This appendix provides additional information to help you get started in developing solutions for use with the integration platform. It includes an introduction to the following resources:

“Debugging Processes and Troubleshooting Problems” on page A-2 “Solution Development Checklist” on page A-3 “Product and Feature Limitations” on page A-4

1 Solution Development Reference

Debugging Processes and Troubleshooting Problems The integration platform provides the Error Codes and Messages Reference Manual as your first step in isolating and resolving errors. This document is available in online help as well as in PDF format.

Testing for Errors When developing your own applications, you need to ensure that the engine is able to anticipate problems that might arise. You must add that error checking in the application.

Testing in Java Applications A try...catch block should be written to catch any errors that occur in your Java application while creating the engine, as in the following sample code block: try{ e.setInitializationFile( "cosmos.ini" ); } //catch any errors creating the Engine catch (ECException ec){ System.out.println("Error creating Engine");

}

Testing in COM Applications To ensure that the engine is able to anticipate problems that might arise, the COM application should be written to trap any errors that occur while the program executes.

On Error GoTo foo: foo: Msgbox ("Error : " & vbCrLf & _ "Error Number: " & Err.Number & vbCrLf & _ "Error Description: " & Err.Description) Eng Sub Also remember to include tests in the applications for possible problems when they execute.

2 Solution Development Checklist

Solution Development Checklist This checklist can help you verify application development details.

Description Done?

The integration platform has been installed and the license is current. If your license files does not fully enable the engine SDK, attempts to use it may return error code 25546.

You have a user account on any databases that you plan to access with your application.

You have create, read, update, and delete rights to any table or tablespace that you plan to access with your application.

You have a transformation file (.tf.xml) created in Map Designer.

You have a map file (.map.xml) created in Map Designer.

You have a process file (.ip.xml) created in Process Designer.

Your development environment has been installed and configured.

If you are creating a COM application, you have imported by reference the COM objects into your COM editor.

Your system path environment variable is set to the directory in which DJEngine.exe is stored.

Your initialization file is pointing to the InstallDir\Common\cosmos.ini file.

The paths to your source and target locations are entered correctly.

You have access to the subdirectories in which your source and target data is stored or is going to be stored.

You loaded the transformation specification file prior to applying the overrides.

Your code is set up to catch exceptions and trap errors.

Your macro definition is pointing to the source file.

If you are creating a Java application, you applied the standard Java double back slashes to the paths added in the code.

If you are creating a Java application, the appropriate breaks and brackets have been applied to each process.

If you are creating a Java application, each line of code ends with a semicolon.

Single and double quotes have been applied appropriately.

You compiled the application before you tried to run it.

3 Solution Development Reference

Product and Feature Limitations You may need to know the limitations of a particular product option, feature, field, or file when you are designing a transformation, process, or schema. This can be an important consideration if you have an especially large file, or want to include a long description in a field. The following sections list certain limitations by product name.

“Map Designer” on page A-4 “Process Designer” on page A-5 “Repository Explorer” on page A-7 “Workspace Manager” on page A-7 “Structured Schema Designer” on page A-7 “RIFL” on page A-8

Map Designer The following table gives maximum recommended settings in Map Designer. Most of these depend either on operating system memory and resources or Visual Basic limitations.

Name Maximum Recommended Notes

Source fields on Map tab 32,000 Visual Basic limitation

Target fields on Map tab 32,000 Visual Basic limitation

File size, source file 4,000+ petabytes Limited by operating system memory

Event actions for a single event 32,000 Limited by operating system memory

Source field name, number of Minimum 256; maximum 1024 Limited by operating system bytes on Map tab memory

Source field description on Map tab 32,000 bytes

Target field description on Map tab 32,000 bytes

Target field name, number of bytes Minimum 256; maximum 1024 Limited by operating system on Map tab memory

Target file length Limited by operating system memory

Target field expression 32,000 bytes Visual Basic limitation

4 Product and Feature Limitations

Name Maximum Recommended Notes

Source record types that can be set 64,000 on Map tab

Target record types that can be set 64,000 on Map tab

Source record type rules that can 32,000 Visual Basic limitation be set on Map tab

Source record validation 32,000 bytes Visual Basic limitation expression

Variables that can be declared 32,000 Operating system memory or Visual Basic limitation

Maximum number of dimensions 3 for variable array

Array size for variable 2,147,483,647 elements Limited by operating system memory

Maximum row count, source or 2,147,483,647 Limited by operating system target memory

Variable declaration screen initial 2,048 characters value field

Variable declaration screen 32,000 bytes description field

Process Designer The following table gives maximum recommended settings in Process Designer. Most of these depend either on operating system memory and resources or Visual Basic limitations.

Name Maximum Recommended Notes

Steps in a single process Variable Limited only by operating system memory. However, if a large number of steps are added to a single process, UI performance can decline. In this case, use sub- processes to divide up the original process.

Number of process variables that 32,000 Limited by operating system can be declared memory

5 Solution Development Reference

Name Maximum Recommended Notes

Maximum number of dimensions 3 for variable array

Array size for variable 2,147,483,647 elements Limited by operating system memory

Variable declaration screen initial 2,048 characters value field

Variable declaration screen 32,000 bytes description field

Process properties tab description 30,000 bytes field

Process step description field 30,000 bytes

SQL Session name 30,000 bytes

SQL Step (SQL Statement field) 30,000 bytes

Decision Step expression field 30,000 bytes

Application Step command line 30,000 bytes field

Application Step Start In field (path 30,000 bytes name length)

Microsoft DTS Step 30,000 bytes Limitation for the following step options: server field, database field, user id field, file field, package name, package id field, and package version id field

Microsoft DTS Step 64 characters Password field and package password field

Subprocess Step process field 30,000 bytes

Document Validator Step 30,000 bytes document field

Script Step expression field 30,000 bytes

XSLT Step source field, target field, 30,000 bytes and stylesheet field

Queue Step description field 30,000 bytes

6 Product and Feature Limitations

Name Maximum Recommended Notes

Queue Session name 30,000 bytes

Annotation text Unknown

Repository Explorer The following table gives maximum recommended settings in Repository Explorer. Most of these depend either on operating system memory and resources or Visual Basic limitations.

Name Maximum Recommended Notes

Number of tools that can be Varies Limited by operating system launched from Repository Explorer memory. Total number of elements at one time must be less than 2,147,483,647, but having more than a few open at the same time is not recommended.

Number of .xml design files that 32,760 can be displayed in Repository Explorer grid

Workspace Manager The following table gives maximum recommended settings in Workspace Manager. Most of these depend either on operating system memory and resources or Visual Basic limitations.

Name Maximum Recommended Notes

Maximum number of workspaces Varies Limited only by operating system and repositories memory

Structured Schema Designer The following table gives maximum recommended settings in Structured Schema Designer. Most of these depend either on operating system memory and resources or Visual Basic limitations.

Name Maximum Recommended Notes

Record types 64,000

Recognition rules 32,000

7 Solution Development Reference

Name Maximum Recommended Notes

Record validation expression 32,000 bytes

Source file or URI name 2,048 bytes

Field description 32,000 bytes

Field name Minimum 256 bytes, maximum Limit varies by operating system 1,024 bytes

Schema properties description 32,000 bytes field

Data Parser, number of records Unlimited

Data Parser field name Minimum 256 bytes, maximum Limit varies by operating system 1,024 bytes

RIFL The following table gives maximum recommended settings for RIFL scripts and code modules. Most of these depend either on operating system memory and resources or Visual Basic limitations.

Name Maximum Recommended Notes

RIFL script for Execute action 32,000 bytes Visual Basic limitation

External RIFL code modules that 32,000 Visual Basic limitation can be declared for use in a single map

Code module size 32,000 bytes Visual Basic limitation; 64,000 limit for profiling. Also limited by operating system memory.

8 Index

Symbols COM API 10-12 .chm defined 1-6 application development 10-13 .cxl defined 1-4 testing applications A-2 .djar defined 1-6 command line .ds.xml defined 1-5 execution 10-5 .ip.xml defined 1-4 syntax 10-5 .jar defined 1-6 Common folder .join.xml defined 1-6 location 2-7 .map.xml defined 1-4 complex options in specification files 10-6 .mdb defined 1-5 components 10-9 .pdf defined 1-6 connectors 10-9 .rifl defined 1-5 Content Extraction Language (CXL) .sc.xml defined 1-4 SDK 10-8 .slc defined 1-5 cosmos.ini .ss.xml defined 1-5 internal 1-13 .tar defined 1-6 preferences 1-13 .tc.xml defined 1-4 ProductInfo 1-13 .tf.xml defined 1-4 settings 1-10 .tpl defined 1-6 UserInfo 1-10 .xmldb defined 1-5 CXL (Content Extraction Language) 10-8 Numerics D 25543 error 1-11 debugging processes A-2 default A FTP mode 3-18 antivirus software Design Studio exiting before installing 2-3 installation 2-5 APIs developing software 10-2, 10-10 COM 10-12 djengine function call 10-12 in side-by-side installation 2-27 Java 10-12 djengine command example 10-5 application development Document Schema Designer COM 10-13 opening for first time 2-18 Java 10-13 documentation application shutdown 2-3 overview 1-7 C E case distinction 3-18 engine checklist, developing solutions A-3 overview and uses 10-3 collections 8-4 errors, initialization 2-28

Index-1 examples M programming 10-12 Macro Manager 8-13 macros F connect information 8-14 file extensions 1-4 defining 8-13 FTP deleting 8-13, 8-14 before running on Unix 3-16 dynamically defining 8-14 default mode 3-18 editing 8-14 running 3-16 for storing passwords 8-15 using 3-16 using 8-13 multiple transformations, running 10-6 H multiple versions on same computer 2-3 help, online 9-2 O I online help 1-7 initialization errors 2-28 overriding installation process options 10-6 9.x side-by-side with 8.x 2-20 transformation options 10-6 directory, alternate 2-7 overview engine only 2-5 engine 10-3 files, downloaded 2-5 solution development 10-1 multiuser license machine 3-5 troubleshooting 2-26 P Unix 3-5 password Unix, before installation 3-3 loss of when upgrading from 7.x 2-24 verifying Unix 3-9 permissions, Unix 3-3 Windows 2-5 pipes Windows, before installation 2-3 anonymous 1-14 Integration Server SDK 10-9 named 1-14 streaming data 1-14 J processes Java debugging A-2 API 10-12 overriding 10-6 application development 10-13 running multiple 10-6 testing applications A-2 programming examples 10-12 L R license file reinstall instructions 2-29 Document Schema Designer 2-18 repositories 1-3 Repository Manager 2-18 managing 8-9 license file, Unix 3-3 Repository Manager License Manager opening for first time 2-18 on Unix 3-6 respositories on Windows 2-17 definition 8-3 local administrator logon for Windows 2-3 root directory, workspace 8-6

Index-2 running loss of passwords 2-24 multiple processes 10-6 from 7.x to 8.x 2-22 installing 9.x side-by-side with 8.x 2-20 S SDKs W Content Extraction Language (CXL) 10-8 Windows installation, Unix 3-5 installation 2-5 Integration Server 10-9 Workspace Manager 8-5 side-by-side installation interface 8-5 running djengine 2-27 root directory 8-6 slibclean 4-2, 4-5 Workspaces tab 8-6 software development 10-2, 10-10 workspace.xml software development, prerequisites 10-2 defined 1-5 solution development workspaces checklist A-3 definition 8-2 reference A-1 managing 8-5 specification files, complex options 10-6 overview 8-2 specification_file defined 1-5 streaming data through pipes 1-14 system requirements 2-2 system requirements, minimum for Unix 3-2 T target files default output location 2-7 testing COM applications A-2 Java applications A-2 transformations overriding 10-6 sharing 2-25 troubleshooting installation 2-26 problems A-2 U uninstall instructions 2-29 Unix installation instructions 3-5 moving files from Windows to Unix 3-13 verifying installation 3-9 Upgrade Utility 2-23 upgrading from 7.x

Index-3 Index-4

Deltek is the leading global provider of enterprise software and information solutions for professional services firms, government contractors, and government agencies. For decades, we have delivered actionable insight that empowers our customers to unlock their business potential. Over 14,000 organizations and 1.8 million users in approximately 80 countries around the world rely on Deltek to research and identify opportunities, win new business, optimize resource, streamline operations, and deliver more profitable projects. Deltek – Know more. Do more.® deltek.com