DOCSLIB.ORG

Talend ESB Infrastructure Services Configuration Guide

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Talend ESB Infrastructure Services Configuration Guide Talend ESB Infrastructure Services Configuration Guide 6.3.2 Talend ESB Infrastructure Services Adapted for v6.3.2. Supersedes previous releases. Publication date: September 7, 2017 Copyright © 2017 Talend Inc. All rights reserved. Copyleft This documentation is provided under the terms of the Creative Commons Public License (CCPL). For more information about what you can and cannot do with this documentation in accordance with the CCPL, please read: http://creativecommons.org/licenses/by-nc-sa/2.0/ This document may include documentation produced at The Apache Software Foundation which is licensed under The Apache License 2.0. Notices Talend and Talend ESB are trademarks of Talend, Inc. Apache CXF, CXF, Apache Karaf, Karaf, Apache Cellar, Cellar, Apache Camel, Camel, Apache Maven, Maven, Apache Archiva, Archiva, Apache Syncope, Syncope, Apache ActiveMQ, ActiveMQ, Apache Log4j, Log4j, Apache Felix, Felix, Apache ServiceMix, ServiceMix, Apache Ant, Ant, Apache Derby, Derby, Apache Tomcat, Tomcat, Apache ZooKeeper, ZooKeeper, Apache Jackrabbit, Jackrabbit, Apache Santuario, Santuario, Apache DS, DS, Apache Avro, Avro, Apache Abdera, Abdera, Apache Chemistry, Chemistry, Apache CouchDB, CouchDB, Apache Kafka, Kafka, Apache Lucene, Lucene, Apache MINA, MINA, Apache Velocity, Velocity, Apache FOP, FOP, Apache HBase, HBase, Apache Hadoop, Hadoop, Apache Shiro, Shiro, Apache Axiom, Axiom, Apache Neethi, Neethi, Apache WSS4J, WSS4J are trademarks of The Apache Foundation. Eclipse Equinox is a trademark of the Eclipse Foundation, Inc. SoapUI is a trademark of SmartBear Software. Hyperic is a trademark of VMware, Inc. Nagios is a trademark of Nagios Enterprises, LLC. All other brands, product names, company names, trademarks and service marks are the properties of their respective owners. This product includes software developed at AOP Alliance (Java/J2EE AOP standards), ASM, AntlR, Apache ActiveMQ, Apache Ant, Apache Avro, Apache Axiom, Apache Axis, Apache Axis 2, Apache Batik, Apache CXF, Apache Camel, Apache Chemistry, Apache Common Http Client, Apache Common Http Core, Apache Commons, Apache Commons Bcel, Apache Commons JxPath, Apache Commons Lang, Apache Derby Database Engine and Embedded JDBC Driver, Apache Geronimo, Apache Hadoop, Apache Hive, Apache HttpClient, Apache HttpComponents Client, Apache JAMES, Apache Log4j, Apache Lucene Core, Apache Neethi, Apache POI, Apache Pig, Apache Qpid-Jms, Apache Tomcat, Apache Velocity, Apache WSS4J, Apache WebServices Common Utilities, Apache Xml-RPC, Apache Zookeeper, Box Java SDK (V2), CSV Tools, DataStax Java Driver for Apache Cassandra, Ehcache, Ezmorph, Ganymed SSH-2 for Java, Google APIs Client Library for Java, Google Gson, Groovy, Guava: Google Core Libraries for Java, H2 Embedded Database and JDBC Driver, HsqlDB, Ini4j, JClouds, JLine, JSON, JSR 305: Annotations for Software Defect Detection in Java, JUnit, Jackson Java JSON-processor, Java API for RESTful Services, Jaxb, Jaxen, Jettison, Jetty, Joda-Time, Json Simple, MetaStuff, Mondrian, OpenSAML, Paraccel JDBC Driver, PostgreSQL JDBC Driver, Resty: A simple HTTP REST client for Java, Rocoto, SL4J: Simple Logging Facade for Java, SQLite JDBC Driver, Simple API for CSS, SshJ, StAX API, StAXON - JSON via StAX, Talend Camel Dependencies (Talend), The Castor Project, The Legion of the Bouncy Castle, W3C, Woden, Woodstox : High-performance XML processor, XML Pull Parser (XPP), Xalan- J, Xerces2, XmlBeans, XmlSchema Core, Xmlsec - Apache Santuario, Zip4J, atinject, dropbox-sdk-java: Java library for the Dropbox Core API, google-guice. Licensed under their respective license. Table of Contents Chapter 1. Introduction ................................................................................................ 1 1.1. Prerequisites to using Talend ESB products .................................................................... 3 Chapter 2. Service Locator ........................................................................................... 5 2.1. Installing Service Locator ........................................................................................ 6 2.1.1. Standard installation ...................................................................................... 6 2.1.2. Alternative installation .................................................................................. 22 2.1.3. (Auto-)Unregister non ESB Provider via an endpoint time-to-live mechanism ........................ 25 2.1.4. Additional Metadata ..................................................................................... 26 2.1.5. Service Locator endpoint selection strategy configuration ............................................. 27 Chapter 3. Service Activity Monitoring ........................................................................ 29 3.1. Technical overview ............................................................................................... 30 3.1.1. Architecture diagram .................................................................................... 30 3.2. Installing the Service Activity Monitoring ..................................................................... 31 3.2.1. DataSource installation .................................................................................. 32 3.2.2. Standard installation ..................................................................................... 35 3.2.3. Alternative installation .................................................................................. 43 3.3. Typical use cases ................................................................................................. 47 3.3.1. Building examples ....................................................................................... 48 3.3.2. Pre-requisites ............................................................................................ 48 3.3.3. General Test ............................................................................................. 48 3.3.4. Filters and Handlers Test ............................................................................... 49 3.3.5. Monitoring events from database ....................................................................... 50 Chapter 4. Event Logging ........................................................................................... 51 4.1. Overview .......................................................................................................... 52 4.2. Installing the Event Logging in the Talend Runtime container .............................................. 53 4.2.1. Default start/stop ........................................................................................ 53 4.2.2. Manual start/stop ........................................................................................ 54 4.3. Event Logging - Listeners ....................................................................................... 55 4.3.1. Log Listener ............................................................................................. 56 4.3.2. OSGi Event Listener .................................................................................... 60 4.3.3. SAM Listener ............................................................................................ 62 4.3.4. Locator Listener ......................................................................................... 64 4.4. Event Logging - Agent ........................................................................................... 64 4.4.1. Agent - Receiver part ................................................................................... 65 4.4.2. Agent - Processing Part ................................................................................. 67 4.5. Event Logging - Sender ......................................................................................... 69 4.5.1. REST Sender ............................................................................................ 69 4.5.2. JMS Sender .............................................................................................. 70 4.6. Event Logging - Server .......................................................................................... 71 4.6.1. Collector ................................................................................................. 71 4.6.2. Server Pre-processing part .............................................................................. 72 4.6.3. Server persistence and post-processing ................................................................. 73 4.7. Event Logging - Service ......................................................................................... 76 4.8. Talend Log Server ............................................................................................... 76 4.8.1. Configuring the EventLogging server to connect to a secured Elasticsearch ........................... 77 4.9. Logging page in Talend Administration Center ............................................................... 78 4.10. Robust event processing ........................................................................................ 78 4.11. Event Logging - API's and Data Structures .................................................................. 80 4.11.1. Event Data - Structure ................................................................................. 80 4.11.2. Event Database Structure .............................................................................. 82 4.11.3. Event Logging Service API ..........................................................................
Load more

Recommended publications
  • Getting Started with Derby Version 10.14
    Getting Started with Derby Version 10.14 Derby Document build: April 6, 2018, 6:13:12 PM (PDT) Version 10.14 Getting Started with Derby Contents Copyright................................................................................................................................3 License................................................................................................................................... 4 Introduction to Derby........................................................................................................... 8 Deployment options...................................................................................................8 System requirements.................................................................................................8 Product documentation for Derby........................................................................... 9 Installing and configuring Derby.......................................................................................10 Installing Derby........................................................................................................ 10 Setting up your environment..................................................................................10 Choosing a method to run the Derby tools and startup utilities...........................11 Setting the environment variables.......................................................................12 Syntax for the derbyrun.jar file............................................................................13
    [Show full text]
  • Oracle Metadata Management V12.2.1.3.0 New Features Overview
    An Oracle White Paper October 12 th , 2018 Oracle Metadata Management v12.2.1.3.0 New Features Overview Oracle Metadata Management version 12.2.1.3.0 – October 12 th , 2018 New Features Overview Disclaimer This document is for informational purposes. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described in this document remains at the sole discretion of Oracle. This document in any form, software or printed matter, contains proprietary information that is the exclusive property of Oracle. This document and information contained herein may not be disclosed, copied, reproduced, or distributed to anyone outside Oracle without prior written consent of Oracle. This document is not part of your license agreement nor can it be incorporated into any contractual agreement with Oracle or its subsidiaries or affiliates. 1 Oracle Metadata Management version 12.2.1.3.0 – October 12 th , 2018 New Features Overview Table of Contents Executive Overview ............................................................................ 3 Oracle Metadata Management 12.2.1.3.0 .......................................... 4 METADATA MANAGER VS METADATA EXPLORER UI .............. 4 METADATA HOME PAGES ........................................................... 5 METADATA QUICK ACCESS ........................................................ 6 METADATA REPORTING .............................................................
    [Show full text]
  • Operational Database Offload
    Operational Database Offload Partner Brief Facing increased data growth and cost pressures, scale‐out technology has become very popular as more businesses become frustrated with their costly “Our partnership with Hortonworks is able to scale‐up RDBMSs. With Hadoop emerging as the de facto scale‐out file system, a deliver to our clients 5‐10x faster performance Hadoop RDBMS is a natural choice to replace traditional relational databases and over 75% reduction in TCO over traditional scale‐up databases. With Splice like Oracle and IBM DB2, which struggle with cost or scaling issues. Machine’s SQL‐based transactional processing Designed to meet the needs of real‐time, data‐driven businesses, Splice engine, our clients are able to migrate their legacy database applications without Machine is the only Hadoop RDBMS. Splice Machine offers an ANSI‐SQL application rewrites” database with support for ACID transactions on the distributed computing Monte Zweben infrastructure of Hadoop. Like Oracle and MySQL, it is an operational database Chief Executive Office that can handle operational (OLTP) or analytical (OLAP) workloads, while scaling Splice Machine out cost‐effectively from terabytes to petabytes on inexpensive commodity servers. Splice Machine, a technology partner with Hortonworks, chose HBase and Hadoop as its scale‐out architecture because of their proven auto‐sharding, replication, and failover technology. This partnership now allows businesses the best of all worlds: a standard SQL database, the proven scale‐out of Hadoop, and the ability to leverage current staff, operations, and applications without specialized hardware or significant application modifications. What Business Challenges are Solved? Leverage Existing SQL Tools Cost Effective Scaling Real‐Time Updates Leveraging the proven SQL processing of Splice Machine leverages the proven Splice Machine provides full ACID Apache Derby, Splice Machine is a true ANSI auto‐sharding of HBase to scale with transactions across rows and tables by using SQL database on Hadoop.
    [Show full text]
  • Apache Log4j 2 V
    ...................................................................................................................................... Apache Log4j 2 v. 2.4.1 User's Guide ...................................................................................................................................... The Apache Software Foundation 2015-10-08 T a b l e o f C o n t e n t s i Table of Contents ....................................................................................................................................... 1. Table of Contents . i 2. Introduction . 1 3. Architecture . 3 4. Log4j 1.x Migration . 10 5. API . 16 6. Configuration . 19 7. Web Applications and JSPs . 50 8. Plugins . 58 9. Lookups . 62 10. Appenders . 70 11. Layouts . 128 12. Filters . 154 13. Async Loggers . 167 14. JMX . 181 15. Logging Separation . 188 16. Extending Log4j . 190 17. Programmatic Log4j Configuration . 198 18. Custom Log Levels . 204 © 2 0 1 5 , T h e A p a c h e S o f t w a r e F o u n d a t i o n • A L L R I G H T S R E S E R V E D . T a b l e o f C o n t e n t s ii © 2 0 1 5 , T h e A p a c h e S o f t w a r e F o u n d a t i o n • A L L R I G H T S R E S E R V E D . 1 I n t r o d u c t i o n 1 1 Introduction ....................................................................................................................................... 1.1 Welcome to Log4j 2! 1.1.1 Introduction Almost every large application includes its own logging or tracing API. In conformance with this rule, the E.U.
    [Show full text]
  • Tuning Derby Version 10.14
    Tuning Derby Version 10.14 Derby Document build: April 6, 2018, 6:14:42 PM (PDT) Version 10.14 Tuning Derby Contents Copyright................................................................................................................................4 License................................................................................................................................... 5 About this guide....................................................................................................................9 Purpose of this guide................................................................................................9 Audience..................................................................................................................... 9 How this guide is organized.....................................................................................9 Performance tips and tricks.............................................................................................. 10 Use prepared statements with substitution parameters......................................10 Create indexes, and make sure they are being used...........................................10 Ensure that table statistics are accurate.............................................................. 10 Increase the size of the data page cache............................................................. 11 Tune the size of database pages...........................................................................11 Performance trade-offs of large pages..............................................................
    [Show full text]
  • Unravel Data Systems Version 4.5
    UNRAVEL DATA SYSTEMS VERSION 4.5 Component name Component version name License names jQuery 1.8.2 MIT License Apache Tomcat 5.5.23 Apache License 2.0 Tachyon Project POM 0.8.2 Apache License 2.0 Apache Directory LDAP API Model 1.0.0-M20 Apache License 2.0 apache/incubator-heron 0.16.5.1 Apache License 2.0 Maven Plugin API 3.0.4 Apache License 2.0 ApacheDS Authentication Interceptor 2.0.0-M15 Apache License 2.0 Apache Directory LDAP API Extras ACI 1.0.0-M20 Apache License 2.0 Apache HttpComponents Core 4.3.3 Apache License 2.0 Spark Project Tags 2.0.0-preview Apache License 2.0 Curator Testing 3.3.0 Apache License 2.0 Apache HttpComponents Core 4.4.5 Apache License 2.0 Apache Commons Daemon 1.0.15 Apache License 2.0 classworlds 2.4 Apache License 2.0 abego TreeLayout Core 1.0.1 BSD 3-clause "New" or "Revised" License jackson-core 2.8.6 Apache License 2.0 Lucene Join 6.6.1 Apache License 2.0 Apache Commons CLI 1.3-cloudera-pre-r1439998 Apache License 2.0 hive-apache 0.5 Apache License 2.0 scala-parser-combinators 1.0.4 BSD 3-clause "New" or "Revised" License com.springsource.javax.xml.bind 2.1.7 Common Development and Distribution License 1.0 SnakeYAML 1.15 Apache License 2.0 JUnit 4.12 Common Public License 1.0 ApacheDS Protocol Kerberos 2.0.0-M12 Apache License 2.0 Apache Groovy 2.4.6 Apache License 2.0 JGraphT - Core 1.2.0 (GNU Lesser General Public License v2.1 or later AND Eclipse Public License 1.0) chill-java 0.5.0 Apache License 2.0 Apache Commons Logging 1.2 Apache License 2.0 OpenCensus 0.12.3 Apache License 2.0 ApacheDS Protocol
    [Show full text]
  • Chainsys-Platform-Technical Architecture-Bots
    Technical Architecture Objectives ChainSys’ Smart Data Platform enables the business to achieve these critical needs. 1. Empower the organization to be data-driven 2. All your data management problems solved 3. World class innovation at an accessible price Subash Chandar Elango Chief Product Officer ChainSys Corporation Subash's expertise in the data management sphere is unparalleled. As the creative & technical brain behind ChainSys' products, no problem is too big for Subash, and he has been part of hundreds of data projects worldwide. Introduction This document describes the Technical Architecture of the Chainsys Platform Purpose The purpose of this Technical Architecture is to define the technologies, products, and techniques necessary to develop and support the system and to ensure that the system components are compatible and comply with the enterprise-wide standards and direction defined by the Agency. Scope The document's scope is to identify and explain the advantages and risks inherent in this Technical Architecture. This document is not intended to address the installation and configuration details of the actual implementation. Installation and configuration details are provided in technology guides produced during the project. Audience The intended audience for this document is Project Stakeholders, technical architects, and deployment architects The system's overall architecture goals are to provide a highly available, scalable, & flexible data management platform Architecture Goals A key Architectural goal is to leverage industry best practices to design and develop a scalable, enterprise-wide J2EE application and follow the industry-standard development guidelines. All aspects of Security must be developed and built within the application and be based on Best Practices.
    [Show full text]
  • Developing Applications Using the Derby Plug-Ins Lab Instructions
    Developing Applications using the Derby Plug-ins Lab Instructions Goal: Use the Derby plug-ins to write a simple embedded and network server application. In this lab you will use the tools provided with the Derby plug-ins to create a simple database schema in the Java perspective and a stand-alone application which accesses a Derby database via the embedded driver and/or the network server. Additionally, you will create and use three stored procedures that access the Derby database. Intended Audience: This lab is intended for both experienced and new users to Eclipse and those new to the Derby plug-ins. Some of the Eclipse basics are described, but may be skipped if the student is primarily focused on learning how to use the Derby plug-ins. High Level Tasks Accomplished in this Lab Create a database and two tables using the jay_tables.sql file. Data is also inserted into the tables. Create a public java class which contains two static methods which will be called as SQL stored procedures on the two tables. Write and execute the SQL to create the stored procedures which calls the static methods in the java class. Test the stored procedures by using the SQL 'call' command. Write a stand alone application which uses the stored procedures. The stand alone application should accept command line or console input. Detailed Instructions These instructions are detailed, but do not necessarily provide each step of a task. Part of the goal of the lab is to become familiar with the Eclipse Help document, the Derby Plug- ins User Guide.
    [Show full text]
  • Java DB Based on Apache Derby
    JAVA™ DB BASED ON APACHE DERBY What is Java™ DB? Java DB is Sun’s supported distribution of the open source Apache Derby database. Java DB is written in Java, providing “write once, run anywhere” portability. Its ease of use, standards compliance, full feature set, and small footprint make it the ideal database for Java developers. It can be embedded in Java applications, requiring zero administration by the developer or user. It can also be used in client server mode. Java DB is fully transactional and provides a standard SQL interface as well as a JDBC 4.0 compliant driver. The Apache Derby project has a strong and growing community that includes developers from large companies such as Sun Microsystems and IBM as well as individual contributors. How can I use Java DB? Java DB is ideal for: • Departmental Java client-server applications that need up to 24 x 7 support and the sophisti- cation of a transactional SQL database that protects against data corruption without requiring a database administrator. • Java application development and testing because it’s extremely easy to use, can run on a laptop, is available at no cost under the Apache license, and is also full-featured. • Embedded applications where there is no need for the developer or the end-user to buy, down- load, install, administer — or even be aware of — the database separately from the application. • Multi-platform use due to Java portability. And, because Java DB is fully standards-compliant, it is easy to migrate an application between Java DB and other open standard databases.
    [Show full text]
  • D3.2 - Transport and Spatial Data Warehouse
    Holistic Approach for Providing Spatial & Transport Planning Tools and Evidence to Metropolitan and Regional Authorities to Lead a Sustainable Transition to a New Mobility Era D3.2 - Transport and Spatial Data Warehouse Technical Design @Harmony_H2020 #harmony-h2020 D3.2 - Transport and SpatialData Warehouse Technical Design SUMMARY SHEET PROJECT Project Acronym: HARMONY Project Full Title: Holistic Approach for Providing Spatial & Transport Planning Tools and Evidence to Metropolitan and Regional Authorities to Lead a Sustainable Transition to a New Mobility Era Grant Agreement No. 815269 (H2020 – LC-MG-1-2-2018) Project Coordinator: University College London (UCL) Website www.harmony-h2020.eu Starting date June 2019 Duration 42 months DELIVERABLE Deliverable No. - Title D3.2 - Transport and Spatial Data Warehouse Technical Design Dissemination level: Public Deliverable type: Demonstrator Work Package No. & Title: WP3 - Data collection tools, data fusion and warehousing Deliverable Leader: ICCS Responsible Author(s): Efthimios Bothos, Babis Magoutas, Nikos Papageorgiou, Gregoris Mentzas (ICCS) Responsible Co-Author(s): Panagiotis Georgakis (UoW), Ilias Gerostathopoulos, Shakur Al Islam, Athina Tsirimpa (MOBY X SOFTWARE) Peer Review: Panagiotis Georgakis (UoW), Ilias Gerostathopoulos (MOBY X SOFTWARE) Quality Assurance Committee Maria Kamargianni, Lampros Yfantis (UCL) Review: DOCUMENT HISTORY Version Date Released by Nature of Change 0.1 02/12/2019 ICCS ToC defined 0.3 15/01/2020 ICCS Conceptual approach described 0.5 10/03/2020 ICCS Data descriptions updated 0.7 15/04/2020 ICCS Added sections 3 and 4 0.9 12/05/2020 ICCS Ready for internal review 1.0 01/06/2020 ICCS Final version 1 D3.2 - Transport and SpatialData Warehouse Technical Design TABLE OF CONTENTS EXECUTIVE SUMMARY ....................................................................................................................
    [Show full text]
  • Apache Geronimo Uncovered a View Through the Eyes of a Websphere Application Server Expert
    Apache Geronimo uncovered A view through the eyes of a WebSphere Application Server expert Skill Level: Intermediate Adam Neat ([email protected]) Author Freelance 16 Aug 2005 Discover the Apache Geronimo application server through the eyes of someone who's used IBM WebSphere® Application Server for many years (along with other commercial J2EE application servers). This tutorial explores the ins and outs of Geronimo, comparing its features and capabilities to those of WebSphere Application Server, and provides insight into how to conceptually architect sharing an application between WebSphere Application Server and Geronimo. Section 1. Before you start This tutorial is for you if you: • Use WebSphere Application Server daily and are interested in understanding more about Geronimo. • Want to gain a comparative groundwork understanding of Geronimo and WebSphere Application Server. • Are considering sharing applications between WebSphere Application Server and Geronimo. • Simply want to learn and understand what other technologies are out there (which I often do). Prerequisites Apache Geronimo uncovered © Copyright IBM Corporation 1994, 2008. All rights reserved. Page 1 of 23 developerWorks® ibm.com/developerWorks To get the most out of this tutorial, you should have a basic familiarity with the IBM WebSphere Application Server product family. You should also posses a general understanding of J2EE terminology and technologies and how they apply to the WebSphere Application Server technology stack. System requirements If you'd like to implement the two technologies included in this tutorial, you'll need the following software and components: • IBM WebSphere Application Server. The version I'm using as a base comparison is IBM WebSphere Application Server, Version 6.0.
    [Show full text]
  • IBM Acquires Gluecode Software 10 May 2005
    IBM Acquires Gluecode Software 10 May 2005 IBM today announced it has acquired Gluecode especially amongst SMB and departmental users," Software, a privately held company based in El said Robert LeBlanc, general manager, Application Segundo, Calif., and provider of software and and Integration Middleware, IBM Software Group. support services for open source application "With the Gluecode acquisition, IBM enables infrastructure software. Financial details were not customers and Business Partners to tap the low disclosed. cost of entry of open source technology to quickly develop and deploy applications, and migrate to Based on core open source technology from the WebSphere software as business needs expand." Apache Geronimo application server, Gluecode's software and related subscription support services With the acquisition, IBM will enable customers and provide a flexible and affordable infrastructure to Business Partners to download Gluecode organizations that need a reliable alternative to application server software and start development traditional commercial software offerings. and deployment at no cost, and then purchase Gluecode helps Java developers, small and mid- software support services as needed from IBM. sized businesses (SMB) and departmental users Gluecode's operations will be integrated into IBM's reduce the complexity of application development Software Group. by pre-integrating the most common services for building mainstream Java applications. "The acquisition of Gluecode by IBM recognizes the value of software solutions that are built in IBM will become an active contributor to the collaboration with the open community," said Chet Apache Geronimo open source project and will Kapoor, CEO, Gluecode. "We are pleased to expand the existing community of developers that enable a new business model for IBM that allows advance this project and innovate on top of it.
    [Show full text]