TIBCO® Messaging - Schema Repository for Apache Kafka User's Guide Version 1.2 December 2020 Copyright © 2018-2020. TIBCO Software Inc. All Rights Reserved. 2 Contents About this Product .....................................................................................3 TIBCO Documentation and Support Services ............................................................4 Architecture ...........................................................................................5 Using the Schema Repository .......................................................................... 6 Schema Repository Command Line Reference ............................................................................. 6 Password Security .....................................................................................................9 Apache Avro Client Library Configuration Options ......................................................10 Legal and Third-Party Notices ......................................................................... 12 TIBCO® Messaging - Schema Repository for Apache Kafka User's Guide 3 About this Product ® ® TIBCO is proud to announce the latest release of TIBCO Messaging - Schema Repository for Apache Kafka. ® This release is the latest in a long history of TIBCO products that leverage the power of Information Bus ® technology to enable truly event-driven IT environments. To find out more about how TIBCO Messaging ® software and other TIBCO products are powered by TIB technology, please visit us at www.tibco.com. ® TIBCO Messaging - Schema Repository for Apache Kafka is part of TIBCO Messaging software. Product Editions TIBCO Messaging software is available in a community edition and an enterprise edition. TIBCO Messaging - Community Edition is ideal for getting started with TIBCO Messaging, for implementing application projects (including proof of concept efforts), for testing, and for deploying applications in a production environment. Although the community license limits the number of production processes, you can easily upgrade to the enterprise edition as your use of TIBCO Messaging expands. The community edition is available free of charge, with the following limitations and exclusions: ● Users may run up to 100 application instances or 1000 web/mobile instances in a production environment. ● Users do not have access to TIBCO Support, but you can use TIBCO Community as a resource (community.tibco.com). TIBCO Messaging - Enterprise Edition is ideal for all application development projects, and for deploying and managing applications in an enterprise production environment. It includes all features presented in this documentation set, as well as access to TIBCO Support. TIBCO® Messaging - Schema Repository for Apache Kafka User's Guide 4 TIBCO Documentation and Support Services How to Access TIBCO Documentation Documentation for TIBCO products is available on the TIBCO Product Documentation website, mainly in HTML and PDF formats. The TIBCO Product Documentation website is updated frequently and is more current than any other documentation included with the product. To access the latest documentation, visit docs.tibco.com. Product-Specific Documentation Documentation for TIBCO products is not bundled with the software. Instead, it is available on the TIBCO Documentation site. To access the documentation web page for this product from a local software installation, open the following file: TIBCO_HOME/akd/repo/1.2/product_info/docinfo.txt How to Contact TIBCO Support You can contact TIBCO Support in the following ways: ● For an overview of TIBCO Support, visit http://www.tibco.com/services/support. ● For accessing the Support Knowledge Base and getting personalized content about products you are interested in, visit the TIBCO Support portal at https://support.tibco.com. ● For creating a Support case, you must have a valid maintenance or support contract with TIBCO. You also need a user name and password to log in to https://support.tibco.com. If you do not have a user name, you can request one by clicking Register on the website. How to Join TIBCO Community TIBCO Community is the official channel for TIBCO customers, partners, and employee subject matter experts to share and access their collective experience. TIBCO Community offers access to Q&A forums, product wikis, and best practices. It also offers access to extensions, adapters, solution accelerators, and tools that extend and enable customers to gain full value from TIBCO products. In addition, users can submit and vote on feature requests from within the TIBCO Ideas Portal. For a free registration, go to https://community.tibco.com. TIBCO® Messaging - Schema Repository for Apache Kafka User's Guide 5 Architecture The following diagram illustrates the interactions between the Schema Repository, the Apache Kafka Brokers, and the Apache Kafka Clients. FTL Server Schema register schema retrieve schema Repository Kafka API Kafka API messages Kafka Broker messages Producer Client Consumer Client The TIBCO schema repository assists with schema storage and management between the Schema Repository and the Apache Kafka messaging system. The FTL server (typically a cluster of three or more) provides back-end storage of the schemas. The Schema Repository daemon (tibschemad) stores and retrieves Apache Avro message schemas. The Apache Avro message format is a widely used and compact representation. Parsing is fast. Apache Avro supports schema evolution and field name aliases. TIBCO® Messaging - Schema Repository for Apache Kafka User's Guide 6 Using the Schema Repository The schema repository manages Avro message schemas by registering Avro schemas, and associating each subject name with its schema. As you update a subject's schema, the subject automatically retains its schema history in versions.. Apache Kafka client applications and Apache Kafka brokers can use the schema repository to manage Avro schemas. Within client processes, TIBCO's Avro serialization library accesses the schema repository by directing REST requests via the FTL server with which it is registered, at ftl_server_host:port/schema/v1/ For REST API documentation, start the schema repository, then use a browser to view http:// schema_repository_host:port. The schema repository uses the FTL realm server/FTL server as back-end storage. Schema Repository Command Line Reference The tibschemad command starts the schema repository process. The schema repository executable is installed in TIBCO_HOME/akd/repo/version/bin/. Client Communication Parameter Arguments Description -l interface:port Optional. -listen The repository listens for requests at this interface and port. It is good practice to direct requests through the realm server, which redirects the requests to an appropriate schema repository. If not specified on the command line, the default location is localhost:8081. -compat loose or Optional. strict Set compatibility level. strict (default) - Guarantee compatibility with other TIBCO products and libraries. loose - Relax requirements on client requests and attempt to accommodate a wider range of third-party tools. TIBCO® Messaging - Schema Repository for Apache Kafka User's Guide 7 Realm Server Back-End Storage Parameter Arguments Description -ftl URL Optional. A list of the realm server/FTL server URLs, separated by the pipe character (|). The schema repository uses the realm server/FTL server at this location for schema storage. If not specified on the command line, the default URL is http://localhost:8080 -u user Optional, required for communication with a secure realm server/FTL server. -user The schema repository authenticates itself to the realm server/FTL server using this user name. -p password Optional, required for communication with a secure realm server/FTL server. -password The repository authenticates itself to the realm server/FTL server using this password. The password can be supplied using one of the following forms: ● stdin ● env:environment_var_name ● file:password_file_path ● pass:password For further details, see Password Security. -trust-file file_path Optional, required for TLS communication with a secure realm server/FTL server. When present, the repository process reads the realm server's/FTL server's trust file from this file, and uses that trust data in communications with the secure realm server/FTL server. -trust-everyone Optional, the schema repository trusts any realm server/FTL server without verifying trust in the realm server's/FTL server's certificate. Production use of this option except for convenience in development and testing is not recommended because it eliminates all authentication. TIBCO® Messaging - Schema Repository for Apache Kafka User's Guide 8 Parameter Arguments Description -mem Optional, when specified the schema repository stores schemas only in process memory (which is not persistent in the event of a schema repository restart) instead of using the realm server/FTL server for backend storage. Do not use this mode in production environments. Configuration Parameter Arguments Description -c file_path Optional, path to a JSON configuration file. -config Command line arguments override environment variables, which override configuration file arguments. When absent, the schema repository first looks for ./.tibschemad, then ~/.tibschemad. -env Optional, the repository displays the environment variables defining its current configuration and exits. -show-config Optional, the repository displays the contents of a configuration file that define