Hortonworks Dataflow V3.3.1 Adds Support for Apache Kafka 2.0
Total Page:16
File Type:pdf, Size:1020Kb
IBM Canada Software Announcement A19-0091, dated January 29, 2019 Hortonworks DataFlow V3.3.1 adds support for Apache Kafka 2.0 and Kafka Streams; Hortonworks Data Platform V3.1 enhances its enterprise-ready component set to enable faster, smarter, and hybrid data Table of contents 1 Overview 4 Technical information 3 Key requirements 5 Ordering information 3 Planned availability date 6 Terms and conditions 3 Program number 8 Prices 4 Publications 9 Order now Overview Hortonworks DataFlow Hortonworks DataFlow is an enterprise-ready, open source, streaming data platform with Enterprise Flow Management, Enterprise Stream Processing, and Enterprise Management Services. It enables organizations to collect, curate, analyze, and act on data in the data centre and cloud. Hortonworks DataFlow is powered by key open sourced projects, including Apache NiFi, Apache Kafka, and Apache Storm. • Hortonworks DataFlow Enterprise Flow Management includes support services for Apache NiFi. These services can help provide easy ingestion, routing, management, and delivery of data to downstream systems with intelligence. • Hortonworks DataFlow Enterprise Stream Processing includes support services for Apache Kafka and Storm and Streaming Analytics Manager. Kafka and Storm enable immediate and continuous insights using aggregations over windows, pattern matching, predictive and prescriptive analytics. • Hortonworks DataFlow Enterprise Management Services can help deliver the provisioning, management, and monitoring to enable IT to manage the entire Hortonworks DataFlow cluster efficiently and comprehensively. New in version 3.3.1 The official Apache versions for Hortonworks DataFlow V3.3.1 now include: • Apache Ambari 2.7.3 • Apache Kafka 2.0.0 (with support for Kafka Streams) • Apache NiFi 1.8.0 • NiFi Registry 0.2.0 • Apache Ranger 1.2.0 • Apache Storm 1.2.1 • Apache ZooKeeper 3.4.6 • Hortonworks Schema Registry 0.5.4 • Hortonworks Streaming Analytics Manager 0.6.0 • Apache Knox 1.0.0 IBM Canada Software Announcement A19-0091 IBM is a registered trademark of International Business Machines Corporation 1 Hortonworks Data Platform Hortonworks Data Platform is a scalable, enterprise-ready, and open source platform for storing, processing, and analyzing large volumes of data-at-rest. It is a key component of the modern data architecture and can be deployed both on-premises and the cloud. Hortonworks Data Platform V3.1 consists of the following set of Apache Hadoop projects: • Apache Accumulo 1.7.0 • Apache Atlas 1.1.0 • Apache Calcite 1.16.0 • Apache DataFu 1.3.0 • Apache Hadoop 3.1.1 • Apache HBase 2.0.2 • Apache Hive 3.1.0 • Apache Kafka 2.0.0 • Apache Knox 1.0.0 • Apache Livy 0.5.0 • Apache Oozie 4.3.1 • Apache Phoenix 5.0.0 • Apache Pig 0.16.0 • Apache Ranger 1.2.0 • Apache Spark 2.3.2 • Apache Sqoop 1.4.7 • Apache Storm 1.2.1 • Apache TEZ 0.9.1 • Apache Zeppelin 0.8.0 • Apache ZooKeeper 3.4.6 New in version 3.1 Hortonworks Data Platform V3.1 brings updates to selected Apache components. Apache Kafka Core capability updates: • Kafka Streams is now officially supported. Kafka Streams is fully integrated with platform services such as Ranger and Schema Registry, and supports secure and unsecure clusters. Security enhancements: • Simplified access control management with support for prefixed access control lists (ACLs) for bulk management of entities • Access control for topic creation for specific topics/topic prefix • Hostname verification to prevent SSL configuration man-in-the-middle attacks • Improved encryption support with faster Transport Layer Security (TLS) and CRC32C implementation; faster over-the-wire encryption • Simpler security configuration with SSL trust stores update without broker restart • Security for Apache ZooKeeper listeners that can be configured before starting brokers Performance improvements: IBM Canada Software Announcement A19-0091 IBM is a registered trademark of International Business Machines Corporation 2 • New configuration options that can help prevent indefinite consumer block • Improved windowed aggregation performance in Kafka Streams • Quota limit notifications that distinguish network errors from quota limits that are reached • Improved broker resiliency by reducing memory footprint of message down conversion • Replication protocol improvements for fixing log divergence during fast leader failover Apache Ranger Integration enhancements: • Support for Kafka 2.0 with Kafka Ranger plug-in • Support "DelegationToken" resource type with "Describe" operation for Kafka • Support for cluster resource type for Kafka • Support for Create operation with Topic resource in Kafka • Support for Knox Trusted Proxy • Support for doAs (Trusted proxy) Key requirements Hortonworks DataFlow and Hortonworks Data Platform support CentOS (64-bit) and Red Hat Enterprise Linux(R) (RHEL). For details, see the Software requirements section. Planned availability date February 1, 2019 Accessibility by people with disabilities A US Section 508 Accessibility Compliance Report containing details on accessibility compliance can be found on the Product accessibility information website. Reference information For more information about Hortonworks DataFlow, see Software Announcement A18-0537, dated September 04, 2018. For more information about Hortonworks Data Platform, see Software Announcement A18-0770, dated October 9, 2018. Program number Program number VRM Program name 5737-I18 3.3.1 Hortonworks DataFlow 5737-H46 3.1.0 Hortonworks Data Platform Offering Information IBM Canada Software Announcement A19-0091 IBM is a registered trademark of International Business Machines Corporation 3 Product information is available on the IBM(R) Offering Information website. More information is also available on the Passport Advantage(R) and Passport Advantage Express(R) website. Publications None Services Software Services IBM Software Services has the breadth, depth, and reach to manage your services needs. You can leverage the deep technical skills of our lab-based, software services team and the business consulting, project management, and infrastructure expertise of our IBM Global Services team. Also, we extend our IBM Software Services reach through IBM Business Partners to provide an extensive portfolio of capabilities. Together, we provide the global reach, intellectual capital, industry insight, and technology leadership to support a wide range of critical business needs. To learn more about IBM Software Services, contact your Lab Services Sales or Delivery Leader. Technical information Specified operating environment Software requirements Hortonworks DataFlow Operating systems: • CentOS 7.2, 7.3, 7.4, and 7.5 • RHEL 7.2, 7.3, 7.4, and 7.5 Processors: • IBM Power SystemsTM. Power8 and Power9 x86-64 Browsers: • Chrome 69 and 70 • Firefox 62 and 63 • Safari 11 and 12 JavaTM Development Kits (JDKs): • Oracle JDK - JDK8 • OpenJDK - JDK8 Hortonworks Data Platform Operating systems: • CentOS 7.2, 7.3, 7.4, and 7.5 IBM Canada Software Announcement A19-0091 IBM is a registered trademark of International Business Machines Corporation 4 • RHEL 7.2, 7.3, 7.4, and 7.5 JDK: • Oracle JDK - JDK8 • OpenJDK - JDK8 Planning information Packaging This offering is delivered through the internet as an electronic download. There is no physical media. This program, when downloaded from a website, contains the applicable IBM license agreement and License Information, if appropriate, which will be presented for acceptance at the time of installation of the program. For future reference, the license and License Information will be stored in a directory such as LICENSE.TXT. Ordering information For ordering information, consult your IBM representative or authorized IBM Business Partner, or go to the Passport Advantage website. This product is only available through Passport Advantage. It is not available as shrinkwrap. These products may only be sold directly by IBM or by authorized IBM Business Partners for Channel Value Rewards. More information can be found on the IBM Channel Value Rewards website. To locate IBM Business Partners for Channel Value Rewards in your geography for a specific Channel Value Rewards portfolio, go to the Find a Business Partner page. Passport Advantage No new parts. Charge metric Definitions of the charge metrics for these licensed products can be found in the following License Information documents: Program name PID number License Information document number Hortonworks DataFlow 5737-I18 L-MLOY-B6WSYN Enterprise Flow Management 3.3 Hortonworks DataFlow L-MLOY-B6WTA6 Enterprise Management Services 3.3 Hortonworks DataFlow L-MLOY-B6WTGE Enterprise Stream Processing 3.3 Hortonworks Data 5737-H46 L-MLOY-B44635 Platform 3.1.0 Hortonworks Data L-MLOY-B7ES6D Platform for IBM Optimized 3.1.0 Hortonworks Data L-MLOY-B7ESA8 Platform for IBM Storage Capacity 3.1.0 IBM Canada Software Announcement A19-0091 IBM is a registered trademark of International Business Machines Corporation 5 Select your language of choice and scroll down to the Charge Metrics section. Terms and conditions The information provided in this announcement letter is for reference and convenience purposes only. The terms and conditions that govern any transaction with IBM are contained in the applicable contract documents such as the IBM International Program License Agreement, IBM International Passport Advantage Agreement, and the IBM Agreement for Acquisition of Software Maintenance. This product is only available through Passport Advantage. Licensing IBM International Program