Ready Solutions for Data Analytics Cloudera Hadoop 6.1 Architecture Guide

April 2019 H17614.1

Abstract This reference architecture guide describes the architectural recommendations for Cloudera Hadoop 6.1 software on Dell EMC PowerEdge servers and Dell EMC Networking switches. Copyright © 2017-2019 Dell Inc. or its subsidiaries. All rights reserved.

Published April 2019

Dell believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS-IS.“ DELL MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. USE, COPYING, AND DISTRIBUTION OF ANY DELL SOFTWARE DESCRIBED IN THIS PUBLICATION REQUIRES AN APPLICABLE SOFTWARE LICENSE.

Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks may be the property of their respective owners. Published in the USA.

Dell EMC Hopkinton, Massachusetts 01748-9103 1-508-435-1000 In North America 1-866-464-7381 www.DellEMC.com

2 Ready Architecture for Cloudera Hadoop 6.1 CONTENTS

Figures 5

Tables 7

Chapter 1 Executive Summary 9 Document purpose...... 10 Audience...... 10 Hadoop overview...... 10 Cloudera Enterprise software overview...... 10 Hadoop for the enterprise...... 10 Data management...... 11 Cloudera Enterprise components...... 11 Cloudera Enterprise Data Hub...... 12 Cloudera Hadoop 6.1 Ready Solution...... 12 Solution use case summary...... 14

Chapter 2 Ready Architecture Components 15 Solution components...... 16 Dell EMC PowerEdge rack servers...... 17 Dell EMC PowerEdge R640 server...... 17 Dell EMC PowerEdge R740xd server...... 18

Chapter 3 Solution Architecture Overview 19 Cluster architecture...... 20 High-level node architecture...... 20 High availability...... 25 Network architecture...... 27 Network definitions...... 28 Cluster physical networks...... 28 Physical network components...... 29 Rack server hardware configurations...... 36 Infrastructure Nodes...... 36 Worker Nodes...... 37 Edge Nodes...... 39 Node configuration ...... 40

Chapter 4 References 43 Cloudera partnership and certification...... 44 Dell EMC Customer Solution Centers...... 44 Technical support...... 45

Appendix A Dell EMC PowerEdge R740xd Worker Nodes Physical Rack Configuration 47 Worker Nodes single-rack configuration...... 48 Worker Nodes initial rack configuration...... 49 Worker Nodes additional pod rack configuration...... 50

Ready Architecture for Cloudera Hadoop 6.1 3 CONTENTS

Appendix B Tested Component Versions 53 Software versions...... 54 Network switch firmware versions...... 54 Dell EMC PowerEdge R640 firmware versions...... 54 Dell EMC PowerEdge R740xd firmware versions...... 55

Glossary 57

Index 61

4 Ready Architecture for Cloudera Hadoop 6.1 FIGURES

1 Solution components...... 16 2 Dell EMC PowerEdge R640 server 10 x 2.5 in. chassis...... 18 3 Dell EMC PowerEdge R740xd server 3.5 in. chassis...... 18 4 Cluster architecture...... 20 5 Cluster network fabric architecture...... 27 6 Hadoop 25 GbE network connections...... 29 7 Dell EMC PowerEdge R640 network ports...... 30 8 Dell EMC PowerEdge R740xd Worker Node network ports...... 30 9 25 GbE single-pod networking equipment...... 32 10 Dell EMC Networking Z9100-ON multiple-pod networking equipment...... 33 11 Multiple-pod view using Dell EMC Networking Z9100-ON switches (based on Layer 3 ECMP) ...... 34

Ready Architecture for Cloudera Hadoop 6.1 5 6 Ready Architecture for Cloudera Hadoop 6.1 TABLES

1 Cloudera Enterprise components/services ...... 11 2 Cloudera Enterprise Data Hub...... 12 3 Solution use cases...... 14 4 Data processing and access components...... 17 5 Cluster node roles...... 20 6 Service locations by node...... 22 7 Recommended number of nodes and pods for 25 GbE cluster...... 24 8 Alternative number of nodes and pods for 25 Gbe cluster...... 24 9 Rack and pod density scenarios...... 25 10 CDH network definitions...... 28 11 Cluster networks...... 28 12 Network/bond/interface cross reference ...... 30 13 Per rack network equipment ...... 35 14 Per pod network equipment...... 35 15 Per cluster aggregation network switches for multiple pods...... 35 16 Per node network cables required...... 35 17 Hardware configurations: Dell EMC PowerEdge R640 Infrastructure Nodes ...... 36 18 Dell EMC PowerEdge R640 Infrastructure Node volumes...... 37 19 Dell EMC PowerEdge R640 Infrastructure Node partitions...... 37 20 Hardware configurations: Dell EMC PowerEdge R740xd Worker Nodes...... 37 21 Dell EMC PowerEdge R740xd Worker Node volumes...... 38 22 Dell EMC PowerEdge R740xd Worker Node partitions...... 38 23 Hardware Configurations – Dell EMC PowerEdge R640 Edge Nodes...... 39 24 Dell EMC PowerEdge R640 Edge Node volumes...... 40 25 Dell EMC PowerEdge R640 Edge Node partitions...... 40 26 Specialized Worker Node subtypes...... 41 27 Solution Support Matrix...... 45 28 Single-rack configuration: Worker Nodes...... 48 29 Initial pod rack configuration: Dell EMC PowerEdge R740xd Worker Nodes...... 49 30 Additional pod rack configuration: Dell EMC PowerEdge R740xd Worker Nodes...... 50 31 Software versions...... 54 32 Network switch firmware versions...... 54 33 Dell EMC PowerEdge R640 firmware versions...... 54 34 Dell EMC PowerEdge R740xd firmware versions...... 55

Ready Architecture for Cloudera Hadoop 6.1 7 8 Ready Architecture for Cloudera Hadoop 6.1 CHAPTER 1 Executive Summary

This chapter presents the following topics:

l Document purpose...... 10 l Audience...... 10 l Hadoop overview...... 10 l Cloudera Enterprise software overview...... 10 l Cloudera Hadoop 6.1 Ready Solution...... 12 l Solution use case summary...... 14

Executive Summary 9 Executive Summary

Document purpose This document describes the Dell EMC server hardware and networking configuration that is recommended for running CDH, the Cloudera distribution including Apache Hadoop. It also includes recommendations for the location of CDH core services and components. The installation of additional components and services is flexible and depends on the applications and workloads. For additional and relevant information, see the Dell EMC Ready Architectures for Hadoop web page.

Audience

This document is for customers and system architects who require information about configuring Hadoop clusters in their information technology environment for Big Data analytics.

Hadoop overview

Hadoop is an Apache project that is being built and used by a global community of contributors, using the Java programming language. Yahoo! has been the largest contributor to this project and uses Apache Hadoop extensively across its businesses. Core committed contributors on the Hadoop project include employees from Cloudera, eBay, Facebook, Getopt, Hortonworks, Huawei, IBM, InMobi, INRIA, LinkedIn, MapR, Microsoft, Pivotal, Twitter, UC Berkeley, VMware, WANdisco, and Yahoo!. Many more individuals and organizations have made contributions.

Cloudera Enterprise software overview Cloudera Enterprise helps enterprises become information-driven. It combines the best of open-source software components with enterprise capabilities.

Hadoop for the enterprise

Specifically for mission-critical environments, Cloudera Enterprise includes CDH, a leading open source Hadoop-based platform. It also includes advanced system management and data management tools plus dedicated support and community advocacy from its team of Hadoop developers and experts.

10 Ready Architecture for Cloudera Hadoop 6.1 Executive Summary

Cloudera Enterprise with Apache Hadoop is:

l Unified—One integrated system that brings diverse users and application workloads to one pool of data on common infrastructure. No data movement is required.

l Secure—Perimeter security, authentication, granular authorization, and data protection.

l Governed—Enterprise-grade data auditing, data lineage, and data discovery.

l Managed—Native high-availability, fault-tolerance and self-healing storage, automated backup and disaster recovery, and advanced system and data management.

l Open—Apache-licensed open source to ensure that your data and applications remain yours, and an open platform to connect with all your existing investments in technology and skills.

Data management With Cloudera Enterprise, organizations put their data at the center of their operations to increase business visibility and reduce costs, while successfully managing risk and compliance requirements. Cloudera Enterprise provides:

l A massively scalable platform to store any amount or type of data, in its original form, for as long as required

l Integration with your existing infrastructure and tools

l Flexibility to run a variety of enterprise workloads such as batch processing, interactive SQL, enterprise search, and advanced analytics

l Robust security, governance, data protection, and management

Cloudera Enterprise components The following table lists the products and services that are included with Cloudera Enterprise.

Table 1 Cloudera Enterprise components/services

Product/Service Description

CDH As the core of Cloudera Enterprise, combines Apache Hadoop with several other open-source projects to create a single, massively scalable system. You can unite storage with an array of powerful processing and analytic frameworks.

Cloudera Manager Helps you easily deploy, manage, monitor, and diagnose issues with your cluster. Cloudera Manager is critical for operating clusters at scale.

Cloudera Support Provides technical support for Hadoop. With Cloudera Support, you gain more uptime, faster issue resolution, better performance to support your mission-critical applications, and faster delivery of platform features.

Data management 11 Executive Summary

Cloudera Enterprise Data Hub Cloudera Enterprise includes several advanced components that extend and complement the value of Apache Hadoop, as shown in the following table.

Table 2 Cloudera Enterprise Data Hub

Component Description

Online NoSQL – HBase HBase is a distributed key-value store that helps you build real-time applications on massive tables (billions of rows and millions of columns) with fast, random access.

Analytic SQL – Impala Impala is a massively parallel processing (MPP) SQL engine that is built for Hadoop.

Search – Cloudera Search Cloudera Search, based on Apache Solr, enables you to query and browse data in Hadoop, similar to searching Google or an eCommerce site.

In-Memory Machine Learning and Stream Spark delivers fast, in-memory analytics and Processing – Apache Spark real-time stream processing for Hadoop.

Data Management – Cloudera Navigator Cloudera Navigator provides critical enterprise data audit, lineage, and data discovery capabilities that enterprises require. It includes Active Data Optimization (Cloudera Navigator Optimizer), Governance and Data Management (Cloudera Navigator including auditing, lineage, discovery, and policy life cycle management), Encryption and Key Management (Cloudera Navigator Encrypt and Key Trustee).

Cloudera Hadoop 6.1 Ready Solution The Cloudera Hadoop 6.1 Ready Solution lowers the barrier to adoption for organizations that are intending to use Apache Hadoop in production. Although Hadoop is popular and widely used, installing, configuring, and running a production Hadoop cluster involves multiple considerations, including:

l Appropriate Hadoop software distribution and extensions

l Monitoring and management software

l Allocation of Hadoop services to physical nodes

l Selection of appropriate server hardware

l Design of the network fabric

l Sizing and scalability

l Performance These considerations are complicated by the need to understand the type of workloads that will run on the cluster, the fast-moving pace of the core Hadoop

12 Ready Architecture for Cloudera Hadoop 6.1 Executive Summary project, and the challenges of managing a system that is designed to scale to thousands of nodes in a single cluster. Dell EMC’s customer-centered approach is to create rapidly deployable and highly optimized end-to-end Hadoop solutions that run on hyperscale hardware. Dell EMC's unique Hadoop solution combines optimized hardware, software, and services to streamline deployment and improve the customer experience. Dell EMC and Cloudera designed this solution jointly. It embodies all the hardware, software, resources, and services that are needed to run Hadoop in a production environment. This end-to-end solution enables you to be in production with Hadoop in a shorter time than is possible with homegrown solutions. The solution is based on Cloudera Enterprise, Dell EMC PowerEdge servers, and Dell EMC Networking hardware. This solution includes best practices, optimized server configurations, and optimized network infrastructure.

Cloudera Hadoop 6.1 Ready Solution 13 Executive Summary

Solution use case summary

This solution is designed to address the use cases described in the following table.

Table 3 Solution use cases

Use case Description

Big Data analytics Rapidly query petabyte-scale unstructured and semistructured data in real time by using HBase and Hive

Data storage Collect and store unstructured and semistructured data in a secure, fault-resilient scalable data store that can be organized and sorted for indexing and analysis

Batch processing of unstructured data Batch process (index, analyze, and so on) tens to hundreds of petabytes of unstructured and semistructured data

Data archiving Archive medium-term (12–36 months) data from EDW/DBMS to expedite access, increase data retention time, or meet data retention policies or compliance requirements

Big Data visualization Capture, index, and visualize unstructured and semistructured Big Data in real time

Search and predictive analytics Crawl, extract, index, and transform semistructured and unstructured data for search and predictive analytics

14 Ready Architecture for Cloudera Hadoop 6.1 CHAPTER 2 Ready Architecture Components

This chapter presents the following topics:

l Solution components...... 16 l Dell EMC PowerEdge rack servers...... 17

Ready Architecture Components 15 Ready Architecture Components

Solution components The Dell EMC PowerEdge servers, Dell EMC Networking switches, and the operating system make up the foundation on which the solution software stack runs. The following figure shows the primary components of this Ready Architecture. Figure 1 Solution components

The Store layer components provide multiple layers of functionality on top of this foundation. The Hadoop Distributed File System (HDFS) provides the core storage for data files in the system. HDFS is a distributed, scalable, reliable, and portable file system. Apache Kudu provides a columnar relational storage option, while Apache HBase provides NoSQL access to storage. Object storage is also available. The Integrate layer shows the components that move data in and out of the Hadoop system. Apache Sqoop provides data transfer to and from relational databases while Apache Flume and Apache Kafka are optimized for real-time processing of event and log data. Also, the HDFS API and tools can be used to move data files to and from the Hadoop system. YARN provides a resource management framework for running distributed applications under Hadoop. The most popular distributed application is Hadoop’s MapReduce. Other applications, such as Apache Spark, Apache Hive, and Apache Pig, also run under YARN. Apache Sentry and RecordService provide enterprise-grade security services. The right side of the figure shows the data management capabilities that are integrated across the entire system. The left side of the figure shows the operational components that Cloudera Manager provides for Hadoop administration and management. The following multiple complementary processing and access alternatives sit on top of the Cloudera Enterprise core:

l Batch data processing

l Stream data processing

l SQL query

l Data search

16 Ready Architecture for Cloudera Hadoop 6.1 Ready Architecture Components

You can use these layers simultaneously or independently, depending on the workload and problems that you need to solve, as shown in the following table.

Table 4 Data processing and access components

Access layer Description

Batch data processing Spark, Hive, Pig, and MapReduce provide access to the massively parallel Hadoop data processing framework.

Stream data processing Spark provides stream processing.

SQL query Impala provides SQL query access to data.

Data search Apache Solr provides real-time search of indexed data.

Dell EMC PowerEdge rack servers This Ready Architecture uses Dell EMC's latest rack or modular server solutions. Highlights

l Highly optimized air flow design that enables exceptional configuration flexibility and industry-leading energy efficiency l Out of band management architecture that facilitates rapid bare metal deployment and remediation regardless of operating system state l Embedded SupportAssist that reduces troubleshooting and downtime with embedded diagnostics and automated case creation Automated productivity

l Up to 4 times performance improvement in common management tasks with the new iDRAC9 dual-core ARM processor (compared to iDRAC8) l Use of the same next-generation of embedded automation to standardize BIOS and secure boot configuration, firmware updates, server asset inventory, health monitoring, and power/reset control across all Dell EMC PowerEdge servers l Embedded proactive automated support that resolves issues up to 90 percent faster Comprehensive security

l Fully signed firmware updates in which embedded trust only allows authenticated code to run l Security lock-down that protects your server configuration and firmware (BIOS, iDRAC, and RAID) from malicious changes l Secure instant erase for HDDs, SSDs, and NVMs l A more secure, unique default password l Redfish, a new REST-based management API, that is more secure and scalable than legacy Intelligent Platform Management Interface (IPMI)

Dell EMC PowerEdge R640 server The Dell EMC PowerEdge R640 server is a dense, general purpose, scale-out compute node. Dell EMC PowerEdge R640 is an ideal choice for dense scale-out data center computing and storage in a 1U/2S platform. It enables optimization of application

Dell EMC PowerEdge rack servers 17 Ready Architecture Components

performance, price performance, or performance per watt per unit of rack space in most data center environments. The following figure shows the server. Figure 2 Dell EMC PowerEdge R640 server 10 x 2.5 in. chassis

Dell EMC PowerEdge R740xd server The Dell EMC PowerEdge R740xd server is a highly configurable software-defined storage server. The Dell EMC PowerEdge R740xd is the ideal platform for uncompromising storage performance and data set processing in a 2U/2S form factor. It provides excellent storage performance and density for applications such as software-defined storage. The Dell EMC PowerEdge R740xd is designed with the versatility that is demanded by cloud service providers, Hadoop and Big Data users, and for colocation hosting. The following figure shows the server. Figure 3 Dell EMC PowerEdge R740xd server 3.5 in. chassis

18 Ready Architecture for Cloudera Hadoop 6.1 CHAPTER 3 Solution Architecture Overview

This chapter presents the following topics:

l Cluster architecture...... 20 l Network architecture...... 27 l Rack server hardware configurations...... 36

Solution Architecture Overview 19 Solution Architecture Overview

Cluster architecture This Ready Architecture addresses all aspects of a production Hadoop cluster, including the software layers, server hardware, and network fabric, as well as scalability, performance, and ongoing management.

High-level node architecture

The following figure displays the roles for the nodes in a basic cluster. Figure 4 Cluster architecture

The cluster environment consists of multiple software services running on multiple physical server nodes. The implementation divides the server nodes into several roles, and each node has a configuration that is optimized for its role in the cluster. The physical server configurations are divided into two broad classes:

l Worker Nodes, which handle the bulk of the Hadoop processing

l Master Nodes, which support services that are needed for the cluster operation A high-performance network fabric connects the cluster nodes and separates the core data network from management functions. The minimum configuration supports nine cluster nodes, plus an optional Administration Node, as shown in the following table.

Table 5 Cluster node roles

Physical node Required or optional Hardware configuration

Administration Node Optional Infrastructure

Master Node 1 Required Infrastructure

Master Node 2 Required Infrastructure

Master Node 3 Required Infrastructure

Edge Node Required Infrastructure

Worker Node 1 Required Worker

Worker Node 2 Required Worker

20 Ready Architecture for Cloudera Hadoop 6.1 Solution Architecture Overview

Table 5 Cluster node roles (continued)

Physical node Required or optional Hardware configuration

Worker Node 3 Required Worker

Worker Node 4 Required Worker

Worker Node 5 Required Worker

Node definitions The following list provides node definitions for this solution.

l Administration Node—Provides cluster deployment and management capabilities. The Administration Node is optional in cluster deployments, depending on whether existing provisioning, monitoring, and management infrastructure is used.

l Master Node 1—Runs all the services that manage the HDFS data storage and YARN resource management. It is sometimes called the “master name node.” There are four primary services running on the Master Node 1:

n YARN Resource Manager—Supports cluster resource management, including MapReduce jobs

n NameNode—Supports HDFS data storage

n Journal Manager—Supports high availability

n ZooKeeper—Supports coordination

l Master Node 2—When quorum-based HA mode is used, runs the standby namenode process, a second journal manager, and an optional standby resource manager. This node also runs the Spark History Server and a second ZooKeeper service.

l High availability (HA) Node—Provides the third journal node for HA. The Master Node 1s and Master Node 2s provide the first and second journal nodes. It also runs a third ZooKeeper service. The operational databases that are required for Cloudera Manager and additional metastores are on the HA node.

l Edge Node—Provides an interface between the data and processing capacity that is available in the Hadoop cluster and a user of that capacity. An Edge Node has an additional connection to the Edge Network and is sometimes called a “gateway node.” At least one Edge Node is required.

l Worker Node—Runs all the services that are required to store blocks of data on the local hard drives and run processing tasks against that data. A minimum of five Worker Nodes are required. Larger clusters are scaled primarily by adding Worker Nodes. The primary services running on the Worker Nodes are:

n DataNode daemon (to support HDFS data storage)

n NodeManager daemon (to support YARN job execution)

n Services managed with Cloudera Manager service pools instead of YARN, such as Impala and HBase Spark jobs run on the Worker Nodes. However, there is no persistent service that is associated with Spark jobs.

High-level node architecture 21 Solution Architecture Overview

Node locations The following table describes the node locations and functions of the cluster services.

Table 6 Service locations by node

Physical node Software function

Administration Node Systems Management Services

First Edge Node l Hadoop Clients

l Cloudera Manager

l YARN Management

Master Node 1 l NameNode

l Resource Manager

l ZooKeeper

l Quorum Journal Node

l Impala State Store and Catalog Daemons

l Kudu Master

l YARN Management

Master Node 2 l Yum Repositories

l Standby NameNode

l Standby Resource Manager (optional)

l Spark History Server

l Spark2 History Server

l Quorum Journal Node

l Hbase Master

l Hbase REST Server

l Thrift Server

l Hue Server

l Hue Load Balancer

l ZooKeeper

Master Node 3 l ZooKeeper

l Quorum Journal Node

l Operational Databases (PostgreSQL)

Worker Node(N) l DataNode

l NodeManager

l HBase RegionServer

l Impala Daemon

l Kudu Tablet Server

22 Ready Architecture for Cloudera Hadoop 6.1 Solution Architecture Overview

Cluster sizing This Ready Architecture is organized into three units to help you size the solution as the Hadoop environment grows. From smallest to largest, the units include:

l Rack on page 23

l Pod on page 23

l Cluster on page 23 Each unit has specific characteristics and sizing considerations. The design goal for the Hadoop environment is to enable you to scale the environment by adding more capacity without replacing existing components. Rack A rack is the smallest unit for a Hadoop environment. A rack consists of the power, network cabling, and data and management switches to support a group of Worker Nodes. A rack is a physical unit and its capacity is defined by physical constraints that include available space, power, cooling, and floor loading. Ensure that a rack uses its own power within the data center, independent from other racks, and is treated as a fault zone. If a rack fails in a multiple-rack pod or cluster, the cluster continues to function with reduced capacity. This Ready Architecture uses 12 nodes as the size of a rack, but higher or lower densities are possible. Typically, a rack contains about 12 nodes using a scale-out server such as the Dell EMC PowerEdge R740xd server. The node density of a rack does not affect overall cluster scaling and sizing, but it does affect fault zones in the cluster. Pod A pod is the set of nodes that is connected to the first level of network switches in the cluster. It consists of one or more racks. A pod can include a small number of nodes initially and expand to the maximum number of nodes over time. A pod is a second-level fault zone above the rack level. If a pod fails in a multiple-pod cluster, the cluster continues to function with reduced capacity. A pod can support enough Hadoop server nodes and network switches for a minimum commercial-scale installation. In this Ready Architecture, a pod supports up to 36 nodes (typically three racks). This size results in a bandwidth oversubscription of 2.25:1 between pods in a full cluster. The size of a pod can vary from this baseline recommendation. Changing the pod size affects the bandwidth oversubscription at the pod level, the size of the fault zones, and the maximum cluster size. Cluster A cluster is a single Hadoop environment that is attached to a pair of network switches that provide an aggregation layer for the entire cluster. A cluster can range in size from a pod consisting of a single rack up to many pods. A single-pod cluster is a special case and can function without an aggregation layer. This scenario is typical for smaller clusters before the addition of more pods. In this Ready Architecture, the limit on the total size of a cluster depends on the choice of Layer 2 or Layer 3 switching and the switch models that are used. See Sizing summary on page 24 for the limits.

High-level node architecture 23 Solution Architecture Overview

Sizing summary The minimum configuration supports nine nodes:

l Master Node 1

l Master Node 2

l Master Node 3

l Edge Node

l Five Worker Nodes Although each cluster requires a minimum of one Edge Node, larger clusters and clusters with high ingest volumes or rates might require additional Edge Nodes. Cloudera recommends a baseline of one Edge Node for every twenty Worker Nodes. The hardware configuration for the Infrastructure Nodes supports clusters in the range of petabyte storage. Other than the Infrastructure Nodes, cluster capacity is primarily a function of the server platform and disk drives that are chosen, and the number of Worker Nodes. The following table shows the recommended number of nodes per pod and pods per cluster for 25 GbE clusters using the S5048-ON switch.

Table 7 Recommended number of nodes and pods for 25 GbE cluster

Nodes per Nodes per pod Pods per Nodes per Bandwidth rack cluster cluster oversubscription

12 36 8 288 2.25 : 1

The following table shows alternatives for cluster sizing with different bandwidth oversubscription ratios. Table 8 Alternative number of nodes and pods for 25 Gbe cluster

Nodes per Nodes per pod Pods per Nodes per Bandwidth rack cluster cluster oversubscription

12 48 8 384 3 : 1

12 36 10 360 3 : 1

12 24 16 384 3 : 1

24 Ready Architecture for Cloudera Hadoop 6.1 Solution Architecture Overview

Power and cooling are typically the primary constraints on rack density. However, a rack is a potential fault zone and rack density affects overall cluster reliability, especially for smaller clusters. The following table shows possible scenarios that are based on typical data center constraints.

Table 9 Rack and pod density scenarios

Server platform Nodes Racks Comments per rack per pod

Dell EMC PowerEdge 12 3 Typical configuration that requires less than R740xd 10 kW power per rack and provides good rack-level fault zone isolation

Dell EMC PowerEdge 10 2 Smaller rack and pod fault zones with R740xd slightly higher bandwidth oversubscription of 2.5 : 1

High availability

This Ready Architecture implements high availability (HA) at multiple levels through a combination of hardware redundancy and software support.

Hadoop redundancy The Hadoop distributed file system implements redundant storage for data resiliency, and is aware of node and rack location. Data is replicated across multiple nodes and across racks. This replication provides multiple copies of data for reliability if there are disk or node failures. It can also increase performance. The number of replicas defaults to three and can be changed easily at the cluster and file level. The specified networks provide sufficient bandwidth for replication traffic as well as production traffic. Hadoop automatically balances data across the cluster nodes and creates additional replicas when a node fails. The bandwidth that is used for replication can also be controlled.

Note

The Hadoop job parallelism model can scale to larger and smaller numbers of nodes, enabling jobs to run when parts of the cluster are offline.

Network redundancy

The production network can optionally use bonded connections to pairs of switches in each pod and switch pairs at the aggregation level. This configuration provides increased bandwidth capacity and allows operation at reduced capacity if a network port, network cable, or switch fails. When using 25 GbE as the core fabric, we typically do not use bonded networking. For large clusters, we recommend the use of Layer 3 aggregation, which provides network redundancy at the spine-switch level. Refer to 25 GbE Layer 3 Dell EMC Networking Z9100-ON cluster aggregation on page 33 for details.

HDFS highly available NameNodes This Ready Architecture implements high availability (HA) for the Hadoop Distributed File System (HDFS) directory through a quorum mechanism that replicates critical

High availability 25 Solution Architecture Overview

namenode data across multiple physical nodes. Production clusters implement namenode HA. In quorum-based HA, there are typically two namenode processes running on two physical servers. At any point in time, one of the NameNodes is in an Active state and the other is in a Standby state. The Active NameNode is responsible for all client operations in the cluster, while the Standby NameNode acts as a slave, maintaining enough state to provide a fast failover if necessary. For the Standby NameNode to keep its state synchronized with the Active NameNode in this implementation, both nodes communicate with a group of separate daemons called JournalNodes. When the Active NameNode modifies any namespace, it consistently logs a record of the modification to a majority of these JournalNodes. The Standby NameNode can read the edits from the JournalNodes, and is constantly watching them for changes to the edit log. As the Standby NameNode detects the edits, it applies them to its own namespace. If a failover occurs, the Standby NameNode ensures that it has read all the edits from the JournalNodes before promoting itself to the Active state. This action ensures that the namespace state is fully synchronized before a failover occurs. To provide a fast failover, it is necessary that the Standby NameNode has up-to-date information about the location of blocks in the cluster. Therefore, the Worker Nodes are configured with the location of both the NameNode and Standby NameNode, and they send block location information and heartbeats to both. Because edit log modifications must be written to a majority of JournalNodes, there must be an odd number of (and at least three) JournalNode daemons. The JournalNode daemons run on the Master Node 1, Master Node 2, and Master Node 3 in this Ready Architecture.

YARN resource manager high availability This Ready Architecture supports high availability (HA) for the Hadoop YARN resource manager. Without resource manager HA, currently running jobs to fail when a Hadoop resource manager fails. When resource manager HA is enabled, jobs can continue running if a resource manager fails. On failover, the applications can resume from their last check-pointed state. For example, completed map tasks in a MapReduce job are not rerun on a subsequent attempt. This action enables events such as machine crashes or planned maintenance to be handled without any significant performance effect on running applications. An Active/Standby pair of resource managers implements resource manager HA. On startup, each resource manager is in the standby state, which means that the process is started, but the state is not loaded. When transitioning to the active state, the resource manager loads the internal state from the designated state store and starts all the internal services. The stimulus to transition-to-active comes from either the administrator or through the integrated failover controller when automatic failover is enabled.

Note

Resource manager HA is not always implemented in production clusters.

26 Ready Architecture for Cloudera Hadoop 6.1 Solution Architecture Overview

Database server high availability This Ready Architecture supports high availability (HA) for the operational databases. The database server that is used for both the Cloudera Manager operational and metadata databases stores its data on a RAID 10 partition to provide redundancy in the event of a drive failure.

Note

Our default installation uses a single PostgreSQL instance. Therefore, there is a single point of failure. You can implement database server HA by using one or more additional PostgreSQL instances on other nodes in the cluster or by using an external database server.

Network architecture The cluster network is designed to meet the needs of a high performance and scalable cluster, while providing redundancy and access to management capabilities. The architecture is a leaf/spine model that is based on 25 GbE networking technologies. It uses Dell EMC Networking S5048-ON switches for the leaves and Dell EMC Networking Z9100-ON switches for the spine. IPv4 is used for the network layer. At this time, the architecture does not support the use of IPv6 for network connectivity. The following figure shows the logical network architecture. Figure 5 Cluster network fabric architecture

Network architecture 27 Solution Architecture Overview

Network definitions Three distinct networks are used in the cluster. The following table describes the CDH networks and their purposes. Table 10 CDH network definitions

Network Description Available services

Cluster Data The Data network carries the bulk of the traffic within the The Cloudera Enterprise services are Network cluster. This network is aggregated within each pod, and pods available on this network. are aggregated into the cluster switch. Note

The Cloudera Enterprise services do not support multihoming and are only accessible on the Cluster Data Network.

iDRAC/BMC The BMC network connects the BMC or iDRAC ports and the This network provides access to the Network out-of-band management ports of the switches. It is used for BMC and iDRAC functionality on the hardware provisioning and management. This network is servers. It also provides access to the aggregated into a management switch in each rack. management ports of the cluster switches.

Edge Network The Edge network provides connectivity from the Edge SSH access to Edge Nodes is available Nodes to an existing premises network, either directly, or by on this network, and other application using the pod or cluster aggregation switches. services may be configured and available.

Cluster physical networks The following table lists the distinct networks that are used in the cluster.

Table 11 Cluster networks

Logical network Connection Switch

Cluster Data network 25 GbE Top-of-rack (ToR) (pod) switches and aggregation switches

BMC network 1 GbE Dedicated switch per rack

Edge network 25 GbE Direct to Edge network or through a pod or aggregation switch

28 Ready Architecture for Cloudera Hadoop 6.1 Solution Architecture Overview

Each network uses a separate VLAN and dedicated components when possible. The following figure shows the logical organization of the network. For more information about the configuration of the interfaces and switches, see the solution deployment guide. Figure 6 Hadoop 25 GbE network connections

Physical network components The physical networks of this Ready Architecture consist of the following components:

l Server node connections

l Network fabric on page 30

n 25 GbE pod switches on page 31

n 25 GbE cluster aggregation switches on page 33

l iDRAC management network on page 34 Network integration information is presented in:

l Core network integration on page 34

l Layer 2 and Layer 3 separation on page 34 All equipment is listed in:

l Network equipment summary: 25 GbE configurations on page 35

Server node connections Server connections to the network switches for the Data network use Ethernet technology. Connections to the network use 25 GbE, which is recommended for new Dell EMC PowerEdge R640 and Dell EMC PowerEdge R740xd server deployments. Edge Nodes have an additional available network connection. This connection facilitates high-performance cluster access between applications running on those nodes and the optional Edge network.

Physical network components 29 Solution Architecture Overview

Server connections to the BMC network use a single connection from the iDRAC port to a S3048-ON management switch in each rack, as shown in the following figures. Figure 7 Dell EMC PowerEdge R640 network ports

Figure 8 Dell EMC PowerEdge R740xd Worker Node network ports

The following table shows the mapping of individual interfaces to networks and bonds.

Table 12 Network/bond/interface cross reference

Server Platform Network Interface Bond

Dell EMC PowerEdge R740xd Cluster Data em1 none

Dell EMC PowerEdge R640 Cluster Data em1 none

Dell EMC PowerEdge R640 Edge em2 none

Network fabric We recommend 25 GbE for new deployments of Dell EMC PowerEdge R740xd and Dell EMC PowerEdge R640 servers. Clusters larger than a single pod require an aggregation layer. The aggregation layer can be implemented at either Layer 2 (L2) or Layer 3 (L3). The choice depends on the initial size and planned scaling. Layer 2 aggregation provides lower cost and medium scalability, and can support approximately 250 nodes.

30 Ready Architecture for Cloudera Hadoop 6.1 Solution Architecture Overview

Layer 3 aggregation is recommended for: l Larger initial deployments of over 250 nodes l Deployments where extreme scale-out is planned to about 1500 nodes l Instances where the cluster must be colocated with other applications in a different rack The scalability depends on the switches that are used and the oversubscription ratio, and is summarized in Sizing summary on page 24. The standard implementation instructions and tools that are described in the solution deployment guide are oriented to a Layer 2 aggregation implementation, while Layer 3 aggregation is a customized deployment. The following sections describe the fabric: l 25 GbE pod switches on page 31 l 25 GbE cluster aggregation switches on page 33 25 GbE pod switches Each pod uses a Dell EMC Networking S5048-ON switch as the first layer switch. The pod switches are often referred to as top of rack (ToR) switches, although this architecture splits a physical rack from a logical pod. The Dell EMC Networking S5048-ON switch is a multiple-rate 100 GbE 1U spine switch that is optimized for high-performance, ultra-low-latency data center requirements. The Dell EMC Networking Z9100-ON switch can provide a cumulative bandwidth of 7.4 Tb/sec of throughput at line-rate traffic from every port. It can be configured with up to: l 32 ports of 100 GbE (QSFP28) l 64 ports of 50 GbE (QSFP+) l 32 ports of 40 GbE (QSFP+) l 128 ports of 25 GbE (QSFP+) l 128+2 ports of 10 GbE The following figure shows the single-pod network configuration, with a Dell EMC Networking S5048-ON switch aggregating the pod traffic.

Physical network components 31 Solution Architecture Overview

Figure 9 25 GbE single-pod networking equipment

For a single pod, the pod switch can act as the aggregation layer for the entire cluster. For multiple-pod clusters, a cluster aggregation layer is required. In this architecture, each pod is managed as a separate entity from a switching perspective, and the individual pod switches connect only to the aggregation switch.

32 Ready Architecture for Cloudera Hadoop 6.1 Solution Architecture Overview

25 GbE cluster aggregation switches For clusters consisting of more than one pod, the architecture uses the Dell EMC Networking Z9100-ON switch for aggregation. The Dell EMC Networking Z9100-ON can be used for both Layer 2 and Layer 3 implementations. 25 GbE Layer 2 Dell EMC Networking Z9100-ON cluster aggregation The following figure illustrates the configuration for a multiple-pod cluster using the Z9100-ONswitch for cluster aggregation switch with Layer 2 networking. The uplink from each S5048-ON pod switch to the aggregation layer uses four 100 GbE interfaces in a bonded configuration, providing a collective bandwidth of 400 Gb from each pod. Figure 10 Dell EMC Networking Z9100-ON multiple-pod networking equipment

25 GbE Layer 3 Dell EMC Networking Z9100-ON cluster aggregation The Dell EMC Networking Z9100-ON core switch can be used for aggregation at Layer 3 in larger clusters using 25 GbE. We use a different network architecture for a cluster that uses Layer 3 aggregation, based on Equal-Cost Multipath (ECMP) routing and a leaf/spine organization. In this configuration, a cluster can scale to over 1,500 nodes, with a low 3 :1 oversubscription per pod.

Physical network components 33 Solution Architecture Overview

The following figure shows this alternative configuration for a multiple-pod cluster by using Layer 3 and ECMP routing. Figure 11 Multiple-pod view using Dell EMC Networking Z9100-ON switches (based on Layer 3 ECMP)

For more details about Layer 3 Leaf/Spine deployment, see Leaf-Spine Deployment and Best Practices Guide for Greenfield Deployments. iDRAC management network In addition to the Cluster Data network, a separate network is provided for cluster management - the iDRAC (or BMC) network. The iDRAC management ports are aggregated into a per-rack Dell EMC Networking S3048-ON switch with a dedicated VLAN. This aggregate provides a dedicated iDRAC/BMC network for hardware provisioning and management. Switch management ports are also connected to this network. The management switches can be connected to the core or connected to a dedicated management network if out of band management is required.

Core network integration The aggregation layer functions as the network core for the cluster. In most instances, the cluster connects to a larger core in the enterprise, as shown in Figure 10 on page 33. When you use the Dell EMC Networking S5048-ON switch, two 100 GbE ports are reserved at the aggregation level for connection to the core. Connection details are site-specific and must be determined as part of the deployment planning.

Layer 2 and Layer 3 separation

The Layer 2 and Layer 3 boundaries are separated at either the pod or the aggregation layer. Either option is equally viable. This Ready Architecture is based on Layer 2 for switching in the cluster. The colors blue and red in Figure 11 on page 34 represent the Layer 2 and Layer 3 boundaries.

34 Ready Architecture for Cloudera Hadoop 6.1 Solution Architecture Overview

Network equipment summary: 25 GbE configurations The following tables summarize the required cluster networking equipment.

Table 13 Per rack network equipment

Component Quantity

Total racks 1 (12 nodes nominal)

Management switch 1 x Dell EMC Networking S3048-ON

Switch interconnect cables 1 x 1 GbE cables (to next rack management switch)

Table 14 Per pod network equipment

Component Quantity

Total racks 3 (36 Nodes)

Top-of-rack switches 1 x Dell EMC Networking S5048-ON

Pod uplink cables (to aggregate switch) 4 x 100 Gb QSFP+ cables

Table 15 Per cluster aggregation network switches for multiple pods

Component Quantity

Total pods 8

Aggregation layer switches 1 x Dell EMC Networking Z9100-ON

The following table summarizes the number of cables that are needed for a cluster.

Table 16 Per node network cables required

Description 1 GbE cables required 25 GbE connections with QSFP+ required

Master Nodes 1 x number of nodes 1 x number of nodes

Edge Nodes 1 x number of nodes 2 x number of nodes

Worker Nodes 1 x number of nodes 1 x number of nodes

Note

25 GbE node connections typically use a QSFP+ to Quad QSPF breakout cable, so the cable count is typically one-fourth the number of connections in the preceding table.

Physical network components 35 Solution Architecture Overview

Rack server hardware configurations This Ready Architecture supports the Dell EMC PowerEdge R640 and Dell EMC PowerEdge R740xd servers using configurations for the following:

l Infrastructure Nodes on page 36

l Worker Nodes on page 37

l Edge Nodes on page 39 For more information about configuration, refer to Appendix A, which provides the recommended rack layout for Dell EMC PowerEdge R740xd clusters.

Infrastructure Nodes Infrastructure Nodes host the critical cluster services. The configuration is optimized to reduce downtime and provide high performance. The following table shows the recommended configuration.

Table 17 Hardware configurations: Dell EMC PowerEdge R640 Infrastructure Nodes

Components Details

Platform Dell EMC PowerEdge R640

Chassis 2.5 in. chassis with up to 10 hard drives and 2 PCIe slots

Processor Dual Intel Xeon Gold 6134 3.2 GHz (8 core) 24.75M Cache

RAM 192 GB (12 x 16 GB 2,667 MT/s)

Network Daughter Card Mellanox ConnectX-4 Lx Dual Port 25 GbE DA/SFP rNDC

Boot configuration From PERC controller

Storage controller Dell EMC PERC H740P 2 Gb NV Cache, Minicard

Disk - Spindles 8 x 1 TB 7.2K RPM NLSAS 12 Gb/s

Disk - SSD 2 x 480 GB SSD SAS mixed-use 12 Gb/s

Drive configuration Combination of RAID 1, RAID 10, and dedicated drives

Note

Consult your Dell EMC account representative before changing the recommended disk sizes.

The Infrastructure Nodes (Master Node 1, Master Node 2, Master Node 3, and Edge Node) are configured as multiple partitions and file systems by using all available drives. Each partition is optimized for both performance and reliability. The following table shows the recommended disk and partition layout for the Infrastructure Nodes.

36 Ready Architecture for Cloudera Hadoop 6.1 Solution Architecture Overview

Table 18 Dell EMC PowerEdge R640 Infrastructure Node volumes

Physical Used by Volume type disks

2-3 Operating system RAID1

0 ZooKeeper journal Passthrough SSD

1 NameNode journal Passthrough SSD

4-5 HDFS metadata RAID1

6-9 Database storage RAID10

Table 19 Dell EMC PowerEdge R640 Infrastructure Node partitions

Disk Partition Mount Size Filesystem Description point type

Virtual 0 Primary /boot 1,074 MB ext4 BIOS boot files that must be within the first 2 GB of disk

Virtual 0 LVM / 100 GB ext4 Root file system

Virtual 0 LVM swap 4 GB swap Operating system swap space partition

Virtual 0 LVM /home 1 GB ext4 User home directories

Virtual 0 LVM /var 825 GB ext4 Operational data directory for databases. It primarily contains the Cloudera Manager databases because the Postgres Data Directory (PGDATA) is typically in /var/lib/pgsql. Configure alternatives to Postgres to store their data files here.

Worker Nodes Worker Nodes are the workhorses of the cluster. Worker Nodes combine compute and storage. Depending on the intended workload, they can be optimized for storage- heavy, compute-heavy, or mixed loads. The following table shows a 2U chassis option using large form-factor (LFF) 3.5 in. drives for data. This option provides dense storage capability with high performance compute and solid state storage for fast caching of temporary data.

Table 20 Hardware configurations: Dell EMC PowerEdge R740xd Worker Nodes

Component Details

Platform Dell EMC PowerEdge R740xd server

Chassis Chassis with up to 12 x 3.5 in. HDD, 4 x 3.5 in. HDDs on MP and 4 x 2.5 in. HDDs on Flex Bay

Processor Dual Intel Xeon Gold 6140 2.3 GHz, 18 Core, 25 M Cache

RAM (minimum) 384 GB (12 x 32 GB 2667 MT/s)

Network Daughter Card Mellanox ConnectX-4 Lx Dual Port 25 GbE DA/SFP rNDC

Boot configuration BOSS controller card + with 2 M.2 Sticks 240 GB

Worker Nodes 37 Solution Architecture Overview

Table 20 Hardware configurations: Dell EMC PowerEdge R740xd Worker Nodes (continued)

Component Details

Storage controller Dell EMC PERC HBA330 RAID Controller, 12 Gb Minicard

Disk - spindles 16 x 4 TB 7.2 K RPM SATA 6 Gb/ps 512n 3.5 in. hot-plug hard drive

Disk - SSD 4 x 480 GB SSD SAS mixed-use 12 Gb/ps

Drive configuration RAID 1 - OS JBOD - data drives

The following tables show the recommended disk and partition layout for the Worker Nodes.

Table 21 Dell EMC PowerEdge R740xd Worker Node volumes

Physical Usage Volume type disks

BOSS 0, BOSS Operating system RAID 1 1

0-11 HDFS data Passthrough

12-15 Selectable Passthrough SSD

16-19 HDFS data Passthrough

Table 22 Dell EMC PowerEdge R740xd Worker Node partitions

Virtual disk Partition Mount Size Filesystem Description point type

DellBOSS 1 Primary /boot 1074 MB ext4 Contains BIOS boot files that must be within the first 2 GB of disk

DellBOSS 2 LVM / 100 GB ext4 Root file system

DellBOSS 3 LVM swap 4 GB swap Operating system swap space partition

DellBOSS 4 LVM /home 1 GB ext4 User home directories

DellBOSS 5 LVM /var 117.5 GB ext4 Contains variable data such as system logging files, databases, mail and printer spool directories, and transient and temporary files

sda Primary /data/1 4096 GB ext4 Contains HDFS data

sdb Primary /data/2 4096 GB ext4 Contains HDFS data

sdn Primary /data/n 4096 GB ext4 Contains HDFS data

ssd1 Primary /datassd/1 4096 GB ext4 Tiered HDFS Storage, Spark cache, MapReduce temp files, or HBase tiered cache

ssd2 Primary /datassd/2 4096 GB ext4 Tiered HDFS Storage, Spark cache, MapReduce temp files, or HBase tiered cache

38 Ready Architecture for Cloudera Hadoop 6.1 Solution Architecture Overview

Table 22 Dell EMC PowerEdge R740xd Worker Node partitions (continued)

Virtual disk Partition Mount Size Filesystem Description point type

sss3 Primary /datassd/3 4096 GB ext4 Tiered HDFS Storage, Spark cache, MapReduce temp files, or HBase tiered cache

ssd4 Primary /datassd/4 4096 GB ext4 Tiered HDFS Storage, Spark cache, MapReduce temp files, or HBase tiered cache

Note

l Dell EMC does not recommend that you configure a large swap space. Due to the large and random performance degradation that might result, avoid swapping in a Hadoop cluster.

l Operating system partitions are configured with the Logical Volume Manager enabled.

Edge Nodes Edge Nodes are the primary interface through which the data moves in and out of the cluster. They are also used to run applications that access the cluster. Because of the wide variation in applications, Edge Node configurations can vary significantly. The main characteristic of Edge Nodes is a connection to the Cluster Data network and additional network connections for external access. A common baseline choice for Edge Node configuration uses the same configuration as an infrastructure node, as shown in the following table.

Table 23 Hardware Configurations – Dell EMC PowerEdge R640 Edge Nodes

Component Details

Platform Dell EMC PowerEdge R640

Chassis 2.5 in. Chassis with up to 10 Hard Drives and 2 PCIe slots

Processor Dual Intel Xeon Gold 6134 3.2 GHz (8 Core) 24.75 M Cache

RAM 192 GB (12 x 16 GB 2667 MT/s)

Network Daughter Card Mellanox ConnectX-4 Lx Dual Port 25 GbE DA/SFP rNDC

Boot Configuration From PERC controller

Storage Controller Dell EMC PERC H740P Gb NV Cache, Minicard

Disk - Spindles 8 x 1 TB 7.2 K RPM NLSAS 12 Gb/ps

Disk - SSD 2 x 480 GB SSD SAS mixed-use 12 Gb/ps

Drive Configuration Combination of RAID 1, RAID 10, and dedicated drives

Edge Nodes 39 Solution Architecture Overview

Table 24 Dell EMC PowerEdge R640 Edge Node volumes

Physical Used by Volume type disks

2-3 Operating system RAID1

0 Spare Flash storage Passthrough SSD

1 Spare Flash storage Passthrough SSD

4-9 Determined by application RAID10, or determined by application

Table 25 Dell EMC PowerEdge R640 Edge Node partitions

Disk Partition Mount Size Filesystem Description point type

Virtual 0 Primary /boot 1,074 MB ext4 BIOS boot files that must be within first 2 GB of disk

Virtual 0 LVM / 100 GB ext4 Root file system

Virtual 0 LVM swap 4 GB swap Operating system swap space partition

Virtual 0 LVM /home 1 GB ext4 User home directories

Node configuration

See Dell EMC PowerEdge R740xd Worker Nodes Physical Rack Configuration on page 47 for the recommended rack layout for Dell EMC PowerEdge R740xd clusters.

Infrastructure Node sizing The hardware configuration for Infrastructure Nodes supports petabyte-scale clusters, based on the number of HDFS blocks that are used. The size of the HDFS metadata storage must be adjusted for clusters:

l Larger than 250 nodes

l With per-node HDFS storage larger than 64 TB

l With very large HDFS block sizes Approximately 2 TB of RAID10 storage is available for the operational databases, including the HIVE metastore and the Cloudera Manager databases. This storage is enough for a typical large cluster. You might need to adjust the size of this partition for very large clusters. The load on the Master Node 3 is less than on the other Infrastructure Nodes. The Master Node 3 configuration is the same as Master Node 1 and Master Node 2. It simplifies operational hardware maintenance and the Master Node 3 can be used as a spare node. You can specialize the configuration of this node, if necessary.

40 Ready Architecture for Cloudera Hadoop 6.1 Solution Architecture Overview

Worker Node sizing

Storage sizing Drive capacities greater than 4 TB or node storage density greater than 48 TB require special consideration for HDFS setup. Configurations of this size approach the limit of Hadoop per-node storage capacity. At a minimum, the HDFS block size must be no less than 128 MB and can be as large as 1,024 MB. Because the number of files, blocks per file, compression, and reserved space factor into the calculations, the configuration requires an analysis of the intended cluster usage and data. When sizing nodes, per-node density also has an impact on cluster performance if nodes fail. The bandwidth that is required to replicate the lost data affects overall performance, the time that is required to finish the recovery is lengthy, and data is under-replicated and at risk during the recovery.

CAUTION

Do not configure a single Worker Node with more than 100 TB of storage.

Note

Your Dell EMC representative can assist you with estimates and calculations.

Node subtypes You can configure Worker Nodes to match their intended use in the cluster. We recommend the subtypes that are described in the following table. Table 26 Specialized Worker Node subtypes

Subtype Cloudera Workload/usage Mount point category

Generic Data Engineering Batch processing, data lake, data /datassd/ pipelines, Spark, and MapReduce

HBase Operational HBase with inserts, updates, and /hbase/ Database queries

Kudu Analytical Database Interactive and analytical queries with /kudu/ Kudu and Impala

Tiered Data Engineering Batch processing/HDFS with tiered /datassd/ storage

l Generic—The node is configured for general Hadoop data engineering operations. Rotational storage is configured for HDFS use, and solid state drives are configured for temporary file storage for Spark and MapReduce spill data. l HBase—The node is configured for operational usage of HBase. Solid state drives are configured for use as the HBase write ahead log and bucket cache. l Kudu—The node is configured for Kudu and Impala analytical operations. Solid state drives are set up for the Kudu write ahead log, Impala scratch space, and general temporary files. l Tiered—The node is configured for general Hadoop data engineering operations. Solid states drives are configured as a separate HDFS storage tier. Processors The recommended Intel Xeon SP processors provide the best balance among several characteristics that include cost, performance, and power consumption. Alternative

Node configuration 41 Solution Architecture Overview

processors can be used to optimize different scenarios. The recommended processors include dual AVX-512 units for the highest performance on analytic applications. Memory sizing The base recommendation of 384 GB assumes a mixture of MapReduce, query, and computational analytic workloads. Spark can take advantage of substantially higher memory footprints for RDD caches. Additional memory can be useful for Spark workloads. You might need to change the Spark spark.memory.offHeap.enabled configuration setting to take advantage of the larger available memory. You can also configure HBase with a large memory cache. Many HBase workloads can benefit from the larger cache from larger memory configurations. When changing memory configurations, keep the number of DIMMs to 6 or 12 per processor, if possible. Having fewer than six DIMMs per processor incurs a significant performance penalty because not all memory channels are used. Using more than two DIMMs per processor channel slightly reduces memory speed.

Edge Node sizing The baseline Edge Node configuration uses the same configuration as an Infrastructure Node. In practice, Edge Nodes are typically customized based on their intended purpose. The first Edge Node typically runs Cloudera Manager, but it has additional capacity to run other applications. If the Edge Node is used to run driver or front-end programs for cluster applications, additional memory might be required. The processors rarely need to be upgraded. For instances in which Edge Nodes are used for streaming data or general ingest operations, you can configure disk space as a staging area for input data. The recommended chassis configuration has 10 x 2.5 in. drive bays available, which can be configured with HDDs or SSDs as needed with no restrictions beyond the underlying server platform restrictions. You can use alternative server platforms when appropriate. Edge Nodes include dual 25 GbE connections. One connection is used for the cluster data network, and the other connection is available for interfacing to the core network or external networks. You can add more network interfaces for additional capacity or you can use alternative network interface cards. We recommend using a 25 GbE connection to the cluster data network. However, you can use a lower-speed connection if it is adequate for your environment.

42 Ready Architecture for Cloudera Hadoop 6.1 CHAPTER 4 References

This chapter presents the following topics:

l Cloudera partnership and certification...... 44 l Dell EMC Customer Solution Centers...... 44 l Technical support...... 45

References 43 References

Cloudera partnership and certification

Note

Cloudera, Inc., as a result of a merger transaction, is now the parent company of Hortonworks, Inc.

Cloudera is a key contributor to the Apache Hadoop project. CDH, the Cloudera distribution including Apache Hadoop, is a highly scalable open-source platform for high-volume data management and analytics. CDH integrates with existing enterprise IT infrastructure, enabling data engineers and data scientists to quickly and easily develop and deploy Hadoop applications in a cost-efficient manner. Dell EMC is a Platinum member of the Cloudera IHV Program. Platinum membership is the highest level of partnership and indicates Dell EMC's ongoing commitment to Cloudera and our customers. The Dell EMC infrastructure in this guide is Cloudera-certified.

Dell EMC Customer Solution Centers

Our global network of dedicated Dell EMC Customer Solution Centers are trusted environments where world class IT experts collaborate with customers and prospects to share best practices, facilitate in-depth discussions of effective business strategies using briefings, workshops, or proof of concept (PoCs), and help businesses become more successful and competitive. Dell EMC Customer Solution Centers reduce the risks associated with new technology investments and can help improve speed of implementation.

44 Ready Architecture for Cloudera Hadoop 6.1 References

Technical support The following table shows the supported components and operating environments for this Ready Architecture.

Table 27 Solution Support Matrix

Category Component Available support

Operating system Red Hat Enterprise Linux Red Hat Linux support Server

CentOS Dell EMC Hardware support

Java Virtual Machine Sun Oracle JVM Not available

Hadoop Cloudera Enterprise Cloudera support

Hadoop Cloudera Manager Cloudera support

Hadoop Cloudera Navigator Cloudera support

Technical support 45 46 Ready Architecture for Cloudera Hadoop 6.1 APPENDIX A Dell EMC PowerEdge R740xd Worker Nodes Physical Rack Configuration

This appendix contains suggested rack layouts for single-rack, single-pod, and multiple-pod installations. Rack layouts vary depending on power, cooling, and loading constraints.

l Worker Nodes single-rack configuration...... 48 l Worker Nodes initial rack configuration...... 49 l Worker Nodes additional pod rack configuration...... 50

Dell EMC PowerEdge R740xd Worker Nodes Physical Rack Configuration 47 Dell EMC PowerEdge R740xd Worker Nodes Physical Rack Configuration

Worker Nodes single-rack configuration

Table 28 Single-rack configuration: Worker Nodes

RU RACK1

42 R1 - Switch 1: Dell EMC Networking S5048-ON

41 Cable management

40 Cable management

39 Cable management

38 R1 - Dell EMC Networking S3048-ON iDRAC Management switch

37 Cable management

36 Cable management

35 Empty 29

28 Edge01: Dell EMC PowerEdge R640 27

26 Master Node 1: Dell EMC PowerEdge R640 25

24 Master Node 2: Dell EMC PowerEdge R640 23

22 Master Node 3: Dell EMC PowerEdge R640 21

20 Empty 19

18 Empty 17

16 R1 - Chassis08: Dell EMC PowerEdge R740xd 15

14 R1 - Chassis07: Dell EMC PowerEdge R740xd 13

12 R1 - Chassis06: Dell EMC PowerEdge R740xd 11

10 R1 - Chassis05: Dell EMC PowerEdge R740xd 9

8 R1 - Chassis04: Dell EMC PowerEdge R740xd

48 Ready Architecture for Cloudera Hadoop 6.1 Dell EMC PowerEdge R740xd Worker Nodes Physical Rack Configuration

Table 28 Single-rack configuration: Worker Nodes (continued)

RU RACK1

6 R1 - Chassis03: Dell EMC PowerEdge R740xd 5

4 R1 - Chassis02: Dell EMC PowerEdge R740xd 3

2 R1 - Chassis01: Dell EMC PowerEdge R740xd 1

Worker Nodes initial rack configuration

Table 29 Initial pod rack configuration: Dell EMC PowerEdge R740xd Worker Nodes

RU RACK1 RACK2 RACK3

42 Empty R2 - Switch 1: Dell EMC Networking Empty S5048-ON

41 Empty Empty Empty

40 Cable management Cable management Cable management

39 Cable management Cable management Cable management

38 R1 - Dell EMC Networking S3048-ON R2 - Dell EMC Networking S3048- R3 - Dell EMC Networking S3048- iDRAC Management switch ON iDRAC Management switch ON iDRAC Management switch

37 Cable management Cable management Cable management

36 Cable management Cable management Cable management

35 Master Node 1: Dell EMC PowerEdge Edge01: Dell EMC PowerEdge R640 R3 - Switch 1: Dell EMC Networking R640 Z9100-ON

34 Empty Empty Empty

33 Empty Master Node 2: Dell EMC PowerEdge Master Node 3: Dell EMC PowerEdge R640 R640 32

31 Empty Empty Empty 21

20 R1 - Chassis10: Dell EMC PowerEdge R2 - Chassis10: Dell EMC PowerEdge R3 - Chassis10: Dell EMC PowerEdge R740xd R740xd R740xd 19

18 R1 - Chassis09: Dell EMC PowerEdge R2 - Chassis09: Dell EMC R3 - Chassis09: Dell EMC R740xd PowerEdge R740xd PowerEdge R740xd 17

Worker Nodes initial rack configuration 49 Dell EMC PowerEdge R740xd Worker Nodes Physical Rack Configuration

Table 29 Initial pod rack configuration: Dell EMC PowerEdge R740xd Worker Nodes (continued)

RU RACK1 RACK2 RACK3

16 R1 - Chassis08: Dell EMC PowerEdge R2 - Chassis08: Dell EMC R3 - Chassis08: Dell EMC R740xd PowerEdge R740xd PowerEdge R740xd 15

14 R1 - Chassis07: Dell EMC PowerEdge R2 - Chassis07: Dell EMC PowerEdge R3 - Chassis07: Dell EMC PowerEdge R740xd R740xd R740xd 13

12 R1 - Chassis06: Dell EMC PowerEdge R2 - Chassis06: Dell EMC R3 - Chassis06: Dell EMC R740xd PowerEdge R740xd PowerEdge R740xd 11

10 R1 - Chassis05: Dell EMC PowerEdge R2 - Chassis05: Dell EMC R3 - Chassis05: Dell EMC R740xd PowerEdge R740xd PowerEdge R740xd 9

8 R1 - Chassis04: Dell EMC PowerEdge R2 - Chassis04: Dell EMC R3 - Chassis04: Dell EMC R740xd PowerEdge R740xd PowerEdge R740xd 7

6 R1 - Chassis03: Dell EMC PowerEdge R2 - Chassis03: Dell EMC R3 - Chassis03: Dell EMC R740xd PowerEdge R740xd PowerEdge R740xd 5

4 R1 - Chassis02: Dell EMC PowerEdge R2 - Chassis02: Dell EMC R3 - Chassis02: Dell EMC R740xd PowerEdge R740xd PowerEdge R740xd 3

2 R1 - Chassis01: Dell EMC PowerEdge R2 - Chassis01: Dell EMC PowerEdge R3 - Chassis01: Dell EMC PowerEdge R740xd R740xd R740xd 1

Worker Nodes additional pod rack configuration

Table 30 Additional pod rack configuration: Dell EMC PowerEdge R740xd Worker Nodes

RU RACK1 RACK2 RACK3

42 Empty R2 - Switch 1: Dell EMC Networking Empty S5048-ON

41 Empty Empty Empty

40 Cable Management Cable management Cable management

39 Cable Management Cable management Cable management

38 R1 - Dell EMC Networking S3048-ON R2 - Dell EMC Networking S3048- R3 - Dell EMC Networking S3048- iDRAC Management switch ON iDRAC Management switch ON iDRAC Management switch

37 Cable management Cable management Cable management

36 Cable management Cable management Cable management

35 Empty Empty Empty 25

50 Ready Architecture for Cloudera Hadoop 6.1 Dell EMC PowerEdge R740xd Worker Nodes Physical Rack Configuration

Table 30 Additional pod rack configuration: Dell EMC PowerEdge R740xd Worker Nodes (continued)

RU RACK1 RACK2 RACK3

24 R1 - Chassis12: Dell EMC PowerEdge R2 - Chassis12: Dell EMC PowerEdge R1 - Chassis12: Dell EMC PowerEdge R740xd R740xd R740xd 23

22 R1 - Chassis11: Dell EMC PowerEdge R2 - Chassis11: Dell EMC PowerEdge R3 - Chassis11: Dell EMC PowerEdge R740xd R740xd R740xd 21