DELL EMC SCALEIO Networking Best Practices and Design Considerations ABSTRACT This document describes core concepts and best practices for designing, troubleshooting, and maintaining a Dell EMC™ ScaleIO® network of any size. October 2016 1 The information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any software described in this publication requires an applicable software license. Copyright © 2016 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks may be the property of their respective owners. Published in the USA 10/16, White Paper, H14708.3 Dell EMC believes the information in this document is accurate as of its publication date. The information is subject to change without notice. TABLE OF CONTENTS EXECUTIVE SUMMARY ...........................................................................................................6 AUDIENCE AND USAGE ..........................................................................................................6 SCALEIO FUNCTIONAL OVERVIEW.......................................................................................6 SCALEIO SOFTWARE COMPONENTS ...................................................................................7 ScaleIO Data Servers (SDS) ............................................................................................................. 8 ScaleIO Data Clients (SDC) .............................................................................................................. 8 Meta Data Managers (MDM) ............................................................................................................. 9 TRAFFIC TYPES .......................................................................................................................9 ScaleIO Data Client (SDC) to ScaleIO Data Server (SDS) ............................................................... 9 ScaleIO Data Server (SDS) to ScaleIO Data Server (SDS) .............................................................. 9 Meta Data Manager (MDM) to Meta Data Manager (MDM) ............................................................ 10 Meta Data Manager (MDM) to ScaleIO Data Client (SDC) ............................................................. 10 Meta Data Manager (MDM) to ScaleIO Data Server (SDS) ............................................................ 10 Other Traffic .................................................................................................................................... 10 NETWORK INFRASTRUCTURE ............................................................................................ 10 Flat Network Topologies .................................................................................................................. 11 IPv4 and IPv6 .................................................................................................................................. 12 NETWORK PERFORMANCE AND SIZING ........................................................................... 12 Network Latency .............................................................................................................................. 12 Network Throughput ........................................................................................................................ 12 Example: An SDS-only node with 10 HDDs ............................................................................................. 13 Example: An SDS-only node with 6 SSDs and 10 HDDs ......................................................................... 13 Write-heavy environments ....................................................................................................................... 13 Hyper-converged environments ............................................................................................................... 14 NETWORK HARDWARE ....................................................................................................... 14 Two NICs vs. Four NICs and Other Configurations ......................................................................... 14 Switch Redundancy ......................................................................................................................... 15 Buffer Capacity ................................................................................................................................ 15 IP CONSIDERATIONS ........................................................................................................... 15 IP-level Redundancy ....................................................................................................................... 15 ETHERNET CONSIDERATIONS ........................................................................................... 16 Jumbo Frames ................................................................................................................................ 16 VLAN Tagging ................................................................................................................................. 16 LINK AGGREGATION GROUPS ........................................................................................... 17 LACP ............................................................................................................................................... 17 Load Balancing................................................................................................................................ 17 Multiple Chassis Link Aggregation Groups ...................................................................................... 18 THE MDM NETWORK ............................................................................................................ 18 NETWORK SERVICES ........................................................................................................... 18 DNS ................................................................................................................................................. 18 DHCP .............................................................................................................................................. 19 DYNAMIC ROUTING CONSIDERATIONS ............................................................................ 19 Bidirectional Forwarding Detection (BFD) ....................................................................................... 19 Physical Link Configuration ............................................................................................................. 20 ECMP .............................................................................................................................................. 20 OSPF .............................................................................................................................................. 20 Link State Advertisements (LSAs) ................................................................................................... 20 Shortest Path First (SPF) Calculations ............................................................................................ 21 BGP ................................................................................................................................................. 21 Host to Leaf Connectivity ................................................................................................................ 22 Leaf and Spine Connectivity ............................................................................................................ 23 Leaf to Spine Bandwidth Requirements .......................................................................................... 23 VRRP Engine .................................................................................................................................. 24 AMS Considerations ........................................................................................................................ 24 VMWARE CONSIDERATIONS .............................................................................................. 24 IP-level Redundancy ....................................................................................................................... 24 LAG and MLAG ............................................................................................................................... 24 SDC ................................................................................................................................................. 25 SDS ................................................................................................................................................. 25 MDM ................................................................................................................................................ 25 VALIDATION METHODS ....................................................................................................... 25 ScaleIO Native Tools ....................................................................................................................... 25 SDS Network Test ..................................................................................................................................