EMC SCALEIO DESIGN CONSIDERATIONS AND BEST PRACTICES ABSTRACT This white paper provides technical information, best practices, and hardware and software design considerations for planning the deployment of ScaleIO June, 2016 WHITE PAPER To learn more about how EMC products, services, and solutions can help solve your business and IT challenges, contact your local representative or authorized reseller, visit www.emc.com, or explore and compare products in the EMC Store Copyright © 2016 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. VMware and <insert other VMware marks in alphabetical order; remove sentence if no VMware marks needed. Remove highlight and brackets> are registered trademarks or trademarks of VMware, Inc. in the United States and/or other jurisdictions. All other trademarks used herein are the property of their respective owners. Part Number HXXXXX <required, see Part numbers below for more info> 2 TABLE OF CONTENTS INTRODUCTION ....................................................................................... 5 Audience ......................................................................................................... 5 SCALEIO OVERVIEW ................................................................................ 5 SCALEIO ARCHITECTURE ......................................................................... 6 CONCEPTS AND DESIGN CONSIDERATIONS ............................................ 8 Protection Domains ........................................................................................... 8 Storage Pools ................................................................................................... 9 Storage Pool Sizing .................................................................................................. 10 Fault units and fault Sets .................................................................................. 11 Sizing Usable Storage and Spare Capacity ........................................................... 12 Example 1 .............................................................................................................. 13 Example 2 .............................................................................................................. 13 System Bandwidth Guidelines ............................................................................ 14 Rebuild Throttling .................................................................................................... 14 Rebalance Throttling ................................................................................................ 14 Concurrent IO Per Device ......................................................................................... 14 MDM and Tie-Breaker Placement ........................................................................ 14 Networking ..................................................................................................... 15 Network Design Considerations .......................................................................... 16 Multipathing ............................................................................................................ 16 Metadata Manager ................................................................................................... 16 Jumbo Frames ........................................................................................................ 16 Network Testing ...................................................................................................... 17 ScaleIO Installer and Initial setup considerations ................................................. 17 ESX settings – VMDK ............................................................................................... 17 General – Zero Padding ............................................................................................ 17 Other settings - VMware ........................................................................................... 17 Host Storage Devices ............................................................................................... 18 Caching Strategies ................................................................................................... 18 Caching Recommendations ....................................................................................... 18 Two-Layer, Converged, and Hyper-converged Design ........................................... 19 Homogenous Node Design ................................................................................ 20 3 Recovery Scenarios .......................................................................................... 21 EXAMPLE CUSTOMER DEPLOYMENTS ...................................................... 22 Deployment I: Small system ............................................................................. 22 Deployment II: Small to Mid-Sized system .......................................................... 23 Deployment III: Large system, Multiple departments ............................................ 24 CONCLUSION .......................................................................................... 25 Factors To Consider ......................................................................................... 25 SUMMARY OF SCALEIO DEPLOYMENT BEST PRACTICES .......................... 27 PRODUCT LIMITS ................................................................................... 29 REFERENCES .......................................................................................... 30 TABLE OF FIGURES Figure 1) Traditional storage vs. ScaleIO ................................................................ 5 Figure 2) ScaleIO host architecture ........................................................................ 7 Figure 3) ScaleIO ESXi host architecture ................................................................. 8 Figure 4) ScaleIO Protection Domain ...................................................................... 8 Figure 5) ScaleIO Storage Pools ........................................................................... 10 Figure 6) Storage Pool example with multiple drives in a node .................................. 10 Figure 7) Storage pool with both SSD and HDD ...................................................... 10 Figure 8) Fault Sets defined on a per-rack basis ..................................................... 12 Figure 9) Spare capacity with three fault sets ......................................................... 13 Figure 10) MDM and Tie-Breaker placement ........................................................... 15 Figure 11) Flat network topology .......................................................................... 15 Figure 12) Leaf-spine network topology ................................................................. 16 Figure 13) Host caching strategies overview ........................................................... 18 Figure 14) Two-Layer architecture implementation .................................................. 19 Figure 15) Hyperconverged architecture implementation .......................................... 20 Figure 16) Homogenous SDS nodes ...................................................................... 21 Figure 17) SDS node failure ................................................................................. 22 Figure 18) Fault set failure ................................................................................... 22 Figure 19) Small six-node system ......................................................................... 23 Figure 20) Small to mid-sized 16-node system ....................................................... 24 Figure 21) Large multi-departmental system .......................................................... 25 VERSION HISTORY Version Date Details 1.0 May 2016 Initial release of guide. Covers ScaleIO versions 1.33 and 2.0. 1.1 June 2016 Renamed guide, edited/updated best practices. 4 INTRODUCTION ScaleIO is an industry-leading software-defined storage solution that enables customers to extend their existing virtual infrastructure into a high-performing virtual SAN. All the hosts (real or virtual), with direct attached storage (DAS) can be pooled together in a single storage system such that all the servers participate in servicing the I/O requests using massive parallel processing. ScaleIO can scale from as little as three hosts to just over a thousand. Capacity and throughput can be increased or decreased on the fly by adding or removing hosts, with no impact to applications or users and no disruption to operations. The goal of this white paper is to provide best practices for ScaleIO deployment. This white paper also describes performance tunings that should be applied to achieve the optimal performance for different workloads. This guide is intended to provide details on: • ScaleIO deployment best practices