Optimizing Vmware Vsan™ Performance on a Dell EMC® Poweredge™ R7525 Server with Nvme™ Namespaces and KIOXIA CM6 Series Ssds
Total Page:16
File Type:pdf, Size:1020Kb
Reference Architecture Optimizing VMware vSAN™ Performance on a Dell EMC® PowerEdge™ R7525 Server with NVMe™ Namespaces and KIOXIA CM6 Series SSDs Published May 2021 KIOXIA: Tyler Nelson Adil Rahman Scott Harlin Participating Companies: AMD, Inc. Dell, Inc. VMware®, Inc. Table of Contents Introduction . .3 Background . 3 The Industry Challenge . 4 System Architecture . .4 System Configuration . .4 Hardware Configuration . 5 Host Installation / VMware vCenter Setup . 5 Network Configuration . 6 VMkernel Adapter Configuration . .7 NVMe Namespace Configuration . 8 Cluster Configuration . .9 VMware vSAN Configuration . 10 Creating Fault Domains . .11 Creating VM Storage Policies . .11 Test Methodology . .13 Test Cases . .13 Test Results . 13 Without NVMe Namespaces . .13 With Two (2) NVMe Namespaces . .14 With Five (5) NVMe Namespaces . .14 With Eight (8) NVMe Namespaces . .15 Summary of IOPS Performance . .15 Summary of Throughput Performance . .16 Test Analysis . .17 Recommendations . 18 Summary . .18 Summary of Tuning Parameters and Settings . 19 Reference Architecture Introduction This reference architecture (RA) presents the synergy between NVMe protocol namespaces and VMware vSAN virtualization software to demonstrate increases in SSD input/output operations per second (IOPS) performance. It includes hardware and software tuning, and the test process, methodology, results and analysis for a configuration that features a Dell EMC PowerEdge R7525 rack server with two-socket AMD EPYC™ CPUs and the ability to utilize up to twenty-four (24) PCIe® 4.0 NVMe SSDs per server. The testing conducted showcases the raw computing power of a hyper-converged infrastructure (HCI) in combination with fast, scalable local storage. Using an aggregate of twelve (12) KIOXIA CM6 Series PCIe 4.0 NVMe SSDs, the configuration was able to achieve random read performance up to 896,555 IOPS1. Configuration variability is also addressed to enable storage architects, system integrators, channel partners and other key stakeholders to create flexible designs that scale up to meet data center compute and storage needs. An optimal configuration is also presented that will help application administrators and storage architects obtain optimal storage performance from this configuration. Additional benefits include: Summary A Dell EMC PowerEdge R7525 server cluster with KIOXIA CM6 Series PCIe 4.0 NVMe SSDs demonstrates impressive IOPS and throughput performance, with low-latencies in a vSAN HCI environment More clusters and drives increase system performance as vSAN virtualization performance scales with the cluster size and available SSDs NVMe protocol namespaces with vSAN virtualization enables full utilization of CM6 Series SSD capacity: • Into both cache and capacity tiers • Paramount in scaling local storage when duplicated in HCIs Efficient scaling of compute and storage resources reduces data center cost without sacrificing quality vSAN virtualization with NVMe namespaces are applicable to many different real-world workloads Background Businesses rely on IT specialists to assemble the compute, storage and networking resources for their respective data centers and to address critical demands such as exponential data growth, technological advances and constant upgrades to hardware resources. Having to upgrade hardware for each resource pool is both expensive in cost and in lost productivity as the upgrades require additional testing to determine compatibility with the pre- existing infrastructure. These hardware challenges have pushed many IT departments to move to a software-defined, HCI approach that virtualizes compute and storage resources using a combination of commercially-available server hardware and very fast SSDs for local-attached storage. Each hyper-converged compute/ storage node in the virtualized cluster runs a hypervisor that contributes its resources into a cluster of other hyper-converged nodes. The hypervisors are controllable by management software to form and manage the server clusters, and to assign resource allocations depending on individual virtual machine (VM) needs. The HCI approach enables data centers to evolve with small server clusters and/or scaled storage based on their unique business requirements. For this RA, KIOXIA selected VMware Inc. components that included the VMware ESXi™ hypervisor, VMware vSphere® enterprise-scale virtualization platform, and VMware vCenter® management software, as they are helping to evolve HCIs. These platforms serve to abstract compute resources from various hosts into a cluster of resources that can be provisioned for use by VMs or containers. When VMware vSAN enterprise-class, storage virtualization software is combined with VMware vSphere, IT departments can manage compute and storage resources within a single platform. VMware vSAN software integrates seamlessly with the VMware vSphere stack for easy management, and the KIOXIA CM6 Series SSDs and namespace configuration helps to minimize latency and deliver fast IOPS performance. It also can utilize NVMe SSDs as local storage enabling them to be divided into cache and capacity tiers to increase performance. © 2021 KIOXIA Corporation. All rights reserved. Reference Architecture | Optimizing VMware vSAN™ Performance on a Dell EMC® PowerEdge™R7525 Server with NVMe™ Namespaces and KIOXIA CM6 Series SSDs | May 2021 | Rev. 1.0 https://business.kioxia.com/ 3 Reference Architecture NVMe SSDs are recommended for HCIs and represent the latest and fastest protocol for flash-based storage media. They are built on the high-speed PCIe interface that uses lanes to directly connect to the CPU and delivers significantly improved performance and low-latency versus SAS and SATA options. The NVMe protocol can support up to 65,535 input/output (I/O) queues, each having 65,535 commands per queue, where multiple CPU cores can access the queues in parallel simultaneously. The NVMe protocol also supports namespaces that enable individual logical block addresses (LBA) to be accessible to host software, and each LBA has a namespace ID that the underlying storage controller uses to identify it as a unique drive. Before the NVMe protocol existed, an SSD would need to be dedicated as a cache OR capacity drive as VMware vSAN software was only able to utilize 600 gigabyte2 (GB) of a cache drive. It is common for the cache drives to partially utilize capacity, which in turn negatively impacted the ability to scale local-based storage. Using namespaces, SSD capacity can be fully utilized as both cache AND data storage. The Industry Challenge In an HCI, entire disks are used for caching while others are used for data. There are also limits to the number of cache devices and disk groups that can be used. Each cache device in VMware software requires an entire disk be allocated to cache with a limited cache size of 600GB. To obtain optimal performance from a dedicated cache SSD, a system administrator would need to purchase one 3 DWPD3, 800GB or 1.6 terabyte2 (TB) SSD and allocate the entire disk to cache. This process is highly inefficient, under-utilizes the capacity of higher end and faster performing SSDs, and makes it difficult to scale storage capacities independent of nodes, forcing system admins to use lower performing SSDs to meet the cache requirements. System Architecture The system architecture and associated SSDs were selected, with different configurations tested, to determine an optimal setup. KIOXIA CM6 Series PCIe 4.0 NVMe SSDs with 7.68TB capacities were chosen for their PCIe 4.0 capability and DWPD. In order to utilize namespaces (so that the SSD can perform as both a cache and data storage), larger capacity points were selected to utilize higher overall data written per day. Instead of using a smaller 3 DWPD SSD for cache, a larger 1 DWPD drive was chosen so that the SSD can utilize higher overall data written per day as follows: 7.68TB x 1 DWPD = 7.68TB written per day VS 800GB x 3 DWPD = 2.4TB written per day The larger drive allows a higher overall daily data write than a smaller 3 DWPD SSD. Testing was then conducted to demonstrate the performance of the configuration running NVMe namespaces and VMware vSAN software. The system configuration used for testing included: System Configuration Server Configuration: Dell EMC PowerEdge R7525 Total number of systems 3 BIOS and version PowerEdge R7525 Version 1.7.3 Operating system and version VMWare ESXi 7.0.1 17325551 U1 P25 Last date of OS patching 10/05/2020 Power management policy High performance Processor Configuration: Number of processors 2 Vendor and model AMD EPYC 7552 48-Core Processor Core count 48 Core frequency 2200 MHz Stepping 0 Memory Module Configuration: Total system memory 256 GB Number of memory modules 16 Vendor and model Micron® MTA18ASF2G72PDZ-2G9E1TI 16 GB 2RX8 PC4-2933Y-RE2-12 © 2021 KIOXIA Corporation. All rights reserved. Reference Architecture | Optimizing VMware vSAN™ Performance on a Dell EMC® PowerEdge™R7525 Server with NVMe™ Namespaces and KIOXIA CM6 Series SSDs | May 2021 | Rev. 1.0 https://business.kioxia.com/ 4 Reference Architecture Size 16 GB Type ECC DDR-4 Speed (MHz) 2933 MHz NVMe SSD Configuration: Number of SSDs 12 (4 per system) Drive vendor and model KIOXIA KCM6XRUL7T68 (CM6 Series) Capacity (GB) 7680 SSD information 2.5-inch4 (15mm) PCIe Gen 4.0, NVMe v1.4 Hardware Configuration BIOS Processor Settings: Dell EMC PowerEdge R7525 Logical Processor Enabled Virtualization Technology Enabled IOMMU Support Enabled L1 Stream HW Prefetcher Enabled L2 Stream HW Prefetcher