Fibre Channel Storage Area Network Design Considerations for Deploying Virtualized SAP HANA

WHITE PAPER

Business Case Enterprises are striving to virtualize their mission-critical applications such as SAP HANA, Oracle Applications, and others to meet their goals in terms of performance, availability, and cost. The benefits of virtualization help organizations optimize and utilize resources, but they also include new challenges. Often, organizations that move to virtualized environments are still relying on storage systems that were not designed with the flexibility and ease of deployment and management that a virtualized data center requires. As a result, their application performance is impacted and they are forced to sacrifice efficiency and deployment time for the agility and cost advantages that virtualization offers. To compensate, organizations commonly over-provision storage to address latency and application performance, however this introduces increased costs and does not solve the need for the scalability, speed, and resilience required by virtualized environments. Enterprises need a storage and networking solution that is easy to deploy and user-friendly while also enabling the ability to support application performance and evolve as the environment scales.

Solution Overview HANA Tailored Data Center Integration Scope This solution provides a reference (TDI)-Certified hardware. Definition This solution focuses on virtualized SAP architecture to deploy virtualized SAP and testing of this solution followed HANA with a TDI environment. Best HANA in a VMware environment based the process and best practices as per practices for virtualizing SAP HANA for on VMware vSphere and VMware NSX, Application Workload Guidance (formerly scale-up and scale-out configurations offering customers agility, resource VMware Validated Design). This white are utilized in this design. The solution optimization, and ease of provisioning. paper provides the best practices and is sized, tested, and validated on SAP- This solution enables SAP customers to architecture to deploy the Fibre Channel certified partner hardware for different use provision SAP HANA instances faster SAN used in this solution. cases. This reference architecture provides and more effectively by using vSphere The design details of the SAN solution information specific to SAP HANA and Virtual Machines (VMs). VMware selected is based on Brocade® Gen 6 Fibre leverages existing VMware Validated Brocade, a leading technology partner, Channel products to satisfy the storage Designs for general-purpose guidelines. to provide the products and architecture requirements as specified by this solution. The physical architecture includes Dell for a Fibre Channel-based Storage Area Please refer to reference section at the end R630 and R730 PowerEdge Servers Network (SAN) together with Pure Storage, of this paper for more detailed information with up to 1 TB memory, a Pure Storage another leading technology partner, to about this solution. TDI array, and Brocade SAN and network provide an all-flash high-performance components. storage array. This work involved defining and testing a VMware Software-Defined Data Center-based reference architecture to deploy virtualized SAP HANA on SAP Use Cases Design Overview SAP has published guidelines for This design is targeted for the following A prototypical enterprise SAP customer calculating and sizing the correct storage use cases: with example requirements is used as requirements for SAP HANA TDI a basis for all design activities in this configurations. These guidelines change •• Rapid provisioning to increase timelines solution. Requirements, assumptions, from time to time; therefore, it is always for testing, training, and SAP upgrades and constraints are carefully logged so all recommended to download the latest ––Clones SAP systems quickly and logical and physical design elements can SAP HANA storage requirements white easily be easily traced back to their source and paper. It is also recommended to involve the storage vendor offering an SAP HANA •• Enhanced application availability justification. TDI storage solution for SAP HANA ––High out-of-the-box availability Storage Requirements for Scale-Out configurations to ensure they without complex configuration and SAP S4/HANA have followed the correct space and IP setup to protect against VMware ESXi All SAP HANA worker VMs have a sizing guidelines. server failure; no downtime due to database log, data, root, local SAP, and The SAP HANA nodes connect to the ESXi server maintenance shared SAP volume. The storage capacity storage arrays through a Fibre Channel sizing calculation of these volumes is ––Continuous utilization monitoring SAN. SAN components require a based on the overall amount of memory across resource pools and intelligent minimum 8 Gbps link speed, and the SAN needed by SAP HANA’s in-memory allocation of available resources topology must follow best practices for database. SAP has defined very strict among VMs, based on predefined all redundant components and links. The performance KPIs that should be met rules that reflect customer business connectivity, which includes host HBAs, when configuring a storage subsystem. needs and changing priorities SAN ports, switches, and array front-end With conventional arrays of spinning ––VMware vSphere Fault Tolerance for ports, requires careful planning. 16 Gbps disks, this often resulted in low storage SAP Central Services Fibre Channel (or faster) is recommended utilization in order to achieve required IO for the best performance and all Fibre •• Server consolidation and data center performance and latency. Incorporating Channel ports can negotiate to lower energy cost reduction; runs SAP in VMs all-flash arrays dramatically changes both speeds. consolidated onto fewer physical servers IO performance and latency resulting in that use less energy overall increased demand for a high-performing •• Unified performance monitoring of SAN. the virtualized SAP environment; coordinates performance reporting and analytics across the solution Table 1: Shared Storage Physical Design Specifications. infrastructure stack, including the SAP and database tier, the guest operating Attribute Specification system, the hypervisor layer, and storage Vendor and model Pure M50 Storage Array

•• Enhanced security that leverages Type Active/Active VMware NSX; enables restricted ESXi host multi-pathing policy Round Robin communication between application components Minimum/maximum speed rating 16 Gbps of switch ports Audience This white paper is intended for administrators, architects, and System Engineers who are responsible for designing and deploying infrastructure for enterprise mission-critical applications.

2 Building the Storage Area •• Leverages Brocade Fabric Vision™ SAN Topology Redundancy and Network technology to simplify administration, Resiliency The SAN used for this SAP S4/HANA quickly resolve problems, increase An important aspect of SAN topology is Solution is based on the Brocade G620 uptime, and reduce costs the resiliency and redundancy of the fabric. switch, though it is fully interchangeable The main objective is to remove any single Fibre Channel SAN Design point of failure. Resiliency is the ability of with other switches in the Brocade Considerations Gen 6 Fibre Channel portfolio depending the network to continue to function and/ When designing the SAN for an all-flash on the required scale. The entire Brocade or recover from a failure, while redundancy array, understanding the application Gen 6 portfolio also includes Brocade describes duplication of components, workloads and the intended scalability, Fabric Vision technology, which provides even an entire fabric, to eliminate a single redundancy, and resiliency requirements unprecedented insight and visibility point of failure in the network. Brocade are the main factors to consider. across the storage network. With fabrics have resiliency built into Brocade enterprises migrating to high-performance When deploying a SAN, the main design FOS, the software that runs on all Brocade flash storage for their mission-critical consideration is the adequate sizing of B-Series switches, which can quickly applications, Brocade Gen 6 switches the ISLs between the edge switches “repair” the network to overcome most are best suited to build the Fibre Channel where the servers are connected and failures. For example, when a link between SAN to support applications such as SAP the core switches where the storage switches fails, FSPF quickly recalculates S4/HANA. arrays are connected. In implementations all traffic flows. Of course, this assumes where the storage arrays are deployed that there is a second route, which is Brocade Gen 6 is backward-compatible to serve specific latency- and IO- when redundancy in the fabric becomes with previous generations providing intensive applications such as SAP important. 16 Gbps and 8 Gbps connectivity. While Hana, Oracle and other business-critical this solution is tested with a Pure Storage The key to high availability and enterprise- systems connecting both servers and M50 array connected with 16 Gbps class installation is redundancy. By Fibre Channel storage array to the core speed, there are features listed below eliminating a single point of failure, backbone switch(es) can be advantageous. unique to Gen 6 FC that benefit this business continuity can be provided deployment. When deploying dedicated SANs for Fibre through most foreseeable and even Channel storage array services, the SAN unforeseeable events. At the highest level •• Provides high scalability in an ultra- design can be tailored directly based on of fabric design, the complete network dense, 1U, 64-port switch to support the application workloads and scalability should be redundant, with two completely high-density server virtualization, cloud requirements. separate fabrics that do not share any architectures, and flash-based storage network equipment (routers or switches). environments Storage Performance and Scaling A best practice for SAN infrastructure •• Increases performance for demanding design is to plan for a three- to five-year workloads across 32 Gbps links and lifespan for the solution to be deployed. shatters application performance Considerations for the length of the barriers with up to 100 million IOPS life of a SAN infrastructure include a •• Provides proactive, non-intrusive, combination of: real-time monitoring and alerting of •• Equipment depreciation considerations storage IO health and performance with IO Insight, a Gen 6 platform capability •• The limited predictability of business that provides built-in instrumentation to transformation/development directly measure device IO performance •• Technology refresh and improvements •• Enables VM visibility in a storage fabric Having clear indications or understanding to monitor VM performance, identify of application needs over the same period VM anomalies, and optimize VM determines the range of scalability and performance flexibility necessary in the design. •• Increases resiliency by automatically discovering and recovering from device or network errors

3 Servers and storage devices should be connected to both networks utilizing some Dual-attached form of Multi-Path IO (MPIO) solution that hosts enables data to flow across both networks seamlessly in either an active/active or active/passive mode. MPIO ensures that Redundant if one path fails, an alternative is readily HBAs available. Ideally, the networks would be identical, but at a minimum they should be based on same-switch architecture. In some cases, these networks are in the same location. However, to provide for SAN A SAN B Fabric Fabric disaster recovery, two separate locations are often used, either for each complete network or for sections of each network. Regardless of the physical geography, Redundant there are two separate networks for controllers complete redundancy. In summary, recommendations for the SAN design are to ensure application availability and resiliency via the following: Dual-attached •• Redundancy built into fabrics to avoid a storage

single point of failure Figure 1: Connecting devices through redundant fabrics. •• Servers connected to storage via redundant fabrics •• MPIO-based failover from server to SAN Design Guidelines Brocade Best Practice for Zoning storage Designing SANs fully dedicated with Fibre Zoning is a fabric-based service in SANs •• Redundant fabrics based on similar Channel storage array using the three SAN that groups host and storage nodes need architectures designs reviewed in the previous section to communicate. It only allows nodes is often a straightforward sizing exercise. to communicate with each other if they •• Separate storage and server tiers for Calculate the ISL bandwidth necessary for are members of the same zone. Nodes independent expansion the server density and number of arrays in can be members of multiple zones, •• Core switches of equal or higher each Fibre Channel storage array cluster allowing for a great deal of flexibility when performance compared to the edges combined with the anticipated server and implementing a SAN using zoning. storage growth during the lifecycle of the •• The principal switch defining the highest Zoning not only prevents a host from computing infrastructure. performance in the fabric unauthorized access of storage assets, •• For core-edge designs, a good rule of but it also stops undesired host-to- thumb is to design with ISL bandwidth host communication and fabric-wide equal to total capacity of all array ports Registered State Change Notification which are accessible (zoned) to hosts (RSCN) disruptions. attached to each ToR switch. Brocade recommends that users always •• For collapsed SAN designs, calculate implement zoning, even if they are using the number of host and storage ports for LUN masking. Also, PWWN identification the lifetime of the infrastructure. for zoning is recommended for both security and operational consistency. For details, please refer to the Brocade SAN Fabric Administration Best Practices Guide.

4 Fibre Channel SAN Automation Availability Availability and Reliability Monitoring and Alerts Policy Suite Configuration and Operational This solution is using Fibre Channel SAN- (MAPS): A policy-based monitoring and Monitoring Policy Automation Services based on Brocade Gen 6 technology alerting tool that proactively monitors Suite (COMPASS): An automated with Fabric Vision technology and the health and performance of the SAN configuration and operational monitoring industry-standard 32 Gbps connectivity. infrastructure based on pre-defined policy tool that enforces consistency The key features available in Fabric policies that cover more than 170 of configuration across the fabric and Vision technology and its integration customizable rules, ensuring application monitors changes, simplifying SAN with VMware vRealize suite help build a uptime and availability. Administrators configuration and alerting about changes. scalable and reliable solution architecture desiring a pristine network can set an In medium- to large-sized environments, through automation and high availability. “Aggressive” policy level that has rules this can prevent inadvertent changes to and actions with strict thresholds to switch configurations that may impact the minimize the possibility of data errors. preferred parameters set across the fabric With the ability to tailor the MAPS policies, to optimize performance. organizations can monitor all-flash ports Buffer Credit Recovery: Automatically closely to faster identify any performance Hardware detects and recovers buffer credit loss degradation. Management at the virtual channel level, providing protection against performance degradation and enhancing application availability. Fabric Vision Technology Forward Error Correction (FEC): Automatically detects and recovers Fabric from bit errors, enhancing transmission Software reliability and performance. FEC can reduce latency time significantly by preventing the need to retransmit frames Figure 2: Brocade Fabric Vision Technology: with bit errors. Hardware, Software, and Management working together.

5 Simplified Monitoring of Dashboard: Includes customizable health VM Insight: VM Insight uses standards- Mission-Critical Applications and performance dashboard views, based, end-to-end VM tagging to gain Organizations face a constant struggle of providing all critical information in one VM visibility in a storage fabric. VM Insight both managing data growth and delivering screen. Viewable dashboard “widgets” that enables storage administrators to monitor actionable intelligence from raw data—all should be monitored include errors on VM-level application performance and while meeting SLAs. As a result, even all-flash array facing ports, top 10 flows, set baseline workload behavior. Using this well-managed IT organizations must often memory usage, and port health. information, they can quickly determine whether a storage fabric is the source of make difficult choices about resource IO Insight: Monitors individual host and performance anomalies for applications allocation, weighing the benefits of storage devices to gain deeper insight running on VMs. VM Insight also enables focusing more resources on monitoring, into the performance of the network to fast correlation with other Fabric Vision for instance, and fewer on planning or maintain SLA compliance, obtain total IOs, technology metrics to identify the root optimizing. With Fabric Vision technology, implement first response time, IO latency cause of problems before operations organizations can achieve unprecedented (Exchange Completion Time, or ECT), are affected. VM Insight provides the insight and visibility across the storage and outstanding IOs performance metrics visibility for administrators to provision network through critical monitoring and for a specific host or storage device., It and plan storage networks based on diagnostic capabilities. also diagnoses IO operational issues and application requirements, and to fine-tune enables tuning of device configurations the infrastructure to meet service-level with integrated IO metrics to optimize objectives (see Figure 5). storage performance (see Figure 4).

Brocade Hardware/ Network ASICs Advisor Non-disruptively monitor application ﬂows of interest and obtain device-level IO insight

Fabric Vision Technology

Brocade Fabric OS

Figure 3: Brocade poducts behind Fabric Vision technology.

Figure 4: IO Insight: Monitor storage IO metrics to gain insights into operational issues.

6 Flow Vision: A comprehensive tool that allows administrators to identify, monitor, and analyze specific application and data flows to maximize performance, avoid congestion, and optimize resources. Flow Vision consists of:

•• Flow Monitoring: Monitors specified traffic flows from source to destination through the SAN

•• Flow Generator: Generates traffic between any two ports in a Gen 5 fabric

•• Flow Mirroring: Captures packet data as it flows through the SAN, then displays and analyzes the captured packet’s data Flow Vision is most suited to temporary use while troubleshooting high-latency Figure 5: IO Insight delivering visibility of each VMs IO to shared LUNs. conditions, as continual use results in the collection of substantial amounts of diagnostic data. Flow Vision can be used as needed to verify optimal performance ClearLink Diagnostics: Diagnostic troubleshooting with actionable analytics. for the most demanding applications Port (D_Port) provides loopback test Faster troubleshooting provides time requiring optimal performance. capabilities for link latency and distance to proactively drive the value of IT measurement at the optical and resources. Leveraging Fabric Vision Fabric Performance Impact Monitoring: electrical level to validate the integrity technology provides thorough knowledge Identifies and alerts administrators and performance of optics and cabling, of the behavior and health of Brocade to device or ISL congestion and high ensuring signal and optical quality SAN fabrics over time to identify and levels of latency in the fabric which can and optimal performance across SAN remediate patterns impacting virtual have a severe impact on all-flash array and WAN connections. Pre-validating machine performance and application performance. FPI Monitoring provides the integrity of cables and optics with responsiveness. visualization of bottlenecks and identifies ClearLink prior to deployment identifies For enterprises where deeper application slow-drain devices and impacted hosts potential support issues before they performance insight is necessary, the and storage. occur and enhances the resiliency of high Brocade Application Monitoring Platform performance fabrics. For all-flash array SAN appliance can measure application performance, any impurity in the physical performance and traffic behavior in infrastructure can impact performance. a Brocade Fibre Channel networks. VMware vRealize and Log Insight: Providing actionable intelligence to Brocade has worked closely with VMware maximize performance across the data to offer deeper investigation capabilities center and ensure application delivery for root-cause analysis and remediation SLAs are met. of SANs in virtualized environments. The For more information, see Brocade Brocade SAN Content Pack eliminates Applications Monitoring Platform. noise from millions of events and amplifies critical SAN alerts to accelerate

7 Summary •• Separate vSphere clusters for Acknowledgements Using SAP HANA in TDI deployments management, SAP applications, and Pure Storage SAP HANA databases with SAP and VMware certified high- Krishna Satyavarupu performance storage arrays such as •• vSphere HA and vSphere FT for VMware all-flash storage arrays provides many simplified high availability benefits, including reducing hardware Mohan Potheri •• VMware NSX for networking with micro- and operational costs, lowering risks, and segmentation for security on Brocade Brocade increasing hardware vendor flexibility. But VDX switches Marcus Thordal to benefit from these high-performance Nanjunda Somayaji storage arrays, customers need a high- •• vRealize Automation with Adapter for performance and highly available SAN SAP Landscape Management Resources based on newer generations of SAN •• vRealize Operations with Blue Medora For more information about the SAP technology such as Brocade Gen 6 SAP dashboards for operations and HANA and VMware products and SAN products and deployed using best capacity planning technologies discussed in this guide, practices. This will help customers to review the links and references below: easily transition to this new application The solution was then successfully architecture using SAP HANA. validated to meet and exceed the The Winding Road to Virtual SAP HANA requirements. The results of the validation Application Workload Guidance Design for Following are some of the major tests clearly show that vSphere offers SAP S/4HANA on VMware vSphere 6.5 components used by VMware in their lab a robust platform for running SAP Application Workload Guidance and to architect and test the solution: S/4HANA production workloads. Design for Virtualized SAP S/4HANA on •• vSphere for virtualized compute and vSphere (Part 1, Part 2, Part 3, Part 4) infrastructure Getting Storage Right for Business-Critical ––High-performance Dell R630 and Applications on VMware R730 PowerEdge Servers Fit for the Future ––Pure Storage–based SAP TDI and Western Digital all-flash VMware vSAN The Modernization of Storage storage Architectures ––Brocade Generation 6 SAN fabric Brocade Fabric Vision Technology Brocade G620 Switch - Gen 6 Fibre Channel Switch For more information about Brocade solutions, visit www.brocade.com.

Corporate Headquarters European Headquarters Asia Pacific Headquarters San Jose, CA USA Geneva, Switzerland Singapore T: +1-408-333-8000 T: +41-22-799-56-40 T: +65-6538-4700 [email protected] [email protected] [email protected]

Brocade, the B-wing symbol, and MyBrocade are registered trademarks of Brocade Communications Systems, Inc., in the United States and in other countries. Other brands, product names, or service names mentioned of Brocade Communications Systems, Inc. are listed at www.brocade.com/en/legal/brocade-Legal-intellectual-property/brocade-legal-trademarks.html. Other marks may belong to third parties.

Notice: This document is for informational purposes only and does not set forth any warranty, expressed or implied, concerning any equipment, equipment feature, or service offered or to be offered by Brocade. Brocade reserves the right to make changes to this document at any time, without notice, and assumes no responsibility for its use. This informational document describes features that may not be currently available. Contact a Brocade sales office for information on feature and product availability. Export of technical data contained in this document may require an export license from the United States government.