25Gb : The new standard for data center connectivity Accelerate network performance and reduce costs of server-to-switch connections

Data center requirements are growing at double- Networking at the crossroads digit rates. On the demand side, data centers are expanding As organizations push more traffic into their enterprise and with a 20 percent growth in servers, and virtualization private data centers, and with bigger “pipes” coming into the density is increasing by 30 percent,1 driving increased enterprise, more bandwidth is also required as data moves down bandwidth requirements. On the supply side, multi-core into the fabric or ToR. ToR architectures utilizing 10GbE to the processing power, NVMe and PCIe 3.0 I/O buses, and high- server are among the largest number of connections in data speed flash storage require networking that can keep pace centers today. But these architectures are rapidly outgrowing with increased throughput. 10GbE capabilities, often requiring multiple ports.

To meet these needs, networking and the Ethernet industry Organizations would need to deploy twice as many 10GbE are moving in a new direction. Discussions with enterprise switches to keep up with required network bandwidth, along customers that previously centered on 10GbE, 40GbE and with additional cables, space, power and cooling. That would 100GbE networking have expanded to include 25GbE and 50GbE mean a significant increase in capital and operating expenses. technology as well. In particular, many of these discussions The traditional next step up to meet demands, 40GbE, is center on moving from 10GbE to 25GbE for top-of-rack (ToR) not cost-effective or power-efficient enough for many switching. This paper explains why, and shows how changes in organizations. Moreover, as organizations look ahead, there is demand, technology and networking economics are working no simple, economical path from 10GbE to 40GbE to 100GbE — together to alter the networking landscape. it requires organizations to make three product transitions. To better address high-performance number of lanes as 10GbE. That means ToR network and scalability requirements, 25GbE switches can be manufactured industry leaders including and very cost-effectively. Using a single lane Switch Dell are increasingly implementing on a switch or adapter, 25GbE ports also ASIC 25GbE for their customers. These compare favorably to 40GbE technology implementations capitalize on the 25GbE that requires four lanes. 32-port 100GbE specification adopted by the 25 Consortium. The specification Deploying 25GbE networks also enables makes use of single-lane 25 Gbps organizations to significantly reduce Ethernet links and is based on the the required number of switches and existing IEEE 100GbE standard (802.3bj). cables — along with space, power and cooling — compared to similar bandwidth

25GbE breakout cable 25GbE breakout Server With the adoption of the 25GbE solutions using 10GbE and 40GbE specification, a new generation of switch technology. Fewer network components and adapter chips is becoming available. reduce ongoing management and cable 25GbE breakout In the last few years, the underlying maintenance costs while more efficiently high-speed communication technology utilizing servers already in place. Most Server has advanced to the point that 25 GHz important, with 25/50/100GbE multi- signaling interfaces have become rate connectivity, enterprises now have economically viable, and all of the various a path to 100GbE that is nondisruptive, Figure 2. Breakout of four 25GbE physics-related challenges in signal can be implemented today and provides connections per 100GbE port integrity have found reliable solutions. economical steps along the way. The impact of ASIC development The economics of 25GbE Considering the economics, it is no A 100GbE switch previously required The availability of 25GbE switches and surprise that the move to 25GbE (and multiple application-specific integrated adapters has changed the economics of 50GbE) is accelerating — a recent circuits (ASICs) because the number upgrading server-to-switch connectivity. five-year forecast by industry analyst of ports per ASIC were limited. The New 25GbE chips use the same class Dell’Oro Group predicts 25GbE will new generation of 100GbE switch of cables as 10GbE but deliver 2.5 times deliver a significant amount of Ethernet implementations is based on high- the performance, while using the same bandwidth by 2018 (see Figure 1). density, industry-standard ASICs that can support up to 32 ports of 100GbE or 128 breakout ports of 25GbE in a single chip, Speed migration on servers—total market a huge leap forward. With the QSFP28 (Included in Dell’Oro Group’s server report) form factor, the four lanes can be used to enable four 25GbE server connections 50 Gigabit Ethernet from each 100GbE port (see Figure 2). Gigabit Ethernet 40 Gigabit Ethernet 100 Gigabit Ethernet The move from 10 lanes to 4 100% Until recently, a majority of available 100GbE implementations used 10 lanes of 10GbE.

Using four-lane variants such as QSFP28 is more economical in several ways:

• Recent innovations make four 25 Gbps lane less expensive than ten 10 Gbps lanes because the is simpler and less costly to manufacture. Percent of server shipments 0% • The power required to run that transceiver is much less than required for a typical 10-lane transceiver. Figure 1. Projected growth of 25GbE and 50GbE networking2

2 • Short run (150M or less) multi-mode Partnering for your success About Dell fiber (MMF) cabling runs are also Enterprise organizations continue to push Dell integrates QLogic connectivity less expensive because instead of the envelope as they deploy the latest into Dell Networking switches, requiring 10 transmit (TX) and technologies to keep up with exponential PowerEdge servers and other solutions. 10 receive (RX) fiber strands, only data growth. As members of the 25 Gigabit Dell listens to customers and delivers 4 pairs of fiber for transmit and Ethernet Consortium, QLogic and Dell worldwide innovative technology, receive are needed. are dedicated to bringing the benefits of business solutions and services that • Moving from 10GbE to 40GbE the new technology to their customers. give them the power to do more. typically requires a forklift upgrade QLogic was the first to demonstrate an to thicker, more expensive cables, end-to-end 25GbE solution, and Dell but a 25GbE direct attach copper introduced the industry’s first high-density, connection does not. 100GbE multi-rate switch in a 1RU fixed- • For fiber connections, moving from form factor for aggregation and access 10GbE to 40GbE may require an layers (see sidebar on Dell Networking At a glance: Reshaping data upgrade to four times the number of switches, next page). Both Dell and QLogic center networks fibers (MPO), but a 25GbE connection 25GbE technologies are backward- Factors enabling 25GbE does not because it is the same as compatible with 10GbE, allowing advances include: 10GbE (single TX and single RX, not customers to build and cross-connect a four each for TX and RX). heterogeneous-speed Ethernet network. New silicon • ASIC development now allows The new standard for ToR upgrades Considering the uses and benefits, new high-density 100GbE switches New single-lane 25GbE is not only switches based on single-lane 25GbE in affordable form factors economical compared to 40GbE, it also technology represent an important • 32x100GbE switching bandwidth provides more available bandwidth than step forward in the performance and is now available on a single chip 40GbE (see table). While 25GbE may economics of networking: two-and-a- • The new switches offer the seem like a step back from 40GbE, it half times the performance of existing ability to run 4x25GbE off of a actually delivers more. 10GbE, with greater port density and 100GbE port lower costs than 40GbE. The 25GbE 25GbE is an easier upgrade path from technology empowers organizations to New optics 10GbE as it fits into the existing model cost-effectively scale network bandwidth • MMF optics have been and requires half the number of PCIe in support of next-generation server reduced to 4 RX and TX fibers lanes compared to 40GbE, leading solutions in data center environments. (100GBASE-SR4) to better PCIe bandwidth utilization • New 100GbE optical modules and lower power consumption. The About QLogic use a x4 electrical interface 25GbE physical interface specification QLogic is a global leader and technology • These optical modules require also supports a variety of form factors, innovator in high-performance server and less power than currently allowing flexible configuration options. storage networking connectivity products. comparable solutions

Example: Ethernet switch with 3.2 Tbps capacity and 128 ports New economics • 25GbE reduces CapEx with Port Lane speed Lanes per Usable Total BW fewer ToR switches at lower speed (Gbps) port ports (Gbps) cost than 40GbE • It also reduces OpEx because 10GbE 10 1 128 1280 25GbE switches require less power, cooling and space 25GbE 25 1 128 3200 • 25 Gbps Ethernet adapters adhere to standards, helping 40GbE 10 4 64 2560 protect technology investments 100GbE 25 4 32 3200

Bandwidth comparison for 25GbE and other Ethernet speeds

3 Dell Networking Z9100-ON Dell Networking S6100-ON QLogic FastLinQ 100GbE fabric switch multi-rate switch QL4521x-DE Series Next-generation fixed-form factor Fully modular 2RU fixed-form factor 25 Gbps Ethernet-to-PCIe 3.0 10/25/40/50/100GbE 10/25/40/50/100GbE switch Intelligent Ethernet Adapters

Industry’s first 100GbE multi-rate Industry’s first multi-rate modular 1RU switch in-rack switch • Multi-rate switching with 32 ports • 3 unique modules providing 100GbE (QSFP28), 32 ports additional multi-rate flexibility: With superior price/performance 40GbE, 128 ports 1GbE, 128 ports — 16x40GbE QSFP+ enabling versus 10GbE, the powerful ® ™ 25GbE or 64 ports 50GbE high-density 40GbE and up to QLogic FastLinQ QL4521x-DE • Additional 2 fixed 1/10GbE 64 ports QSFP+ in 2RU Series is optimized for use across SFP+ ports enterprise, managed service — 8xQSFP28 ports, allowing cost- provider (MSP) and large public efficient 10/25/40/50/100GbE cloud deployments. —4xQSFP28 and 4xCXP, bringing both cost-effective multi-rate • Leverages QLogic fifth-generation switching and CXP interconnects technology to deliver true for legacy 100GbE 25 Gbps Ethernet performance • Up to 32x100GbE, 64x40GbE, • Increases virtual machine 64x50GbE, 128x25GbE, 128x10GbE density and accelerates multi- or 128x1GbE tenant networks with full offload for tunneling protocols • Enables organizations Both the Dell Networking Z9100-ON and S6100-ON switches are built to to achieve new levels of support future-ready, high-end data center applications and a range of switching performance in physical, speeds that provide flexibility for the life of the investment. Both switches virtual and cloud environments also support third-party operating systems and the Open Networking Install Environment (ONIE). Integrated, advanced networking helps eliminate I/O bottlenecks, conserve CPU cycles and enable greater virtual machine density in Learn More To learn more about QLogic 25GbE the next generation of software- For more information about Dell 25GbE technology, contact your QLogic defined data centers. switches, contact your Dell representative representative or visit: www.qlogic.com/ or visit: Dell.com/networking Products/adapters/Pages/25Gb- Ethernet.aspx

1 Infonetics Research: Data Center Strategies North American Enterprise Survey, May 2015 2 Dell’Oro Group: Server Report, March 2016.

© 2016 Dell, Inc. ALL RIGHTS RESERVED. No part of this document may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording for any purpose without the written permission of Dell, Inc. (“Dell”).

Dell, the Dell logo and products — as identified in this document — are registered trademarks of Dell, Inc. in the U.S.A. and/or other countries. All other trademarks and registered trademarks are property of their respective owners.

THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES. THE CONTENT IS PROVIDED AS IS, WITHOUT EXPRESS OR IMPLIED WARRANTIES OF ANY KIND.

4 White Paper - 25Gb Ethernet: The new standard for data center connectivity - US - TDA - 2016-04-19