Unified Networking on 10 Gigabit Ethernet Intel and NetApp provide a simple and flexible path to cost- effective performance of next-generation storage networks. EXECUTIVE SUMMARY CONTENTS Unified networking over 10 Gigabit Ethernet (10GbE) in the data center Executive Summary . .1 offers compelling benefits, including a simplified infrastructure, lower Simplifying the Network with 10GbE . 2 equipment and power costs, and the flexibility to meet the needs of the Evolving with the Data Center . .2 evolving, virtualized data center. In recent years, growth in Ethernet-based The Promise of Ethernet Storage . .3 storage has surpassed that of storage-specific fabrics, driven in large part by Ethernet Enhancements for Storage . 4 the increase in server virtualization. The ubiquity of storage area network (SAN) access will be critical as virtualization deployments continue to Data Center Bridging for Lossless Ethernet .................. 4 grow and new, on-demand data center models emerge. Fibre Channel over Ethernet: With long histories in Ethernet networking and storage, Intel and NetApp Extending Consolidation............. 4 are two leaders in the transition to 10GbE unified networking. This paper Contrasting FCoE Architectures....... 4 explores the many benefits of unified networking, the approaches to Open FCoE: Momentum enabling it, and the pivotal roles Intel and NetApp are playing in helping in the Linux* Community . 4 to bring important, consolidation-driving technologies to enterprise data Intel Ethernet Unified Networking . 5 center customers. Reliability . 5 Cost-Effectiveness . 5 Scalable Performance .............. 6 Ease of Use ...................... 6 NetApp Ethernet Storage: Proven, I/O Unification Engine . .6 Conclusion . 7 Solution provided by: SIMPLIFYING THE As server virtualization continues to 16 times the network capacity, and over 20 NETWORK WITH 10GBE grow, 10GbE and unified networking times the compute capacity by 2015.4 A are simplifying server connectivity. new infrastructure must emerge to power As IT departments look to reduce Consolidating the traffic of multiple GbE this growth and enable the most efficient costs and improve server efficiency, connections onto a single 10GbE adapter use of resources; this is cloud computing. they are turning increasingly to server significantly reduces cable and infrastructure The cloud is an evolution of computing virtualization and consolidation. The complexity and overall TCO. Recent that delivers services over the Internet benefits of virtualization are widely enhancements to the Ethernet standard to consumers and enterprises. Services understood: less server hardware to enable 10GbE support for both LAN and scale—as needed and only when needed— purchase, lower power and cooling SAN traffic, allowing IT to realize further without user intervention. A highly scalable benefits by converging data and storage and efficient cloud architecture is needed needs, and centralized management. infrastructures. Thanks to its ubiquity, cost to provide both the technical attributes Today’s servers are based on powerful effectiveness, flexibility, and ease of use, and the extreme resource utilization and new processors, including the Intel® Ethernet has emerged as the unified data efficiency cloud computing promises. Xeon® processor 5600 and 7500 center fabric. With its reduced hardware requirements, series, that support more virtual EVOLVING WITH THE fewer points of management, and broad machines (VMs) per physical host DATA CENTER ecosystem support, 10GbE delivers the than ever before, helping IT realize flexible, simplified network infrastructure greater consolidation ratios. The growth in server virtualization has needed to support cloud computing. These helped data center networks evolve from key characteristics make 10GbE the ideal • The latest generation of Intel® Xeon® discrete, siloed infrastructures to more fabric for cloud infrastructures: processors enables IT to consolidate servers at a 15:1 ratio, delivering power savings of up to 90 percent and a five- Over 2.5 billion users will connect to the Internet in the month return on investment. next five years2 with over 10 billion devices.3 This usage • New four-socket processors are will require eight times the amount of storage capacity, 16 delivering 20 times the performance of previous-generation processors. times the network capacity, and over 20 times the compute • Nearly 50 percent of the four-socket capacity by 2015.4 A new infrastructure must emerge to servers shipped today are being used power this growth and enable the most efficient use of for virtualization.1 Unfortunately, the success achieved by many resources; this is cloud computing. organizations in attaining these benefits has been limited by the complications flexible fabrics with the scalability and Ubiquity. Ethernet connectivity ships of networking virtualized servers. As agility necessary to address the needs of standard on nearly every server today, and VM density increases, a physical server’s new usage models and provide an excellent Ethernet infrastructures are a universal data networking needs also increase, adding both foundation for enterprise cloud computing. center component. 10GbE products have cost and complexity. A typical virtualized been available for several years, but when server uses eight to 10 GbE local area Over 2.5 billion users will connect to the 10GbE LAN on motherboard (LOM) 2 network (LAN) ports and two dedicated Internet in the next five years with over connections are integrated in the next 3 SAN ports. 10 billion devices. This usage will require generation of servers, true unified LAN and eight times the amount of storage capacity, SAN connectivity will be available by default. 2 Unified Networking for 10 Gigabit Ethernet Advanced virtualization support. THE PROMISE OF Given its flexibility and long history, it Advanced server virtualization enables ETHERNET STORAGE is not surprising that Ethernet storage dynamic resource allocation and is required is the fastest growing segment of the New usage models and the explosive for any cloud computing infrastructure. storage systems market. The industry growth of data in their organizations 5 Technologies from companies such as Intel research firm IDC estimates that the have forced IT administrators to deal and NetApp, and hypervisor vendors are worldwide Ethernet-based storage systems with complicated technical and business delivering line-rate 10GbE throughput (NAS and iSCSI SAN) market grew at a challenges. Today, most IT departments compounded annual growth rate (CAGR) and support for platform virtualization deploy separate LAN and storage networks, of approximately 23 percent between enhancements. with storage often divided between NAS 2005 and 2009. In comparison, unit Unified networking. A 10GbE unified and SAN, which requires multiple data shipments for FC storage shipments grew fabric simplifies the network infrastructure recovery solutions, a variety of data at approximately 10 percent CAGR during by consolidating LAN and SAN traffic. management models, and, potentially, the same period. different support teams. The goal of unified Internet small computer system interface iSCSI storage shipments experienced the networking is to allow a single fabric— (iSCSI) and network file system (NFS) highest growth rates (70 percent) during Ethernet—to carry these disparate are examples of storage protocols that use 2005–2009, driven by broad iSCSI traffic types. Ethernet, and the recent ratification of adoption in Microsoft Windows*, virtual the Fibre Channel over Ethernet (FCoE) Ethernet has served as a unified data server, and blade server environments. NAS standard extends this capability. Recent center fabric for years, supporting LAN, unit shipments grew at a 14 percent CAGR Ethernet enhancements ensure quality of NAS (NFS, common Internet file system during this time period due to continued service (QoS) for critical traffic. (CIFS)) and iSCSI SAN traffic. With NAS deployments for business-critical recent Ethernet enhancements and the database applications and large-scale virtual Intel and NetApp are two companies ratification of the FCoE specification, server deployments. helping to advance the shift to 10GbE- standard Ethernet adapters can now based unified networking in the data As shown in Figure 1, Ethernet storage unit connect servers to Fibre Channel (FC) center. The latest Intel® Ethernet 10 Gigabit shipments surpassed FC storage shipments SANs. Extending Ethernet’s inherent controller and server adapters include in 2008. Industry analysts project continued advantages, including proven reliability, gains in the Ethernet storage market share virtualization optimizations and advanced ubiquity, and wide familiarity, to FC due to increasing deployment of “Ethernet unified networking features, including SAN traffic will help accelerate the move only” data centers (which use a unified optimizations for lossless Ethernet, to 10GbE-based I/O consolidation in 10GbE infrastructure for all data and storage intelligent, hardware-based accelerations virtualized data centers, reduce costs, and traffic), the emergence of cloud computing, for FCoE and iSCSI, and support for Open improve simplification and agility. and as FCoE solutions enter the mainstream. FCoE, which is discussed later in the paper. NetApp is a leader in supporting Ethernet Ethernet versus Fibre Channel Target Unit Shipments storage, first as a pioneer for network area storage (NAS), next as an early 300,000