Running SAN traffic over the LAN is becoming a realistic proposition, but will this future converged network be based on Ethernet, InfiniBand, or Fibre Channel IN DEPTH / STORAGE realistic proposition, but will this future converged network e Channel? By Andy Dornan One Network To Rule Them All ONVERGENCE IS about more iSCSI is a viable option, it’s mostly used in than just voice and data. Storage low-end networks that need speeds of 1 Gbps and networking vendors prom- or less. And even here, many users prefer to ise that the next-generation net- keep the SAN separate from the LAN, even if work will unite local and storage both are built from the same commodity Gi- Carea networks, virtualizing both over a single gabit Ethernet switches. network at 20 Gbps, 40 Gbps, or even 100 The new push toward unified networks dif- Gbps. And it isn’t just the networks that are fers from iSCSI in both its ambition and the coming together.The future SAN also involves resources behind it. Vendors from across the convergence of storage with memory. The networking and storage industries are collabo- same principles that originally abstracted rating on new standards that aim to unite mul- storage out of a local server can be applied to tiple networks into one. Startups Xsigo and RAM, too, while storage targets shift from 3Leaf Systems already have shipped propri- hard disks to flash memory. etary hardware and software aimed at virtual Longtime SAN users might feel a sense of I/O that can converge SAN with LAN, though n a m déjà vu. After all, SAN and LAN convergence not yet memory. i e N was an early claim of iSCSI, which promised The big disagreement is about what this sin- h p o t s to make Fibre Channel unnecessary by rout- gle network will be.This time around, no one is i r h C ing storage traffic over IP networks. While talking about routing everything over IP; Copyright 2008 United Business Media LLC. Important Note: This PDF is provided solely as a reader service. It is not intended for reproduction or public distribution. For article reprints, e-prints and permissions please contact: PARS International Corp., 102 West 38th Street, Sixth Floor, New York, NY 10018; (212) 221-9595 www.magreprints.com/quickquote.asp IN DEPTH / STORAGE the low-latency and low-over- all three types of traffic. head requirements of data center How real is this? 3Leaf networks dictate that the only re- says its chip is in the test alistic choice is to use a fully What It All Means phase, with a version for switched fabric, meaning the con- AMD due to ship before the SERVER VIRTUALIZATION means that verged network will be confined storage networks will be more critical than end of the year and one for to the data center. But will it run ever: Virtual servers need virtual storage. Intel a year later. (As with over Ethernet, Fibre Channel, or multiple CPUs on the same NETWORK CONSOLIDATION is the end, InfiniBand? server, it won’t be possible to I/O virtualization is the means. Uniting SAN This industry isn’t talking con- and LAN into a single fabric can pay big mix AMD and Intel proces- solidation just for consolidation’s dividends. sors together, as each uses a sake.The catalyst is server virtual- different proprietary bus.) MEMORY NETWORKS are the next step ization, for which either a SAN or after storage networks, but few apps will 3Leaf already ships a box that network-attached storage is critical really need them for the foreseeable future. can virtualize Fibre Channel as a way to separate workloads SANs and Ethernet LANs, as INFINIBAND is currently the only realistic from data. Without virtual storage, transport for a converged network that does competitor Xsigo, which the flexibility inherent in virtual- unites memory, storage, and Internet traffic, is backed by Juniper. ization is reduced as data must be but that will change within a year or two. 3Leaf calls its box an I/O moved or replicated whenever a Server, reflecting that it’s 100-GBPS ETHERNET is the long-term virtual machine is set up or torn future, but it could still be a decade or more built from standard PC com- down. Decoupling storage from away. Waiting for it means being left behind. ponents and could be li- processing makes it easier for censed by server vendors; workloads to be moved around. Xsigo calls its product an I/O Virtualization also brings a greater urgency to net- Director to emphasize that it’s a SAN appliance. But ar- work consolidation. A single nonvirtualized server chitecturally, both work in the same way.Virtual NICs would usually have only two network connections: a and HBAs within each server (or each VM) tunnel Eth- network interface card for the LAN and a host bus ernet and Fibre Channel traffic to the box, where it’s adapter for the SAN. In a server running multiple VMs, linked to a SAN and LAN via physical Ethernet and Fi- each virtual server needs its own NIC and HBA, which bre Channel. These aren’t conventional virtual LANs, will quickly become unmanageable unless they’re vir- as the boxes don’t terminate connections or have MAC tual.And if the networks need to be virtualized anyway, addresses. As far as switches on the SAN and LAN are aggregating them together over a single high-perform- concerned, traffic goes straight to the virtual adapter. ance transport will make the data center more flexible. Storage is just the first step. The SAN has been so ETHERNET EVERYWHERE successful that companies from Cisco Systems to Intel- At present, both the Xsigo and 3Leaf appliances re- backed startup 3Leaf see remote memory as the next quire InfiniBand for their physical connections to step, doing to RAM chips what the SAN did for disk servers. It’s an obvious choice: Designed in part for vir- drives. “Most servers have enough compute power, but tual I/O, InfiniBand offers very low overhead and net- they’re constrained by the number of memory slots work latency of less than 100 nanoseconds—comparable available,” 3Leaf CEO B.V.Jagadeesh says. to that of a PC’s local memory bus. Like Fibre Channel, it The slots in a typical server can physically support can scale to 20 Gbps, which is again comparable to the about 32 GB of locally installed memory,whereas 64-bit bandwidth of the AMD and Intel chip interconnects. CPUs and operating systems support 16 TB or more. In InfiniBand is also cheaper than alternatives: A 10- 3Leaf’s envisaged architecture, a CPU in one server is Gbps InfiniBand host channel adapter (HCA) costs connected to other servers that take on roles analogous around $700, compared with at least $1,000 for an Eth- to storage targets. The CPUs are linked through Ad- ernet NIC and more than $2,000 for a Fibre Channel vanced Micro Devices’ Coherent HyperTransport and HBA. Switch port prices show similar variation, though Intel’s QuickPath—technologies developed for intercon- some users may not initially need a switch on the server necting CPUs within a PC, but which 3Leaf says it can side of the box.While switches are necessary for 3Leaf’s extend to run over a network using a special chip that planned memory networks, Xsigo’s I/O Director has plugs into one of the processor sockets on each server. add-on InfiniBand modules that let it connect directly The same network carries one or more virtual SAN or to servers. LAN links, so that only one physical cable is needed for Despite InfiniBand’s advantages, most in the indus- informationweek.com May 18, 2008 28 IN DEPTH / STORAGE try see Ethernet as the long-term future for a single, net’s speed beyond 10 Gbps, though its usual tenfold converged transport. Both Xsigo and 3Leaf plan to sup- speed boost probably isn’t realistic in the short term. port it eventually, as do many larger players. In Febru- “It’ll be double-digit years before we see 100-Gig Eth- ary,Cisco launched the Nexus 7000, a giant data center ernet on the market,” says Koby Segal, COO at InfiniBand switch aimed at consolidating multiple networks into vendor Voltaire. “It’s not just switches, but the whole one. Unlike the startups, Cisco isn’t even bothering to ecosystem of the cables, connectors, and backplanes.” support InfiniBand, though it says it may add Infini- The Ethernet community doesn’t really dispute this. Band modules if there’s enough customer demand. The IEEE’s 802.3ba working group expects to have a The most compelling standard for 100-Gbps argument for moving to DIG DEEPER Ethernet ready by 2010, Ethernet is that everyone THE APP-AWARE NETWORK Switch vendors are making but that doesn’t mean has it anyway.The persist- their devices smarter. IT must decide if this should be feared products will support it at ent trend in networking or embraced. Download this InformationWeek Report at: full rate. In fact, the informationweek.com/1173/report_network.htm has been toward increas- group’s charter calls for ing dominance of Ether- See all our Reports at informationweekreports.com the standard to include two net over other technolo- rates: 40 and 100 Gbps. gies. Though some users are replacing physical Though vendors usually compete to exceed standards, cables with Wi-Fi, that’s really an extension of Ether- technical limitations mean that might not be possible net, not a replacement for it.