Choosing the Best Network Interface Card Mellanox Connectx®-3 Pro EN Vs

COMPETITIVE BRIEF

August 2014

Choosing the Best Network Interface Card Mellanox ConnectX®-3 Pro EN vs. Intel X520

Introduction: How to Choose a Network Interface Card...... 1 Comparison: Mellanox ConnectX®-3 Pro EN vs. Intel X520...... 2 Technology ...... 2 Performance...... 2 Acceleration for the Cloud...... 3 Power Consumption...... 5 Return on Investment...... 5 Bottom Line...... 6

Introduction: How to Choose High performance connectivity is required by everyone these days, whether in enterprise data centers, a Network Interface Card cloud computing environments, or Web 2.0 installations. The difference between a successful deployment and one plagued with poor performance is often a matter of the underlying interconnect technology. The Network Interface Card (NIC) is therefore a crucial piece of the puzzle when building a high-performance data center. In choosing one NIC over another, there are various factors to consider. Does the NIC address the needs of your application and market? It is important to make sure that your NIC has a wide enough range of features that accelerate your application and offload your CPU, providing more room for compute and virtualization. What is the technology embedded in the NIC, and what advantages does it provide? The available bandwidth, the ability to support both bare metal and virtual workloads, and specific application offloads are all important features of the NIC technology. A scalable and high performing NIC will have a wider range of application benefits and a longer life span because it can address the changing needs of a data center. Which NIC provides superior performance? Ultimately, a high performaning data center’s networking is largely dependent upon a highly performing NIC. Raw bandwidth, low latency, and application-specific performance are some of the parameters to compare. Power consumption is another factor. Power consumption is a major expense in the data center today. In a large data center with hundreds or thousands of servers, power consumption can become a major drain on profitability, so it is important to consider a NIC that can reduce such an expense. The overall return on investment (ROI) should also be calculated. Cost and performance should be weighed against one another to determine the actual Total Cost of Ownership (TCO).

Comparison: Mellanox Given the aforementioned factors in choosing the best NIC, we compared two of the leading Ethernet ConnectX®-3 Pro EN NICs in the marketplace to see which addresses these concerns better: Mellanox’s ConnectX®-3 Pro EN vs. Intel X520 and Intel’s X520.

Technology When it comes to bandwidth, ConnectX-3 Pro EN offers 10, 40 and 56 Gb/s Ethernet, while the X520 only reaches the 10 GbE level. This constitutes a significant difference in capability. Furthermore, because the ConnectX-3 Pro EN uses a PCIe Generation 3.0 bus, it can handle as many as 8 GT/s (Giga-transfers per second), compared to only 5 GT/s with the X520’s PCIe Gen 2.0 bus. Mellanox’s ConnectX family of adapters also supports RDMA over Converged Ethernet (RoCE), which enables zero-copy data transfers and reduces CPU overhead tremendously. Moreover, ConnectX-3 Pro EN enables hardware offloading via TCP stateless offloads even for overlay networks such as NVGRE and VXLAN, in order to further free the CPU for other activities. Neither RoCE nor hardware offloading for overlay networks is available in the X520. ConnectX-3 Pro EN also includes congestion control features (QCN), which ensure that the maximum bandwidth can be passed efficiently during congestion events. Again, this is lacking in the X520.

Finally, for customers seeking the ultimate in data center simplicity and peace of mind, the ConnectX family is but one piece of Mellanox’s end-to-end suite of interconnect products, including switches, cables, and management software. Mellanox even offers a version of ConnectX-3 Pro EN that uses Virtual Protocol Interconnect® (VPI®), a proprietary Mellanox technology that facilitates the conversion of a device from Ethernet to InfiniBand or vice versa. This guarantees that the customer can grow the network based on the latest requirements instead of based on legacy equipment. The X520, on the other hand, is a standalone offering with little flexibility for current data center integration or for future growth.

Table 1. Technological Features

Infrastructure Mellanox ConnectX-3 Pro EN Intel X520 82599 Controller Network Ports 10 / 40 / 56GbE 1 / 10GbE Ethernet and VPI options Host Interface PCIe 3.0 x8 PCIe 2.0 x8 8GT/s 5GT/s Power Consumption 3.8W 6.5W (IC on 10GbE Board) RDMA RoCE No Latency 0.8us (RDMA) 12us (TCP) 6us (TCP) Overlay Networks VXLAN, NVGRE hardware offloads No Congestion Control QCN (L2) No

Conclusion: From a technological standpoint, ConnectX-3 Pro EN is at least one generation ahead of X520, and it can therefore better address the specific needs of the applications and markets that require high performance interconnect.

Performance Naturally, with higher bandwidth and a more advanced bus comes higher throughput. ConnectX-3 Pro EN reaches a maximum throughput of 38 Gb/s, while X520 can only achieve 9.5 Gb/s.

Figure 1. Maximum bandwidth

But even comparing only at 10GbE speed, ConnectX-3 Pro EN provides better results than the Intel X520 by way of low latency. ConnectX-3 Pro EN shows 2X better latency results than X520, even before incorporating RoCE. With RoCE support, ConnectX-3’s latency numbers are 15 times better than the X520’s.

Figure 2. Latency on 10GbE cards, with and without RDMA

Acceleration for the Cloud When overlay networks (NVGRE and VXLAN) are in use, there is added value in enabling hardware offloading for these protocols to improve utilization of the CPU. ConnectX-3 Pro EN supports this feature, but X520 does not.

Figure 3. 10GbE throughput of VXLAN traffic

Figure 4. CPU Utilization per 1Gb/s with and without NVGRE hardware offloading

Figure 5. CPU Utilization per 1Gb/s with and without VXLAN hardware offloading

Conclusion: ConnectX-3 Pro EN offers a significant performance advantage over Intel’s X520, and it offers additional features that provide an even greater performance boost. Cloud and other environments that prefer to use overlay network protocols such as VXLAN and NVGRE will receive much higher bandwidth per server. Offloading the CPU also enables a much better ratio of VMs per server to the cloud administrator with Mellanox ConnectX-3 Pro EN versus the competition.

Power Consumption Not only does ConnectX-3 Pro EN offer higher performance, but it provides it while also consuming less power than the competition.

Table 2. Power features

Mellanox ConnectX-3 Pro Mellanox ConnectX-3 Pro Intel X520 Dual-Port SFP+ Single-Port QSFP Dual-Port SFP+ Speed 20 40 20 Ports 2 1 2 PCIe x8 Gen3 x8 Gen3 x8 Gen2 Power Consumption1 3.74W 5.32W 4.5W Power per 1Gb 0.187 0.133 0.225 (W/Gb)

Figure 6. Watts per 1Gb/s of supported speed

Conclusion: When power savings are multiplied across hundreds or thousands of NICs, ConnectX-3 Pro EN becomes the clear choice thanks to its more efficient consumption for similar (or even greater) bandwidth.

Return on Investment We have already shown ConnectX-3 Pro EN’s superiority over X520 in bandwidth and latency, but performance alone is not enough to consider. It is also worthwhile to compare the two cards with regard to how much performance they provide per dollar.

______1 Data is collected from Intel and Mellanox specifications and datasheets. Power consumption is based only on the adapter. It does not include cooling, installation, and other tangential factors.

According to the US Energy Information Administration, the average retail price for industrial consumers in April 2014 was 6.75 cents per Kilowatthour. By extrapolating the average power consumption numbers over the course of a year and multiplying by this average retail price, it is possible to see the ongoing cost of each adapter card. However, more important than the cost is the return on investment (ROI). The NIC that provides the highest performance for your money is the card that provides the most value.

Figure 7. Throughput per dollar of power consumption

ConnectX-3 Pro EN has a clear advantage in terms of throughput per dollar. In fact, by choosing the Mellanox card over the Intel one, you not only receive vastly superior performance and save on power consumption; you also receive even greater savings in the long-run. Because ConnectX-3 Pro EN is already at least a generation ahead in bandwidth, it is already future-proofed; when the demands of the data center or cloud increase, there is no need to upgrade the interconnect. Conclusion: ConnectX-3 Pro EN is not only the better NIC on performance; it is also the better investment financially.

Bottom Line Mellanox ConnectX-3 Pro EN is a better NIC than Intel’s X520 on all counts and for all the main use cases. Whether for HPC, cloud, Web 2.0, storage, or data center, ConnectX-3 Pro EN is the leading choice to ensure successful high-performance deployments. When comparing on technology, performance, power consumption, and return on investment, ConnectX-3 Pro EN is the clear leader across the board.

350 Oakmead Parkway, Suite 100, Sunnyvale, CA 94085 Tel: 408-970-3400 • Fax: 408-970-3403 www.mellanox.com © Copyright 2014. Mellanox Technologies. All rights reserved. 15-4193CB Mellanox, Mellanox logo, ConnectX, and Virtual Protocol Interconnect are registered trademarks of Mellanox Technologies, Ltd. All other trademarks are property of their Rev 1.1 respective owners.