SOFTWARE

Linux Edition

COMPONENTS OFED (OpenFabrics Enterprise Distribution) – Driver & Access Layers High performance server and storage connectivity software – OSU MVAPICH and Open MPI* – IP-over-IB for field-proven RDMA and Transport Offload hardware solutions – SDP – SRP Initiator Overview Attain Higher Bandwidth and – iSER Initiator* Use of clustered commodity servers, in lieu of Lower Latency – uDAPL* traditional supercomputers and mainframes, For HPC applications, OFED offers Message – RDS offers tremendous price/performance benefits Passing Interface (MPI) implementation from – Subnet Manager (OpenSM) and unparalleled flexibility in deployment and Ohio State University (OSU) and Open MPI (www. – Installation, Administration and long-term maintenance. To enable distributed open-mpi.org). Bandwidth results in excess of Diagnostics Tools computing transparently, high performance 1400MB/s and application latencies lower than 3 – Performance test suites * Currently not supported by Mellanox computing (HPC) applications require the highest microseconds have been achieved. They include bandwidth and lowest possible latency. In features to enable scaling to large clusters while DEVICES SUPPORT enterprise data center (EDC) applications, these improving on memory usage and latency related requirements are compounded with the need efficiencies. They are optimized and tested for use – InfiniHost HCA Silicon to support a large interoperable ecosystem of with Fortran, C and C++ compilers from popular – InifiniHost III Ex HCA Silicon networking, virtualization, storage, and other vendors. – InifiniHost III Lx HCA Silicon applications and interfaces. – ConnectX HCA Silicon For traditional TCP/IP and sockets-based applica- – Memory & Memory-free HCA Cards The OFED from OpenFabrics alliance (www. tions, OFED includes the field proven implementa- – InfiniScale Switch Silicon openfabrics.org) has been hardened through tion of IP-over-IB (as defined by the IETF, www. – InifiniScale III Switch Silicon collaborative development and testing by all IETF.org), enabling IP-based applications to work major InfiniBand vendors. OFED is supported by seamlessly over InfiniBand and perform at levels BENEFITS Mellanox and major InfiniBand vendors to enable more than three times higher than Ethernet. It OEMs and System Integrators to meet the needs also includes the IBTA (www.IBTA.org) defined – Single software stack that operates across all available InfiniBand devices of HPC and EDC applications. Sockets Direct Protocol (SDP) enabling traditional and configurations such as mem-free, TCP/IP sockets-based applications to capitalize DDR/SDR, PCI-X, and PCI Express modes Sockets Performance Storage Performance – Support for HPC applications for 900 1000 SDP IPoIB Writes Reads scientific research, oil and gas 800 900

exploration, car crash tests, bench 700 800

marking etc. E.g., Fluent, LS-Dyna 700 600 – Support for EDC applications such as 600 500 s /

Oracle 10g RAC, IBM DB2, SAP, and B 500 M

BW MB/s 400 financial applications 400 Max 300 theoretical – Support for traditional IP and Sockets 4G Fiber 300 Channel based applications leveraging the 200 200 benefits of RDMA 100 100 – Support for high performance block 0 0 6 8 1 4 0 2 0 8 6 4 2 2 6 8 4 8 6 4 2 2 6 8 4 2 6 2 4 8 6 2 4 8 5 3 1 3 4 6 8 2 0 5 7 8 4 8 6 3 0 9 9 4 2 1 1 3 6 5 2 1 1 1 1 1 5 2 1 1 3 7 5 7 0 0 0 5 2 storage applications utilizing RDMA 1 7 8 4 2 8 6 2 5 1 4 2 1 9 4 2 6 1 3 6 3 0 0 5 2 1 2 benefits 1 Message Size, B Outstanding Random I/O Request (1MB) – Support for cluster file systems for high performance and virtualized file Figure 1: SDP enables 3x performance improve- Figure 2: SRP performance for random system access ment over Ethernet & TCP/IP Read/Write has 2x performance improvement – Support for cluster management tools over 4G Fiber Channel SOFTWARE OFED (OpenFabrics Enterprise Distribution)

BENEFITS on the RDMA and transport offload capabilities of erability of the OpenFabrics software across InfiniBand. different vendor solutions. OFED, running on Sockets Layer To enable traditional SCSI and iSCSI-based servers, interoperates with InfiniBand switches, - SDP and IP-over-IB component storage applications to enjoy similar RDMA gateways and storage targets and value added enable TCP/IP and sockets-based performance benefits, OFED includes the SCSI software provided by vendors in such systems. applications to interface seamlessly over RDMA Protocol (SRP) initiator and the OFED is also planned to be distributed as part of to and benefit from InfiniBand major OS distributions. OFED supports all transport iSCSI RDMA Protocol (iSER) that interoperate with various target components available in the popular CPU platforms. Access Layer industry. The SRP initiator, for example, interoper- - Supports the OpenFabrics defined Enabling a Large Software Verbs API at the user and kernel ates with the SRP target reference implementa- Ecosystem levels. User level verbs allow tion from Mellanox, which is open source and Through its collaboration with the OpenFabrics protocols MPI and other applications available freely from www.openfabrics.org. SRP community, support for multiple industry-standard to interface to Mellanox InfiniBand solutions over InfiniBand have been proven to hardware. Kernel levels verbs interfaces that enable off-the-shelf applications deliver impressive 910MB/s (random read) and allow protocols like SDP, SRP and to work with InfiniBand, and support for popular IP-over-IB to interface to Mellanox 725MB/s (random write) I/O storage performance OS and CPU platforms, Mellanox has created InfiniBand hardware with 1MB blocks. a large, interoperable software ecosystem that SCSI Mid Layer OFED is interoperable across multi- benefits the HPC and EDC markets. - The SCSI Mid Layer interface vendor vendor solutions enables SCSI based block storage and management applications to As a founding member of OpenFabrics, previously interface with the SRP Initiator known as OpenIB, Mellanox is driving interop- component and the Mellanox InfiniBand hardware HPC Embedded Data Center Storage Management

HPC Applications Life Sciences Back-End, App/MiddleWare, Front-End Block Cluster Applications Storage Management IBADM

User MPI *UDAPL Diagnostics OpenSM libsdp uverbs API Sockets Layer umad API FS Kernel TCP UDP ICMP SCSI Kernel IP Mid Layer Bypass RDS SDP IPoIB SRP *iSER

verbs API core (verbs, mads, sa, cm, uverbs, umads)

mthca (HCA Driver)

HCA Hardware

Markets Linux InfiniBand Fabric Applications OFED Linux Modules * Currently Not Supported in OFED by Mellanox

Platform Supplier Products and Support RedHat AS 4.0 – Update 2 and Update 3, Fedora Core 4 Novell SLES 9 – SP3, SUSE 10 Pro, SLES 10 x86, IA64, EM64T, EM64T x86_64, PCI-X and PCIe platforms AMD AMD64 and AMD64-Ex Opteron, PCI-X and PCIe platforms

2900 Stender Way, Santa Clara, CA 95054 Tel: 408-970-3400 • Fax: 408-970-3403 www.mellanox.com

© Copyright 2008. . All rights reserved. Mellanox is a registered trademark of Mellanox Technologies, and ConnectX, InfiniBlast, InfiniBridge, InfiniHost, InfiniRISC, InfiniScale, and InfiniPCI are trademarks of Mellanox Technologies. 2492PB Rev 1.1