OFED (Openfabrics Enterprise Distribution)
Total Page:16
File Type:pdf, Size:1020Kb
SOFTWARE Linux Edition COMPONENTS OFED (OpenFabrics Enterprise Distribution) – Driver & Access Layers High-performance server and storage connectivity software – SRP Target – OSU MVAPICH and Open MPI for field-proven RDMA and Transport Offload hardware solutions – IP-over-IB – SDP Overview includes the IBTA (www.InfiniBandTA.org) defined Sockets Direct Protocol (SDP) enabling traditional – SRP Initiator Use of clustered commodity servers, in lieu of TCP/IP sockets-based applications to capitalize – iSER Initiator* traditional supercomputers and mainframes, – uDAPL* on the RDMA and transport offload capabilities offers tremendous price/performance benefits – RDS of InfiniBand. Reliable Datagram Services (RDS) and unparalleled flexibility in deployment and – Subnet Manager (OpenSM) provides an efficient sockets-based interface for long-term maintenance. To enable distributed – Installation, Administration and Oracle RAC database applications. computing transparently, high-performance Diagnostics Tools To enable traditional SCSI and iSCSI-based – Performance test suites computing (HPC) applications require the highest storage applications to enjoy similar RDMA * Currently not supported by Mellanox bandwidth and lowest possible latency. In enterprise data center (EDC) applications, these performance benefits, OFED includes the SCSI DEVICES SUPPORT requirements are compounded with the need over RDMA Protocol (SRP) initiator and target, – InfiniHost HCA Silicon to support a large interoperable ecosystem of and the iSCSI RDMA Protocol (iSER) that interop- – InifiniHost III Ex HCA Silicon networking, virtualization, storage, and other erate with various target components available – InifiniHost III Lx HCA Silicon applications and interfaces. in the industry. SRP solutions over InfiniBand have been proven to deliver impressive 910MB/s – ConnectX HCA Silicon The OFED from OpenFabrics Alliance (www. (random read) and 725MB/s (random write) I/O – Memory & Memory-free HCA Cards openfabrics.org) has been hardened through storage performance with 1MB blocks. High – InfiniScale Switch Silicon collaborative development and testing by all Availability (HA) Support for Mission Critical – InifiniScale III Switch Silicon major InfiniBand vendors. OFED is supported by Deployments Mellanox and major InfiniBand vendors to enable BENEFITS OEMs and System Integrators to meet the needs High Availability (HA) Support for – Single software stack that operates of HPC and EDC applications. Mission Critical Deployments across all available InfiniBand devices Higher I/O Service Levels for all OFED 1.3 includes high availability support for and configurations such as mem-free, sockets and storage applications. Specifically, DDR/SDR, PCI-X, and PCI Express Data Center Traffic and Applica- modes tions the standard Linux channel bonding module is supported over IPoIB enabling failover across – Support for HPC applications for For HPC applications, OFED offers Message ports on the same HCA, or across HCAs. Simi- scientific research, oil and gas Passing Interface (MPI) implementation from exploration, car crash tests, bench larly, standard multipathing and failover over the Ohio State University (OSU) and Open MPI marking etc. E.g., Fluent, LS-DYNA SRP Initiator is supported for SCSI applications (www.open-mpi.org). Bandwidth results in – Support for EDC applications such as using standard Linux implementations such as Oracle 10g RAC, IBM DB2, SAP, and excess of 1400MB/s and application latencies device mapper multipath (dm-multipath) driver. financial applications lower than 3 microseconds have been achieved. Some vendor-specific fail-over/load-balancing – Support for traditional IP and Sockets They include features to enable scaling to large driver models are supported as well. based applications leveraging the clusters while improving on memory usage and benefits of RDMA latency related efficiencies. Ready for Unified I/O Deployments – Support for high-performance block storage applications utilizing RDMA For traditional sockets-based applications, OFED with Granular Quality of Service benefits includes the field proven implementation of (QoS) – Support for end-to-end QoS using IP-over-IB, enabling IP-based applications to work OFED QoS features enable data center managers centralized QoS manager for delivering seamlessly over InfiniBand and perform at levels to define and deploy QoS policies end-to-end by no compromise unified I/O services more than six times higher than Ethernet. It also traffic type using a centralized QoS manager. All – Support for cluster management tools SOFTWARE OFED (OpenFabrics Enterprise Distribution) BENEFITS network nodes, including servers, switches and erability of the OpenFabrics software across storage end-points, are part of coherent QoS different vendor solutions. OFED, running on Sockets Layer policies, delivering true QoS in a reliable, loss- servers, interoperates with InfiniBand switches, - SDP and IP-over-IB component less medium. Centralized management reduces gateways and storage targets and value added enable TCP/IP and sockets-based management effort and enables faster definition software provided by vendors in such systems. applications to interface seamlessly and deployment of QoS policies that deliver Today, OFED is distributed as part of major Linux to and benefit from InfiniBand immediate results. Based on IBTA defined QoS OS and virtualization software distributions. transport extensions, QoS features supported with OFED OFED supports all popular CPU platforms. Access Layer 1.3 include granular traffic classification, traffic Enabling a Large Software - Supports the OpenFabrics defined marking, and scalable and easy-to-deploy policing Verbs API at the user and kernel Ecosystem and shaping mechanisms that can be easily levels. User level verbs allow Through its collaboration with the OpenFabrics protocols MPI and other applications implemented through customization of sample to interface to Mellanox InfiniBand QoS configuration files used by the QoS Manager community, support for multiple industry-standard hardware. Kernel levels verbs (part of Open Subnet Manager) interfaces that enable off-the-shelf applications allow protocols like SDP, SRP and to work with InfiniBand, and support for popular IP-over-IB to interface to Mellanox OFED is interoperable across multi- OS and CPU platforms, Mellanox has created InfiniBand hardware vendor solutions a large, interoperable software ecosystem that SCSI Mid Layer As a founding member of OpenFabrics, previously benefits the HPC and EDC markets. - The SCSI Mid Layer interface known as OpenIB, Mellanox is driving interop- enables SCSI based block storage and management applications to HPC Embedded Data Center Storage Management interface with the SRP Initiator HPC Applications Life Sciences Back-End, App/MiddleWare, Front-End Block Cluster component and the Mellanox Applications Storage Management IBADM InfiniBand hardware User MPI *UDAPL Diagnostics OpenSM libsdp uverbs API Sockets Layer umad API FS Kernel TCP UDP ICMP SCSI Kernel IP Mid Layer Bypass RDS SDP IPoIB SRP *iSER verbs API core (verbs, mads, sa, cm, cma, uverbs, umads) HCA Driver (mthca/mlx4) HCA Hardware Markets Linux InfiniBand Fabric Applications OFED Linux Modules * Currently Not Supported in OFED by Mellanox Note: SRP Target is included in the OFED package but is not shown in this diagram Platform Supplier Products and Support RedHat EL4 up4,up5, up6; EL5 and up1, Fedora C6 Novell SLES10, SLES10 SP1 and up1; OpenSuSE 10.3 Intel x86, IA64, EM64T, EM64T x86_64, PCI-X and PCIe platforms AMD AMD64 and AMD64-Ex Opteron, PCI-X and PCIe platforms IBM PPC64 2900 Stender Way, Santa Clara, CA 95054 Tel: 408-970-3400 • Fax: 408-970-3403 www.mellanox.com © Copyright 2008. Mellanox Technologies. All rights reserved. Mellanox is a registered trademark of Mellanox Technologies, and ConnectX, InfiniBlast, InfiniBridge, InfiniHost, InfiniRISC, InfiniScale, and InfiniPCI are trademarks of Mellanox Technologies. 2492PB Rev 1.3.