Mellanox Connectx®-6 VPI Adapter Cards User Manual for Dell EMC Poweredge Servers
Total Page:16
File Type:pdf, Size:1020Kb
Mellanox ConnectX®-6 VPI Adapter Cards User Manual for Dell EMC PowerEdge Servers OPNs: Y1T43, 7TKND, 1GK7G, CY7GD Rev 1.1 www.mellanox.com Mellanox Technologies Confidential NOTE: THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT PRODUCT(S) AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES AS-IS WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUSTOMER'S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCT(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY. ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT, INDIRECT, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES OF ANY KIND (INCLUDING, BUT NOT LIMITED TO, PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY FROM THE USE OF THE PRODUCT(S) AND RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Mellanox Technologies 350 Oakmead Parkway Suite 100 Sunnyvale, CA 94085 U.S.A. www.mellanox.com Tel: (408) 970-3400 Fax: (408) 970-3403 © Copyright 2019. Mellanox Technologies Ltd. All Rights Reserved. Mellanox®, Mellanox logo, Mellanox Open Ethernet®, LinkX®, Mellanox Spectrum®, Mellanox Virtual Modular Switch®, MetroDX®, MetroX®, MLNX-OS®, ONE SWITCH. A WORLD OF OPTIONS®, Open Ethernet logo, Spectrum logo, Switch-IB®, SwitchX®, UFM®, and Virtual Protocol Interconnect® are registered trademarks of Mellanox Technologies, Ltd. For the complete and most updated list of Mellanox trademarks, visit http://www.mellanox.com/page/trademarks. All other trademarks are property of their respective owners. Mellanox Technologies Confidential 2 Table of Contents 1 Introduction ................................................................................................................................. 8 1.1 Product Overview ............................................................................................................... 8 1.2 ConnectX-6 SNAPI (Single-slot) Configuration .................................................................. 8 1.2.1 ConnectX-6 Dual-slot Socket Direct Cards 2x PCIe x16 ..................................... 9 1.2.2 ConnectX-6 SNAPI (Single-slot) PCIe 2 x8 in a Row ........................................ 10 1.3 Features and Benefits ...................................................................................................... 11 1.3.1 Operating Systems/Distributions ........................................................................ 12 1.3.2 Connectivity ........................................................................................................ 12 1.3.3 Manageability ..................................................................................................... 12 2 Interfaces ................................................................................................................................... 13 2.1 InfiniBand Interface .......................................................................................................... 13 2.2 Ethernet QSFP56 Interface .............................................................................................. 13 2.3 PCI Express Interface ...................................................................................................... 13 2.4 LED Interface ................................................................................................................... 14 2.5 Heat Sink Interface ........................................................................................................... 14 2.6 SMBus Interface ............................................................................................................... 15 2.7 Voltage Regulators ........................................................................................................... 15 3 Hardware Installation ............................................................................................................... 16 3.1 Safety Warnings ............................................................................................................... 16 3.2 Installation Procedure Overview ...................................................................................... 17 3.3 System Requirements ...................................................................................................... 17 3.3.1 Hardware Requirements .................................................................................... 17 3.3.2 Airflow Requirements ......................................................................................... 18 3.3.3 Software Requirements ...................................................................................... 18 3.4 Safety Precautions ........................................................................................................... 18 3.5 Pre-Installation Checklist .................................................................................................. 18 3.6 Installation Instructions ..................................................................................................... 19 3.6.1 Installation Instructions for R740 Server ............................................................ 20 3.6.2 Installation Instructions for C6400 Server .......................................................... 23 3.6.3 Installation Instructions for C4140 Server .......................................................... 26 3.6.4 Installation Instructions for HDR100 Card .......................................................... 30 3.7 Cables and Modules ......................................................................................................... 31 3.7.1 Cable Installation ................................................................................................ 31 3.8 Identifying the Card in Your System ................................................................................. 31 3.8.1 On Linux ............................................................................................................. 31 3.8.2 On Windows ....................................................................................................... 32 3 Mellanox Technologies Confidential Rev 1.1 4 Driver Installation ..................................................................................................................... 33 4.1 Linux Driver Installation .................................................................................................... 33 4.1.1 Prerequisites ....................................................................................................... 33 4.1.2 Downloading Mellanox OFED ............................................................................ 33 4.1.3 Installing Mellanox OFED ................................................................................... 34 4.1.4 Installing MLNX_OFED Using YUM ................................................................... 38 4.1.5 Installing MLNX_OFED Using apt-get Tool ........................................................ 41 4.1.6 Updating Firmware After Installation .................................................................. 42 4.1.7 UEFI Secure Boot............................................................................................... 43 4.1.8 Performance Tuning ........................................................................................... 45 4.2 Windows Driver Installation .............................................................................................. 46 4.2.1 Software Requirements ...................................................................................... 46 4.2.2 Downloading Mellanox WinOF-2 Driver ............................................................. 46 4.2.3 Installing Mellanox WinOF-2 Driver .................................................................... 47 4.2.4 Uninstalling Mellanox WinOF-2 Driver ............................................................... 52 4.2.5 Extracting Files Without Running Installation ..................................................... 52 4.2.6 Firmware Upgrade .............................................................................................. 54 5 Updating Adapter Firmware .................................................................................................... 55 6 Troubleshooting ....................................................................................................................... 56 6.1 General Troubleshooting .................................................................................................. 56 6.2 Linux Troubleshooting ...................................................................................................... 57 6.3 Windows Troubleshooting ................................................................................................ 58 7 Specifications ..........................................................................................................................