DOCSLIB.ORG

Mellanox OFED Linux User's Manual

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Mellanox OFED Linux User's Manual Mellanox OFED for Linux User Manual Rev 1.5.3-3.0.0 Last Modified: February, 2012 www.mellanox.com NOTE: THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT (“PRODUCT(S)”) AND ITS RELATED DOCUMENTA- TION ARE PROVIDED BY MELLANOX TECHNOLOGIES “AS-IS” WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PROD- UCTS IN DESIGNATED SOLUTIONS. THE CUSTOMER'S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCTO(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY. ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANT- ABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE DISCLAIMED. IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT, INDIRECT, SPE- CIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES OF ANY KIND (INCLUDING, BUT NOT LIMITED TO, PAY- MENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CON- TRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY FROM THE USE OF THE PRODUCT(S) AND RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBIL- ITY OF SUCH DAMAGE. Mellanox Technologies Mellanox Technologies, Ltd. 350 Oakmead Parkway PO Box 586 Hermon Building Sunnyvale, CA 94085 Yokneam 20692 U.S.A. Israel www.mellanox.com Tel: +972-4-909-7200 Tel: (408) 970-3400 Fax: +972-4-959-3245 Fax: (408) 970-3403 © Copyright 2011. Mellanox Technologies. All rights reserved. Mellanox®, BridgeX®, ConnectX®, SwitchX®, CORE-Direct®, InfiniBridge®, InfiniHost®, InfiniScale®, PhyX®, Virtual Protocol Interconnect® and Voltaire® are registered trademarks of Mellanox Technologies, Ltd. FabricIT™, MLNX-OS™ and Unbreakable-Link™ are trademarks of Mellanox Technologies, Ltd. All other trademarks are property of their respective owners.. 2 Mellanox Technologies Document Number: 2877 Table of Contents Chapter 1 Mellanox OFED Overview . 13 1.1 Introduction to Mellanox OFED - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 13 1.2 Introduction to Mellanox VPI Adapters - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 13 1.3 Mellanox OFED Package - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 14 1.3.1 ISO Image. 14 1.3.2 Software Components. 14 1.3.3 Firmware. 15 1.3.4 Directory Structure . 15 1.4 Architecture - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 15 1.4.1 mthca HCA (IB) Driver . 16 1.4.2 mlx4 VPI Driver . 16 1.4.3 Mid-layer Core . 17 1.4.4 ULPs . 17 1.4.5 MPI . 18 1.4.6 InfiniBand Subnet Manager . 18 1.4.7 Diagnostic Utilities . 18 1.4.8 Mellanox Firmware Tools . 19 1.5 Quality of Service - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 19 Chapter 2 Installation . 20 2.1 Hardware and Software Requirements - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 20 2.1.1 Hardware Requirements . 20 2.1.2 Software Requirements. 21 2.2 Downloading Mellanox OFED - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 21 2.3 Installing Mellanox OFED- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 21 2.3.1 Pre-installation Notes . 22 2.3.2 Installation Script . 22 2.3.3 Installation Procedure . 25 2.3.4 Installation Results . 29 2.3.5 Post-installation Notes . 31 2.4 Updating Firmware After Installation - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 31 2.5 Uninstalling Mellanox OFED- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 32 Chapter 3 Configuration Files . 33 Chapter 4 Driver Features . 34 4.1 RDMA over Converged Ethernet - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 34 4.1.1 RoCE Overview . 34 4.1.2 Software Dependencies. 34 4.1.3 Firmware Dependencies . 34 4.1.4 General Guidelines . 34 4.1.5 Ported Applications. 34 4.1.6 GID Tables . 35 4.1.7 Using VLANs . 35 4.1.8 Reading Port Counters Statistics. 36 4.1.9 A Detailed Example . 37 4.1.10 Configuring DAPL over RoCE. 42 4.2 Reliable Datagram Sockets - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 43 4.2.1 Overview. 43 4.2.2 RDS Configuration . 43 4.3 Sockets Direct Protocol - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 43 4.3.1 Overview. 43 4.3.2 libsdp.so Library . 44 4.3.3 Configuring SDP. 44 4.3.4 Environment Variables . 47 4.3.5 Converting Socket-based Applications. 47 Mellanox Techologies 1 4.3.6 BZCopy – Zero Copy Send . 54 4.3.7 Using RDMA for Small Buffers. 55 4.4 SCSI RDMA Protocol- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 55 4.4.1 Overview. ..
Load more

Recommended publications
  • Getting to Grips with Unix and the Linux Family
    Getting to grips with Unix and the Linux family David Chiappini, Giulio Pasqualetti, Tommaso Redaelli Torino, International Conference of Physics Students August 10, 2017 According to the booklet At this end of this session, you can expect: • To have an overview of the history of computer science • To understand the general functioning and similarities of Unix-like systems • To be able to distinguish the features of different Linux distributions • To be able to use basic Linux commands • To know how to build your own operating system • To hack the NSA • To produce the worst software bug EVER According to the booklet update At this end of this session, you can expect: • To have an overview of the history of computer science • To understand the general functioning and similarities of Unix-like systems • To be able to distinguish the features of different Linux distributions • To be able to use basic Linux commands • To know how to build your own operating system • To hack the NSA • To produce the worst software bug EVER A first data analysis with the shell, sed & awk an interactive workshop 1 at the beginning, there was UNIX... 2 ...then there was GNU 3 getting hands dirty common commands wait till you see piping 4 regular expressions 5 sed 6 awk 7 challenge time What's UNIX • Bell Labs was a really cool place to be in the 60s-70s • UNIX was a OS developed by Bell labs • they used C, which was also developed there • UNIX became the de facto standard on how to make an OS UNIX Philosophy • Write programs that do one thing and do it well.
    [Show full text]
  • Linking + Libraries
    LinkingLinking ● Last stage in building a program PRE- COMPILATION ASSEMBLY LINKING PROCESSING ● Combining separate code into one executable ● Linking done by the Linker ● ld in Unix ● a.k.a. “link-editor” or “loader” ● Often transparent (gcc can do it all for you) 1 LinkingLinking involves...involves... ● Combining several object modules (the .o files corresponding to .c files) into one file ● Resolving external references to variables and functions ● Producing an executable file (if no errors) file1.c file1.o file2.c gcc file2.o Linker Executable fileN.c fileN.o Header files External references 2 LinkingLinking withwith ExternalExternal ReferencesReferences file1.c file2.c int count; #include <stdio.h> void display(void); Compiler extern int count; int main(void) void display(void) { file1.o file2.o { count = 10; with placeholders printf(“%d”,count); display(); } return 0; Linker } ● file1.o has placeholder for display() ● file2.o has placeholder for count ● object modules are relocatable ● addresses are relative offsets from top of file 3 LibrariesLibraries ● Definition: ● a file containing functions that can be referenced externally by a C program ● Purpose: ● easy access to functions used repeatedly ● promote code modularity and re-use ● reduce source and executable file size 4 LibrariesLibraries ● Static (Archive) ● libname.a on Unix; name.lib on DOS/Windows ● Only modules with referenced code linked when compiling ● unlike .o files ● Linker copies function from library into executable file ● Update to library requires recompiling program 5 LibrariesLibraries ● Dynamic (Shared Object or Dynamic Link Library) ● libname.so on Unix; name.dll on DOS/Windows ● Referenced code not copied into executable ● Loaded in memory at run time ● Smaller executable size ● Can update library without recompiling program ● Drawback: slightly slower program startup 6 LibrariesLibraries ● Linking a static library libpepsi.a /* crave source file */ … gcc ..
    [Show full text]
  • AR400 User Guide 2.7.1
    AR400 SERIES User Guide Software Release 2.7.1 AR410 AR440S AR441S AR450S AR400 Series Router User Guide for Software Release 2.7.1 Document Number C613-02021-00 REV F. Copyright © 2004 Allied Telesyn International Corp. 19800 North Creek Parkway, Suite 200, Bothell, WA 98011, USA. All rights reserved. No part of this publication may be reproduced without prior written permission from Allied Telesyn. Allied Telesyn International Corp. reserves the right to make changes in specifications and other information contained in this document without prior written notice. The information provided herein is subject to change without notice. In no event shall Allied Telesyn be liable for any incidental, special, indirect, or consequential damages whatsoever, including but not limited to lost profits, arising out of or related to this manual or the information contained herein, even if Allied Telesyn has been advised of, known, or should have known, the possibility of such damages. All trademarks are the property of their respective owner. Contents CHAPTER 1 Introduction Why Read this User Guide? ............................................................................... 7 Where To Find More Information ...................................................................... 8 The Documentation Set .............................................................................. 8 Technical support .............................................................................................. 9 Features of the Router .....................................................................................
    [Show full text]
  • The Linux Command Line
    The Linux Command Line Fifth Internet Edition William Shotts A LinuxCommand.org Book Copyright ©2008-2019, William E. Shotts, Jr. This work is licensed under the Creative Commons Attribution-Noncommercial-No De- rivative Works 3.0 United States License. To view a copy of this license, visit the link above or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042. A version of this book is also available in printed form, published by No Starch Press. Copies may be purchased wherever fine books are sold. No Starch Press also offers elec- tronic formats for popular e-readers. They can be reached at: https://www.nostarch.com. Linux® is the registered trademark of Linus Torvalds. All other trademarks belong to their respective owners. This book is part of the LinuxCommand.org project, a site for Linux education and advo- cacy devoted to helping users of legacy operating systems migrate into the future. You may contact the LinuxCommand.org project at http://linuxcommand.org. Release History Version Date Description 19.01A January 28, 2019 Fifth Internet Edition (Corrected TOC) 19.01 January 17, 2019 Fifth Internet Edition. 17.10 October 19, 2017 Fourth Internet Edition. 16.07 July 28, 2016 Third Internet Edition. 13.07 July 6, 2013 Second Internet Edition. 09.12 December 14, 2009 First Internet Edition. Table of Contents Introduction....................................................................................................xvi Why Use the Command Line?......................................................................................xvi
    [Show full text]
  • Powerview Command Reference
    PowerView Command Reference TRACE32 Online Help TRACE32 Directory TRACE32 Index TRACE32 Documents ...................................................................................................................... PowerView User Interface ............................................................................................................ PowerView Command Reference .............................................................................................1 History ...................................................................................................................................... 12 ABORT ...................................................................................................................................... 13 ABORT Abort driver program 13 AREA ........................................................................................................................................ 14 AREA Message windows 14 AREA.CLEAR Clear area 15 AREA.CLOSE Close output file 15 AREA.Create Create or modify message area 16 AREA.Delete Delete message area 17 AREA.List Display a detailed list off all message areas 18 AREA.OPEN Open output file 20 AREA.PIPE Redirect area to stdout 21 AREA.RESet Reset areas 21 AREA.SAVE Save AREA window contents to file 21 AREA.Select Select area 22 AREA.STDERR Redirect area to stderr 23 AREA.STDOUT Redirect area to stdout 23 AREA.view Display message area in AREA window 24 AutoSTOre ..............................................................................................................................
    [Show full text]
  • BASIC UNIX COMMANDS FILES and DIRECTORIES All Information Is Stored in files
    BASIC UNIX COMMANDS FILES AND DIRECTORIES All information is stored in files. File names and COMMANDS commands are case sensitive. Case matters. Files Commands are what you type at the prompt. Com- are contained in directories. You start out in your mands have arguments on which they operate. For own home directory, and your prompt usually tells example, in rm temp, the command is rm and the ar- its name. At any given time, one of these directories gument is temp; this command removes the file called is your working directory, the one you are in. temp. Here I put arguments in UPPER CASE. Thus, You can refer to files in your working directory by words such as FILE are taken to stand for some other just their names. You can refer to a file that is in a word, such as temp. In the following list, I use [ ] for subdirectory by giving a subdirectory name, a slash, optional arguments that are not typicaly used. and the file name, e.g., Mail/baron. You can refer to Commands have options that are controlled with any file on the computer by giving its full name, start- switches, which are usually letters following a single ing with a slash, such as /home7/b/baron/mbox. dash. Usually you can write several letters after one dash. For example ls -l lists files in a long format, If the file is a program, typing its name will run it. with more information. ls -a lists all the files, in- (That is what commands do.) If the program is some- cluding those that begin with ., which are usually thing you have just written and is in the director you files used by various programs.
    [Show full text]
  • Zypper Cheat Sheet Or Type M an Zypper on a Terminal
    More Information: Page 1 Zypper Cheat Sheet https://en.opensuse.org/SDB:Zypper_usage or type m an zypper on a terminal For Zypper version 1.0.9 Package Management Source Packages and Build Dependencies Basic Help Selecting Packages zypper source-install or zypper si Examples: zypper #list the available global options and commands By capability name: zypper si zypper zypper help [command] #Print help for a specific command zypper in 'perl(Log::Log4perl)' Install only the source package zypper shell or zypper sh #Open a zypper shell session zypper in qt zypper in -D zypper By capability name and/or architecture and/or version Install only the build dependencies zypper in 'zypper<0.12.10' Repository Management zypper in -d zypper zypper in zypper.i586=0.12.11 Listing Defined Repositories By exact package name (--name) Updating Packages zypper in -n ftp zypper repos or zypper lr By exact package name and repository (implies --name) zypper update or zypper up Examples: zypper in factory:zypper Examples: zypper lr -u #include repo URI on the table By package name using wildcards zypper up #update all installed packages zypper lr -P #include repo priority and sort by it zypper in yast*ftp* with newer version as far as possible By specifying a .rpm file to install zypper up libzypp zypper #update libzypp Refreshing Repositories zypper in skype-2.0.0.72-suse.i586.rpm and zypper zypper refresh or zypper ref zypper in sqlite3 #update sqlite3 or install Installing Packages Examples: if not yet installed zypper ref packman main #specify repos to be
    [Show full text]
  • APPENDIX a Aegis and Unix Commands
    APPENDIX A Aegis and Unix Commands FUNCTION AEGIS BSD4.2 SYSS ACCESS CONTROL AND SECURITY change file protection modes edacl chmod chmod change group edacl chgrp chgrp change owner edacl chown chown change password chpass passwd passwd print user + group ids pst, lusr groups id +names set file-creation mode mask edacl, umask umask umask show current permissions acl -all Is -I Is -I DIRECTORY CONTROL create a directory crd mkdir mkdir compare two directories cmt diff dircmp delete a directory (empty) dlt rmdir rmdir delete a directory (not empty) dlt rm -r rm -r list contents of a directory ld Is -I Is -I move up one directory wd \ cd .. cd .. or wd .. move up two directories wd \\ cd . ./ .. cd . ./ .. print working directory wd pwd pwd set to network root wd II cd II cd II set working directory wd cd cd set working directory home wd- cd cd show naming directory nd printenv echo $HOME $HOME FILE CONTROL change format of text file chpat newform compare two files emf cmp cmp concatenate a file catf cat cat copy a file cpf cp cp Using and Administering an Apollo Network 265 copy std input to std output tee tee tee + files create a (symbolic) link crl In -s In -s delete a file dlf rm rm maintain an archive a ref ar ar move a file mvf mv mv dump a file dmpf od od print checksum and block- salvol -a sum sum -count of file rename a file chn mv mv search a file for a pattern fpat grep grep search or reject lines cmsrf comm comm common to 2 sorted files translate characters tic tr tr SHELL SCRIPT TOOLS condition evaluation tools existf test test
    [Show full text]
  • Lab Intro to Console Commands
    New Lab Intro to KDE Terminal Konsole After completing this lab activity the student will be able to; Access the KDE Terminal Konsole and enter basic commands. Enter commands using a typical command line interface (CLI). Explain the use of the following commands, ls, ls –al, dir, mkdir, whoami, Explain the directory structure of a typical user. This lab activity will introduce you to one of the many command line interfaces available in Linux/UNIX operating systems and a few of the most basic commands. The command line interface you will be using for this lab activity is the console called the Konsole and is also referred to as Terminal. Note: As you notice, in the KDE system many features are written with the capital letter “K” in place of the first letter or the utility to reflect the fact it was modified for the KDE system. The original UNIX system did not use a graphical user interface GUI but rather was a command line interface (CLI) similar to the command prompt in Windows operating systems. The command line interface is referred to as a shell. Even today the command line interface (the shell) is used to issue commands on a Linux server to minimize system resources. For example, there is no need to start the GUI on the server to add a new user to an existing system. Starting the GUI will reduce the system performance because it requires RAM to run the GUI. A GUI will affect the overall performance of the server when it is supporting many users (clients).
    [Show full text]
  • UEFI Shell Specification
    UEFI Shell Specification January 26, 2016 Revision 2.2 The material contained herein is not a license, either expressly or impliedly, to any intellectual property owned or controlled by any of the authors or developers of this material or to any contribution thereto. The material contained herein is provided on an "AS IS" basis and, to the maximum extent permitted by applicable law, this information is provided AS IS AND WITH ALL FAULTS, and the authors and developers of this material hereby disclaim all other warranties and conditions, either express, implied or statutory, including, but not limited to, any (if any) implied warranties, duties or conditions of merchantability, of fitness for a particular purpose, of accuracy or completeness of responses, of results, of workmanlike effort, of lack of viruses and of lack of negligence, all with regard to this material and any contribution thereto. Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined." The Unified EFI Forum, Inc. reserves any features or instructions so marked for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. ALSO, THERE IS NO WARRANTY OR CONDITION OF TITLE, QUIET ENJOYMENT, QUIET POSSESSION, CORRESPONDENCE TO DESCRIPTION OR NON-INFRINGEMENT WITH REGARD TO THE SPECIFICATION AND ANY CONTRIBUTION THERETO. IN NO EVENT WILL ANY AUTHOR OR DEVELOPER OF THIS MATERIAL OR ANY CONTRIBUTION THERETO BE LIABLE TO ANY OTHER PARTY FOR THE COST OF PROCURING SUBSTITUTE GOODS OR SERVICES, LOST PROFITS, LOSS OF USE, LOSS OF DATA, OR ANY INCIDENTAL, CONSEQUENTIAL, DIRECT, INDIRECT, OR SPECIAL DAMAGES WHETHER UNDER CONTRACT, TORT, WARRANTY, OR OTHERWISE, ARISING IN ANY WAY OUT OF THIS OR ANY OTHER AGREEMENT RELATING TO THIS DOCUMENT, WHETHER OR NOT SUCH PARTY HAD ADVANCE NOTICE OF THE POSSIBILITY OF SUCH DAMAGES.
    [Show full text]
  • Linux Networking 101
    The Gorilla ® Guide to… Linux Networking 101 Inside this Guide: • Discover how Linux continues its march toward world domination • Learn basic Linux administration tips • See how easy it can be to build your entire network on a Linux foundation • Find out how Cumulus Linux is your ticket to networking freedom David M. Davis ActualTech Media Helping You Navigate The Technology Jungle! In Partnership With www.actualtechmedia.com The Gorilla Guide To… Linux Networking 101 Author David M. Davis, ActualTech Media Editors Hilary Kirchner, Dream Write Creative, LLC Christina Guthrie, Guthrie Writing & Editorial, LLC Madison Emery, Cumulus Networks Layout and Design Scott D. Lowe, ActualTech Media Copyright © 2017 by ActualTech Media. All rights reserved. No portion of this book may be reproduced or used in any manner without the express written permission of the publisher except for the use of brief quotations. The information provided within this eBook is for general informational purposes only. While we try to keep the information up- to-date and correct, there are no representations or warranties, express or implied, about the completeness, accuracy, reliability, suitability or availability with respect to the information, products, services, or related graphics contained in this book for any purpose. Any use of this information is at your own risk. ActualTech Media Okatie Village Ste 103-157 Bluffton, SC 29909 www.actualtechmedia.com Entering the Jungle Introduction: Six Reasons You Need to Learn Linux ....................................................... 7 1. Linux is the future ........................................................................ 9 2. Linux is on everything .................................................................. 9 3. Linux is adaptable ....................................................................... 10 4. Linux has a strong community and ecosystem ........................... 10 5.
    [Show full text]
  • Differential Response: a Primer for Child Welfare Professionals
    FACTSHEETS | OCTOBER 2020 Differential Response: A Primer for Child Welfare Professionals Recognizing that a one-size-fits-all approach WHAT'S INSIDE does not serve children and families well, many agencies have implemented differential response Overview (DR), a system reform that establishes multiple pathways to respond to child maltreatment reports. DR routes families with a screened- Approaches to implementing differential response in report of child maltreatment through a traditional investigation pathway or an alternative assessment response pathway, depending on Differential response in the field other State policies and program requirements. Rather than initiating an investigation every time a family has a screened-in report, DR practice Conclusion seeks to assess a family's needs and connect them with services that will help them keep their References children safe. By linking families with services that will strengthen their ability to safely care for their children, DR can reduce the number of children entering foster care and decrease future involvement with the child welfare system. This factsheet provides child welfare professionals with an overview of DR and considerations for practice. Children's Bureau/ACYF/ACF/HHS | 800.394.3366 | Email: [email protected] | https://www.childwelfare.gov 1 OVERVIEW Both pathways share underlying principles and goals, including a focus on child safety, DR—also called alternative response, family permanency, and well-being, and include assessment response, multiple response, child safety and/or risk assessments. The or dual track—is a way of structuring child pathway assignment for a family could change protective services (CPS) to allow for more based on new information, such as potential flexibility in how it responds to low- and criminal behavior or imminent danger of moderate-risk cases and better meet the maltreatment.
    [Show full text]