Fabric Manager for NVIDIA NVSwitch Systems User Guide / Virtualization / High Availability Modes DU-09883-001_v0.7 | January 2021 Document History DU-09883-001_v0.7 Version Date Authors Description of Change 0.1 Oct 25, 2019 SB Initial Beta Release 0.2 Mar 23, 2020 SB Updated error handling and bare metal mode 0.3 May 11, 2020 YL Updated Shared NVSwitch APIs section with new API information 0.4 July 7, 2020 SB Updated MIG interoperability and high availability details. 0.5 July 17, 2020 SB Updated running as non-root instructions 0.6 Aug 03, 2020 SB Updated installation instructions based on CUDA repo and updated SXid error details 0.7 Jan 26, 2021 GT, CC Updated with vGPU multitenancy virtualization mode Fabric Ma nager fo r NVI DIA NVSwitch Sy stems DU-09883-001_v0.7 | ii Table of Contents Chapter 1. Overview ...................................................................................................... 1 1.1 Introduction .............................................................................................................1 1.2 Terminology ............................................................................................................1 1.3 NVSwitch Core Software Stack .................................................................................2 1.4 What is Fabric Manager?..........................................................................................3 Chapter 2. Getting Started With Fabric Manager ...................................................... 5 2.1 Basic Components...................................................................................................5 2.2 NVSwitch and NVLink Initialization ...........................................................................5 2.3 Supported Platforms................................................................................................6 2.4 Supported Deployment Models.................................................................................6 2.5 Other NVIDIA Software Packages .............................................................................6 2.6 Installation ..............................................................................................................7 2.7 Managing Fabric Manager Service............................................................................7 2.8 Fabric Manager Startup Options ...............................................................................8 2.9 Fabric Manager Service File .....................................................................................9 2.10 Running Fabric Manager as Non-Root ......................................................................9 2.11 Fabric Manager Config Options .............................................................................. 11 Chapter 3. Bare Metal Mode ...................................................................................... 19 3.1 Introduction ........................................................................................................... 19 3.2 Fabric Manager Installation ................................................................................... 19 3.3 Runtime NVSwitch and GPU errors ........................................................................ 19 3.4 Interoperability With MIG........................................................................................ 21 Chapter 4. Virtualization Models ............................................................................... 23 4.1 Introduction ........................................................................................................... 23 4.2 Supported Virtualization Models ............................................................................. 23 Chapter 5. Fabric Manager SDK................................................................................ 25 5.1 Data Structures ..................................................................................................... 25 5.2 Initializing FM API interface.................................................................................... 28 5.3 Shutting Down FM API interface ............................................................................. 28 5.4 Connect to Running FM Instance ............................................................................ 28 5.5 Disconnect from Running FM Instance ................................................................... 29 5.6 Getting Supported Partitions .................................................................................. 29 5.7 Activate a GPU Partition ......................................................................................... 30 5.8 Activate a GPU Partition with VFs ........................................................................... 30 Fabric Ma nager fo r NVI DIA NVSwitch Sy stems DU-09883-001_v0.7 | iii 5.9 Deactivate a GPU Partition ..................................................................................... 31 5.10 Set Activated Partition List after Fabric Manager Restart ........................................ 32 5.11 Get NVLink Failed Devices...................................................................................... 32 5.12 Get Unsupported Partitions .................................................................................... 33 Chapter 6. Full Passthrough Virtualization Model .................................................. 34 6.1 Supported VM Configurations ................................................................................. 36 6.2 Virtual Machines with 16 GPUs ............................................................................... 36 6.3 Virtual Machines with 8 GPUS ................................................................................ 36 6.4 Virtual Machines with 4 GPUS ................................................................................ 36 6.5 Virtual Machines with 2 GPUs................................................................................. 37 6.6 Virtual Machine with 1 GPU .................................................................................... 37 6.7 Other Requirements .............................................................................................. 37 6.8 Hypervisor Sequences............................................................................................ 37 6.9 Monitoring Errors .................................................................................................. 38 6.10 Limitations ............................................................................................................ 38 Chapter 7. Shared NVSwitch Virtualization Model .................................................. 39 7.1 Software Stack ...................................................................................................... 39 7.2 Preparing Service VM............................................................................................. 40 7.3 FM Shared Library and APIs ................................................................................... 41 7.4 Fabric Manager Resiliency ..................................................................................... 44 7.5 Cycle Management ................................................................................................ 45 7.6 Guest VM Life Cycle Management........................................................................... 46 7.7 Error Handling....................................................................................................... 47 7.8 Interoperability With MIG........................................................................................ 48 Chapter 8. vGPU Virtualization Model....................................................................... 49 8.1 Software Stack ...................................................................................................... 49 8.2 Preparing the vGPU Host ....................................................................................... 50 8.3 FM Shared Library and APIs ................................................................................... 51 8.4 Fabric Manager Resiliency ..................................................................................... 51 8.5 vGPU Partitions ..................................................................................................... 51 8.6 Guest VM Life Cycle Management........................................................................... 52 8.7 Error Handling....................................................................................................... 54 8.8 GPU Reset ............................................................................................................. 54 8.9 Interoperability with MIG ........................................................................................ 54 Chapter 9. Supported High Availability Modes......................................................... 56 9.1 Definition of Common Terms.................................................................................. 56 9.2 GPU Access NVLink Failure ................................................................................... 57 9.3 Trunk NVLink Failure ............................................................................................. 58 9.4 NVSwitch Failure ................................................................................................... 60 Fabric Ma nager fo r NVI DIA NVSwitch Sy stems DU-09883-001_v0.7 | iv 9.5 GPU