Intel® Omni-Path Fabric Suite Fabric Manager — User Guide
Total Page:16
File Type:pdf, Size:1020Kb
Intel® Omni-Path Fabric Suite Fabric Manager User Guide November 2015 Order No.: H76468-1.0 You may not use or facilitate the use of this document in connection with any infringement or other legal analysis concerning Intel products described herein. You agree to grant Intel a non-exclusive, royalty-free license to any patent claim thereafter drafted which includes subject matter disclosed herein. No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document. All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest Intel product specifications and roadmaps. The products described may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request. Copies of documents which have an order number and are referenced in this document may be obtained by calling 1-800-548-4725 or visit http:// www.intel.com/design/literature.htm. Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at http://www.intel.com/ or from the OEM or retailer. No computer system can be absolutely secure. Intel, the Intel logo, Intel Xeon Phi, and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others. Copyright © 2015, Intel Corporation. All rights reserved. Intel® Omni-Path Fabric Suite Fabric Manager User Guide November 2015 2 Order No.: H76468-1.0 Revision History—Intel® Omni-Path Fabric Revision History Date Revision Description November 2015 1.0 Document has been updated for Revision 1.0. September 2015 0.7 Document has been updated for Revision 0.7. April 2015 0.5 Initial release of document. Intel® Omni-Path Fabric Suite Fabric Manager November 2015 User Guide Order No.: H76468-1.0 3 Intel® Omni-Path Fabric—Contents Contents Revision History..................................................................................................................3 Preface............................................................................................................................. 10 Intended Audience..................................................................................................... 10 Documentation Set.....................................................................................................10 Documentation Conventions........................................................................................ 11 License Agreements....................................................................................................11 Technical Support.......................................................................................................12 1.0 Overview....................................................................................................................13 1.1 Component Overview............................................................................................ 13 1.1.1 Subnet Manager....................................................................................... 13 1.1.2 Subnet Administration............................................................................... 14 1.1.3 Performance Manager Overview..................................................................14 1.1.4 Fabric Executive....................................................................................... 15 1.2 Embedded and Host Solutions................................................................................ 15 1.2.1 Host FM...................................................................................................15 1.2.2 Embedded FM...........................................................................................16 1.2.3 Choosing Between Host and Embedded FM Deployments................................16 1.3 Multiple FMs in a Fabric......................................................................................... 16 1.3.1 FM State – Master and Standby.................................................................. 16 1.3.2 FM Role Arbitration................................................................................... 17 1.3.3 Master FM Failover.................................................................................... 17 1.3.4 Master Preservation – Sticky Failover...........................................................17 1.4 Fabric Manager Configuration Consistency................................................................18 1.4.1 Parameters Excluded from Configuration Consistency Checking....................... 18 1.4.2 Changes Needed to Run ESM/HSM as Redundant Pairs...................................19 1.5 Congestion Control Architecture..............................................................................19 1.6 Multiple Subnet Support in Host FM.........................................................................20 1.7 Intel® Omni-Path Fabric Suite Fabric Manager GUI and Fabric Manager........................20 1.8 FM Logging.......................................................................................................... 21 1.9 SM Algorithms and Features...................................................................................21 1.9.1 Routing Algorithms in the SM..................................................................... 21 1.9.2 LID Mask Control (LMC)............................................................................. 21 1.9.3 Multicast Group Support............................................................................ 22 1.10 FM’s Subnet Manager.......................................................................................... 22 1.11 FM Interoperability.............................................................................................. 22 1.12 Terminology Clarification – “FM” vs. “SM”............................................................... 22 2.0 Advanced FM Capabilities...........................................................................................24 2.1 Fabric Change Detection........................................................................................ 24 2.1.1 Handling Unstable Fabrics.......................................................................... 24 2.1.2 Tolerance of Slow Nodes............................................................................ 25 2.1.3 Multicast Denial of Service......................................................................... 25 2.2 Fabric Unicast Routing........................................................................................... 25 2.2.1 Credit Loops.............................................................................................26 2.2.2 Routing Algorithm..................................................................................... 26 2.2.3 Adaptive Routing...................................................................................... 27 Intel® Omni-Path Fabric Suite Fabric Manager User Guide November 2015 4 Order No.: H76468-1.0 Contents—Intel® Omni-Path Fabric 2.2.4 LMC, Dispersive Routing, and Fabric Resiliency............................................. 28 2.3 Fabric Multicast Routing.........................................................................................30 2.3.1 Handling Fabric Changes............................................................................30 2.3.2 Conserving Multicast LIDs.......................................................................... 31 2.3.3 Precreated Multicast Groups....................................................................... 31 2.3.4 Multicast Spanning Tree Root..................................................................... 31 2.3.5 Multicast Spanning Tree Pruning................................................................. 32 2.4 Packet and Switch Timers...................................................................................... 32 2.4.1 Switch Timers...........................................................................................32 2.4.2 Packet LifeTime........................................................................................ 33 2.5 Preemption.......................................................................................................... 33 2.6 Fabric Sweeping................................................................................................... 34 2.6.1 Optimized Fabric Programming................................................................... 34 2.6.2 Scalable SMA Retries.................................................................................35 2.7 Link Bandwidth Negotiation.................................................................................... 35 2.8 Fabric Performance Management............................................................................ 35 2.8.1 Port Groups..............................................................................................36 2.8.2 PM Sweeps.............................................................................................. 37 2.8.3 PA Images............................................................................................... 38 2.8.4 PA Client Queries...................................................................................... 38 2.8.5 Error Thresholding...................................................................................