Implementing an IBM High-Performance Computing Solution on IBM Power System S822LC
Total Page:16
File Type:pdf, Size:1020Kb
Front cover Implementing an IBM High-Performance Computing Solution on IBM Power System S822LC Dino Quintero Luis Carlos Cruz Huertas Tsuyoshi Kamenoue Wainer dos Santos Moschetta Mauricio Faria de Oliveira Georgy E Pavlov Alexander Pozdneev Redbooks International Technical Support Organization Implementing an IBM High-Performance Computing Solution on IBM Power System S822LC July 2016 SG24-8280-00 Note: Before using this information and the product it supports, read the information in “Notices” on page vii. First Edition (July 2016) This edition applies to the following products: Red Hat Enterprise Linux (RHEL) Server 7.2 (little-endian) Linux kernel version 3.10.0-327 Extreme Cluster/Cloud Administration Toolkit (xCAT) 2.11 Compute Unified Device Architecture (CUDA) Toolkit 7.5 (7.5-23) Mellanox OpenFabrics Enterprise Distribution (OFED) for Linux 3.2 (3.2-1.0.1.1) XL C/C++ Compiler for Linux V13.1.2 XL Fortran Compiler for Linux V15.1.2 Advance Toolchain 8.0 (8.0-5) GNU Compiler Collection (GCC) 4.8.5 (RHEL) IBM Parallel Environment Runtime Edition (PE RTE) 2.3 IBM Parallel Environment Developer Edition (PE DE) 2.2 IBM Engineering and Scientific Subroutine Library (ESSL) 5.4 IBM Parallel ESSL (PESSL) 5.2 IBM Spectrum Scale (formerly IBM GPFS) 4.1.1.3 IBM Spectrum LSF (formerly IBM Platform LSF) 9.1.3 OpenPower Abstraction Layer (OPAL) firmware OP810.10 (OP8_v1.7_1.13) NAS Parallel Benchmarks version 3.3.1 © Copyright International Business Machines Corporation 2016. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Notices . vii Trademarks . viii IBM Redbooks promotions . ix Preface . xi Authors. xi Now you can become a published author, too! . xiii Comments welcome. xiii Stay connected to IBM Redbooks . xiii Chapter 1. Introduction to the IBM Power System S822LC for high performance computing workloads. 1 1.1 IBM POWER8 technology. 2 1.2 OpenPOWER . 2 1.3 IBM Power System S822LC . 3 1.3.1 Differences between 8335-GCA and 8335-GTA models . 4 Chapter 2. Reference architecture. 7 2.1 Hardware components of an HPC system . 8 2.1.1 Login nodes . 9 2.1.2 Management nodes . 9 2.1.3 Compute nodes. 9 2.1.4 High performance interconnect. 10 2.1.5 Management, service, and site (public) networks. 10 2.1.6 Parallel file system . 12 2.2 Software components of an HPC system . 13 2.2.1 System software . 13 2.2.2 Application development software . 17 2.2.3 Application software . 20 2.3 HPC system solution. 20 2.3.1 Compute nodes. 20 2.3.2 Management node . 21 2.3.3 Login node . 21 2.3.4 Combining the management and the login node . 21 2.3.5 Parallel file system . 21 2.3.6 High performance interconnect switch . 21 Chapter 3. Hardware components . 23 3.1 IBM Power System S822LC . 24 3.1.1 IBM POWER8 processor . 24 3.1.2 Memory subsystem. 30 3.1.3 Input and output . 32 3.1.4 NVIDIA GPU . 38 3.1.5 BMC . 40 3.2 Mellanox InfiniBand. 40 3.3 IBM System Storage . 41 3.3.1 IBM Storwize family. 41 3.3.2 IBM FlashSystem family . 41 © Copyright IBM Corp. 2016. All rights reserved. iii 3.3.3 IBM XIV Storage System . 41 Chapter 4. Software stack . 43 4.1 System management . 44 4.2 OPAL firmware . 44 4.3 xCAT . 44 4.4 RHEL server . 45 4.5 NVIDIA CUDA Toolkit . 45 4.6 Mellanox OFED for Linux . 46 4.7 IBM XL compilers, GCC, and Advance Toolchain . 46 4.7.1 XL compilers . 46 4.7.2 GCC and Advance Toolchain . 47 4.8 IBM Parallel Environment . 48 4.8.1 IBM PE Runtime Edition . 48 4.8.2 IBM PE Developer Edition . 48 4.9 IBM Engineering and Scientific Subroutine Library and Parallel ESSL. 49 4.10 IBM Spectrum Scale (formerly IBM GPFS). 50 4.11 IBM Spectrum LSF (formerly IBM Platform LSF) . 50 Chapter 5. Software deployment . 53 5.1 Software stack. 54 5.2 System management . 54 5.2.1 Build instructions for IPMItool . 54 5.2.2 Frequently used commands with the IPMItool . 55 5.2.3 Boot order configuration . ..