IBM Platform Computing Solutions Reference Architectures and Best Practices
Total Page:16
File Type:pdf, Size:1020Kb
Front cover IBM Platform Computing Solutions Reference Architectures and Best Practices Helps with the foundation to manage enterprise environments Delivers reference architectures and best practices guides Provides case scenarios Dino Quintero Luis Carlos Cruz Ricardo Machado Picone Dusan Smolej Daniel de Souza Casali Gheorghe Tudor Joanna Wong ibm.com/redbooks International Technical Support Organization IBM Platform Computing Solutions Reference Architectures and Best Practices April 2014 SG24-8169-00 Note: Before using this information and the product it supports, read the information in “Notices” on page v. First Edition (April 2014) This edition applies to RedHat 6.4, IBM Platform Cluster Manager Standard Edition (PCM-SE) 4.1.1, IBM Platform Symphony Advanced Edition 6.1.1, GPFS FPO 3.5.0.13, Hadoop 1.1.1. © Copyright International Business Machines Corporation 2014. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Notices . .v Trademarks . vi Preface . vii Authors. vii Now you can become a published author, too! . ix Comments welcome. ix Stay connected to IBM Redbooks . .x Chapter 1. Introduction. 1 1.1 Why IBM Platform Computing?. 2 1.2 High performance clusters . 2 1.3 IBM Platform HPC implementation scenario. 3 1.4 Big Data implementation on an IBM high performance computing cluster . 3 1.5 IBM Platform Computing solutions and products . 5 Intel . 7 Chapter 2. High performance clusters . 9 2.1 Cluster management. 10 2.1.1 IBM Platform HPC. 12 2.1.2 IBM Platform Cluster Manager Standard Edition . 19 2.1.3 IBM Platform Cluster Manager Advanced Edition. 23 2.2 Workload management . 29 2.2.1 IBM Platform Load Sharing Facility. 29 2.2.2 IBM Platform Symphony . 41 2.3 Reference architectures . 54 2.3.1 IBM Application Ready Solution for Abaqus . 54 2.3.2 IBM Application Ready Solution for Accelrys . 55 2.3.3 IBM Application Ready Solution for ANSYS . 55 2.3.4 IBM Application Ready Solution for CLC bio. 56 2.3.5 IBM Application Ready Solution for Gaussian . 56 2.3.6 IBM Application Ready Solution for InfoSphere BigInsights . 56 2.3.7 IBM Application Ready Solution for mpiBLAST . 57 2.3.8 IBM Application Ready Solution for MSC Software . 57 2.3.9 IBM Application Ready Solution for Schlumberger . 57 2.3.10 IBM Application Ready Solution for Technical Computing . 58 2.3.11 IBM System x and Cluster Solutions configurator. 58 2.4 Workload optimized systems . 59 2.4.1 NeXtScale System . 60 2.4.2 iDataPlex. 61 2.4.3 Intelligent Cluster . 62 2.4.4 Enterprise servers. 63 2.4.5 High volume systems . 65 Chapter 3. IBM Platform High Performance Computing implementation scenario . 67 3.1 Application Ready Solution versus scenario-based implementation. 68 3.2 Scenario-based implementation . 68 3.2.1 Cluster hardware. 70 © Copyright IBM Corp. 2014. All rights reserved. iii 3.2.2 Management nodes . 71 3.2.3 Compute nodes. 71 3.2.4 Networking . 72 3.2.5 Cluster prerequisites . 74 3.2.6 Cluster deployment . 75 3.2.7 Provisioning cluster resources . 80 3.2.8 IBM Platform HPC high availability . 119 3.2.9 Running applications in Platform HPC . 130 Chapter 4. IBM Big Data implementation on an IBM High Performance Computing cluster . 133 4.1 IBM high performance computing for Big Data analytics reference architectures. 134 4.2 High performance low latency Big Data solutions stack by using PCM-SE for a Platform Symphony MapReduce cluster. 140 4.2.1 Installing IBM Platform Cluster Manager Standard Edition. 140 4.2.2 Installing the IBM General Parallel File System . 146 4.2.3 GPFS open source portability layer . 150 4.2.4 Configuring GPFS on the cluster initial nodes . 153 4.2.5 Hadoop installation process for the initial nodes. 160 4.2.6 Installing IBM Platform Symphony . 162 4.2.7 Building an automatic kit template . 172 4.2.8 Adding the kit to an image profile . 180 4.2.9 Testing a Platform Symphony MapReduce job. 180 Related publications . 185 IBM Redbooks . 185 Other publications . 185 Online resources . 186 Help from IBM . 187 iv IBM Platform Computing Solutions Reference Architectures and Best Practices Notices This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not grant you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A. The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM websites are provided for convenience only and do not in any manner serve.