IBM Spectrum Computing Solutions

Front cover IBM Spectrum Computing Solutions Dino Quintero Daniel de Souza Casali Eduardo Luis Cerdas Moya Federico Fros Maciej Olejniczak Redbooks International Technical Support Organization IBM Spectrum Computing Solutions May 2017 SG24-8373-00 Note: Before using this information and the product it supports, read the information in “Notices” on page vii. First Edition (May 2017) This edition applies to: Red Hat Linux ppc64 Little Endian version 7.2 IBM Spectrum Scale version 4.2.1 IBM Cluster Foundation version v4.2.2 IBM Spectrum Conductor with Spark version 2.2 IBM Spectrum MPI version 10 © Copyright International Business Machines Corporation 2017. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Notices . vii Trademarks . viii Preface . ix Authors. ix Now you can become a published author, too . .x Comments welcome. xi Stay connected to IBM Redbooks . xi Chapter 1. Introduction to IBM Spectrum Computing . 1 1.1 Overview . 2 1.2 Big data and resource management . 2 1.3 The new era for high-performance computing (HPC) . 2 1.4 Hybrid cloud bursting . 3 1.5 The big data challenge . 4 1.5.1 Hadoop . 4 1.5.2 Apache Spark . 5 1.5.3 Hadoop Distributed File System (HDFS) . 5 1.5.4 Multi-tenancy. 5 1.6 IBM Spectrum Cluster Foundation . 6 1.7 IBM Spectrum Computing . 6 1.7.1 IBM Spectrum Conductor with Spark . 7 1.7.2 IBM Spectrum LSF . 7 1.7.3 IBM Spectrum Symphony . 7 Chapter 2. IBM Spectrum Computing family . 9 2.1 IBM Software-Defined Infrastructure. 10 2.2 IBM Spectrum Computing . 11 2.2.1 IBM Spectrum LSF . 13 2.2.2 IBM Spectrum Symphony . 14 2.2.3 IBM Spectrum Conductor . 16 2.2.4 IBM Cluster Foundation . 16 2.3 IBM Spectrum Storage . 17 Chapter 3. IBM Spectrum Computing requirements . 19 3.1 IBM Spectrum LSF system requirements . 20 3.1.1 Operating system support. 20 3.1.2 Hardware requirements for the master host . 22 3.1.3 Server host compatibility. 22 3.2 IBM Spectrum Symphony system requirements . 22 3.2.1 The Minimum hardware requirements for IBM Spectrum Symphony Developer Edition V7.2 . 22 3.2.2 Software requirements . 23 3.3 IBM Spectrum Conductor with Spark requirements . 24 3.3.1 Hardware requirements. 24 3.3.2 Software requirements . 26 3.4 Our lab test environment. 27 Chapter 4. IBM Spectrum LSF . 29 © Copyright IBM Corp. 2017. All rights reserved. iii 4.1 IBM Spectrum LSF family overview . 30 4.1.1 IBM Spectrum LSF family offerings. 31 4.1.2 IBM Spectrum LSF optional add-ons . 33 4.2 IBM Spectrum LSF integration with Docker . 35 4.2.1 IBM Spectrum LSF and Docker integration. 36 4.2.2 IBM Spectrum LSF 10.1 solution Docker job support . 37 4.3 IBM Spectrum LSF Data Manager . 39 4.3.1 Concepts and terminology . 40 4.3.2 How IBM Spectrum LSF Data Manager works . 41 4.3.3 Single cluster implementation . 42 4.3.4 MultiCluster implementation . 43 4.4 IBM Spectrum Symphony MapReduce Accelerator for IBM Spectrum LSF . 45 4.4.1 Configure the Apache Hadoop integration . 46 4.4.2 Run a Hadoop application on IBM Spectrum LSF . 47 4.5 IBM Spectrum LSF MultiCluster capability . 48 4.6 Resource connector for IBM Spectrum LSF . 50 Chapter 5. IBM Spectrum Symphony . 53 5.1 IBM Spectrum Symphony overview . 54 5.1.1 Key highlights of IBM Spectrum Symphony . 54 5.1.2 Scalability . 55 5.1.3 IBM Spectrum Symphony MapReduce, YARN, and Docker integration. 56 5.2 IBM Spectrum Symphony editions . 56 5.3 IBM Spectrum Symphony MultiCluster . 58 5.3.1 Cluster management. 58 5.3.2 MultiCluster master cluster . 59 5.3.3 MultiCluster silo clusters . 61 5.3.4 MultiCluster roles . 61 5.4 Multidimensional schedule . 62 5.5 Multitenancy infractructure . 63 5.5.1 The narrow view of multitenancy. 63 5.5.2 Advantages and challenges . 64 5.5.3 Multitenant designs ..

214

Copy Link

Recommended publications

IBM Platform Symphony V5.2 Enables Scalable, High- Performance Grid Services for a Variety of Parallel Compute and Data Intensive Applications
IBM United States Software Announcement 212-204, dated June 4, 2012 IBM Platform Symphony V5.2 enables scalable, high- performance grid services for a variety of parallel compute and data intensive applications Table of contents 1 Overview 7 Publications 2 Key prerequisites 8 Technical information 2 Planned availability date 13 Ordering information 2 Description 16 Terms and conditions 4 Product positioning 19 Prices 6 Program number 20 Order now At a glance IBM® Platform SymphonyTM V5.2 helps you exceed your performance goals with a fast, efficient grid computing environment. You can realize: • Faster throughput and performance • Higher levels of resource utilization • Reduced infrastructure and management costs • Reduced application development and maintenance costs • The agility to respond instantly to real-time demands For ordering, contact Your IBM representative or an IBM Business Partner. For more information contact the Americas Call Centers at 800-IBM-CALL (426-2255). Reference: YE001 Overview IBM Platform Symphony V5.2 is an enterprise-class grid manager for running distributed application services on a scalable, shared, heterogeneous grid. It accelerates a wide variety of compute and data-intensive applications, quickly computing results while making optimal use of available infrastructure. Platform Symphony's efficient low-latency middleware and scheduling architecture is designed to provide the performance and agility required to predictably meet and exceed throughput goals for the most demanding analytic workloads. Designed
[Show full text]
Worksoft Certify Installation Guide
Installation Guide Worksoft Certify® Worksoft, Inc. · 15851 Dallas Parkway, Suite 855 · Addison, TX 75001 www.worksoft.com · 866-836-1773 Worksoft Certify Installation Guide Version 11 © Copyright 2019 by Worksoft, Inc. All rights reserved. Worksoft is a business name of Worksoft, Inc. Information in this document is subject to change and revision without notice. The software described herein may only be used and copied as outlined in the Software License Agreement. No part of this manual may be reproduced by any means, electronic or mechanical, for any purpose other than the purchaser’s personal use, without prior written permission from Worksoft. Worksoft provides this documentation “as is” without warranty of any kind, either express or implied. Worksoft may revise information in this document without notice and does not represent a commitment on the part of Worksoft, Inc. Worksoft, Inc. may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents except as expressly provided in any written license agreement from Worksoft, Inc. Patent Worksoft Certify® U.S. Patent No. 7,600,220 Trademarks Worksoft Certify® is a registered trademark of Worksoft, Inc. Microsoft Office® is a registered trademark of Microsoft Corporation in the United States and/or other countries. Microsoft® is a registered trademark of Microsoft Corporation in the United States and/or other countries. All other trademarks and trade names mentioned in this guide are the property of their respective owners. Third-Party Copyrights This product includes software developed and copyrighted by the following persons or companies: Reprise License Manager™ Data Dynamics, Ltd., ActiveReports Microsoft® Enterprise Library Infragistics® NetAdvantage® Apache Logging Services log4net Antlr ANTLR The above copyright holders disclaim all responsibility or liability with respect to its usage or its effect upon hardware or computer systems.
[Show full text]
Installation Guide Release 1.2.5 Contents
Installation Guide Release 1.2.5 Contents 1 Requirements 1 1.1 Software dependencies........................................1 2 Installation 2 2.1 Obtain the Vortex Platform installation ﬁle.............................2 2.2 Run the Vortex Platform Setup Wizard...............................2 2.3 Vortex Web directories organization.................................9 2.4 Complete the installation for an Apache Maven build system.................... 10 2.5 Uninstalling Vortex Platform..................................... 10 3 Licensing Vortex Web 14 3.1 General................................................ 14 3.2 Development and Deployment Licenses............................... 14 3.3 Installing the License File...................................... 14 3.4 Running the License Manager Daemon............................... 15 3.5 Utilities................................................ 16 4 Compiling the examples 17 5 Troubleshooting 18 6 Contacts & Notices 19 6.1 Contacts............................................... 19 6.2 Notices................................................ 19 i CHAPTER 1 Requirements 1.1 Software dependencies 1.1.1 Required before installation The following software must be installed on your host system before installing Vortex Web: • An Oracle Java Development Kit (JDK) v7 minimum (ensure that JAVA_HOME is deﬁned and that the java executable is in your PATH). 1.1.2 Other noteworthy dependencies Vortex Web uses the following open source libraries, which are automatically installed by the Vortex Setup Wizard if they
[Show full text]
Setting up Site Licenses
XC LICENSE SERVER Setting Up the Site Licenses INTRODUCTION To complete the installation of an XC Site License, create an options file that includes the Host Name (computer’s name) of each client machine. Save the options file as microchip.opt in the same directory as the license file. For example, if you have the Windows® version of the Network License Server installed, the path to the options file would be: C:\ProgramData\Microchip\xclm\license\microchip.opt In the options file, create a Host Name Group that lists all of the Host Names for the clients that will have access to the license. For example: HOST_GROUP xc8_users <hostname> <hostname2> ... The xc8_users is an arbitrary name. You can have multiple HOST_GROUP lines with the same name in the options file, and the server will concatenate the lists. To use the group, add: INCLUDE swxc8n-pro host_group xc8_user Where the swxc8n-pro is the product name in the license file, host_group specifies that a host group will be used, and xc8_user is the HOST_GROUP name. Possible product names are given in the table below. Product Name swxc8n-pro swxc16n-pro swxc32n-pro swxc32n-cpp For additional information see: • Starting the License Server • Running the License Server as a Service 2015 Microchip Technology Inc. DS50002337A-page 1 XC License Server STARTING THE LICENSE SERVER The installer does NOT start the server for you. Therefore, you MUST follow these instructions to start a network license server. To start the network license server, you need to have a Command or Terminal window open to the bin directory in the path that you selected to install the license server.
[Show full text]
EGO Reference
EGO Reference Platform EGO Version 1.2.3 January 2008 Copyright © 1994-2007 Platform Computing Corporation All rights reserved. Although the information in this document has been carefully reviewed, Platform Computing Corporation (“Platform”) does not warrant it to be free of errors or omissions. Platform reserves the right to make corrections, updates, revisions or changes to the information in this document. UNLESS OTHERWISE EXPRESSLY STATED BY PLATFORM, THE PROGRAM DESCRIBED IN THIS DOCUMENT IS PROVIDED “AS IS” AND WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. IN NO EVENT WILL PLATFORM COMPUTING BE LIABLE TO ANYONE FOR SPECIAL, COLLATERAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING WITHOUT LIMITATION ANY LOST PROFITS, DATA, OR SAVINGS, ARISING OUT OF THE USE OF OR INABILITY TO USE THIS PROGRAM. We'd like to hear You can help us make this document better by telling us what you think of the content, organization, and usefulness of the information. from you If you find an error, or just want to make a suggestion for improving this document, please address your comments to [email protected]. Your comments should pertain only to Platform documentation. For product support, contact [email protected]. Document This document is protected by copyright and you may not redistribute or translate it into another language, in part or in whole. redistribution and translation Internal You may only redistribute this document internally within your organization (for example, on an intranet) provided that you continue redistribution to check the Platform Web site for updates and update your version of the documentation.
[Show full text]
RLM License Administration Manual Page 2 of 136 Revision History
RLM License Administration RLM v14.2 June, 2021 Table of Contents Section 1 – License Management Introduction Introduction ......................................................................................... 5 What's New in RLM v14.2 ................................................................. 7 Section 2 – License Administration Basics Installing an RLM-licensed Product …............................................. 9 The License Server ............................................................................. 11 The License File ................................................................................... 20 HOST line...................................................................................... 21 ISV line.......................................................................................... 21 LICENSE line............................................................................... 23 UPGRADE line............................................................................. 34 CUSTOMER line.......................................................................... 37 The License Environment............................................................ 38 License Administration Tools ............................................................ 40 Client Authentication.......................................................................... 48 The RLM Web Server......................................................................... 49 The RLM Options File ......................................................................
[Show full text]
SYMPHONY 5.3.4 User's Manual 1
SYMPHONY 5.3.4 User’s Manual 1 T.K. Ralphs2 M. Güzelsoy3 A. Mahajan4 May 30, 2011 1This research was partially supported by NSF Grants DMS-9527124, DMI-0534862, and DMI-0522796, as well as Texas ATP Grant 97-3604-010. A revised version of Chapters 4 of this manual now appears in the Springer-Verlag book Computational Combinatorial Optimization edited by M. Jüngerand D. Naddef, see http://link.springer.de/link/service/series/0558/tocs/t2241.htm 2Department of Industrial and Systems Engineering, Lehigh University, Bethlehem, PA 18017, [email protected], http://www.lehigh.edu/~tkr2 3Department of Industrial and Systems Engineering, Lehigh University, Bethlehem, PA 18017, [email protected], http://coral.ie.lehigh.edu/~menal 4Mathematics and Computer Science Division, Argonne National Lab, Argonne, IL 60439 [email protected], http://www.mcs.anl.gov/~mahajan/ °c 2000-2010 Ted Ralphs Acknowledgments First and foremost, many thanks are due to Laci Ladányi who worked with me on the development of a very early precursor of SYMPHONY called COMPSys many years ago now and who taught me much of what I then knew about programming. Thanks are due also to Marta Esö,who wrote an early draft of this manual for what was then COMPSys. This release would not have been possible without the help of both Menal Güzelsoy, who has been instrumental in the development of SYMPHONY since version 4.0, and Ashutosh Mahajan, who has worked on SYMPHONY since version 5.0. In particular, Ashutosh and Menal did all of the work that went into improving SYMPHONY for release 5.2.
[Show full text]
GASP Installation
GASP Installation AeroSoft, Inc. 2000 Kraft Drive, Suite 1400 Blacksburg, VA 24060-6363 (540) 557-1900 (540) 557-1919 (FAX) http://www.aerosoftinc.com Chapter 1 Getting Started 1.1 Overview of Steps The following list provides a general overview of the steps needed to install GASP. 1. Downloading and installing the software 2. Setting any necessary environment variables 3. Generating your machine's hostId 4. Requesting a license key 5. Installing the license 6. Running the license manager 7. Documentation and tutorial information The GASP software package can be installed on either linux or windows based platforms. Depending on the platform, the user will download either specific linux files or windows files. The specifics of each are explained in the following sections. Note: If a GASP license has already been installed on the local net- work, the user can skip steps 3 to 6. For installations where the license server is run on a different machine, the aerosoft/etc/keys/gasp.lic file should be modified to reflect the correct license server host. This is done using a text editor and changing \localhost" to the license server host name. 1 2 GASP Version 5, AeroSoft, Inc. 1.2 Installing GASP on Linux Platforms For linux installation, the user will need to install a common (platform independent) file archive, and a binary (platform specific) file archive. Both the common file archive and platform file archive help create the aerosoft directory (for more information about the directory structure, see the GASP users manual. The common file archive contains files that are independent of platform (i.e., operating system and hardware).
[Show full text]
SDL XPP Installation and Upgrade Guide for Windows
SDL XPP Installation and Upgrade Guide for Windows for use with SDL XPP 8.4 updated February 2012 Notice © SDL plc 2010, 2011, 2012. All rights reserved. Printed in U.S.A. SDL plc has prepared this document for use by its personnel, licensees, and customers. The information contained herein is the property of SDL and shall not, in whole or in part, be reproduced, translated, or converted to any electronic or machine-readable form without prior written approval from SDL. Printed copies are also covered by this notice and subject to any applicable confidentiality agreements. The information contained in this document does not constitute a warranty of performance. Further, SDL reserves the right to revise this document and to make changes from time to time in the content thereof. SDL assumes no liability for losses incurred as a result of out-of-date or incorrect information contained in this document. Trademark Notice See the Trademark Notice PDF file on your SDL product documentation CD-ROM for trademark information. U.S. Government Restricted Rights Legend Use, duplication or disclosure by the government is subject to restrictions as set forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and Computer Software clause at DFARS 252.227-7013 or other similar regulations of other governmental agencies, which designate software and documentation as proprietary. Contractor or manufacturer is SDL plc, 101 Edgewater Drive, Wakefield, MA 01880-1296. ii SDL XPP Installation and Upgrade Guide for Windows Contents Chapter 1 Licensing About Licensing . 1-1 About Failover Servers . 1-2 Multiprocessor Reporting Tool .
[Show full text]
CTC BIM and HIVE Suites Installation and Configuration Guide
CTC BIM and HIVE Suites Installation and Configuration Guide Contents CTC Express Tools Overview ................................................................................................................................................ 5 General Security Requirements Summary........................................................................................................................... 5 Revit Workstations .............................................................................................................................................................. 5 Network Floating License Servers ....................................................................................................................................... 6 Upgrading a CTC Suite ......................................................................................................................................................... 6 Upgrading the Same Year Suite .......................................................................................................................................... 6 Upgrading to a New Year Suite ........................................................................................................................................... 6 General Licensing Information (Excluding HIVE) ................................................................................................................. 7 Timed Trial Licensing ..........................................................................................................................................................
[Show full text]
Oracle® Java Micro Edition Software Development Kit Developer's Guide Release 3.2 for Windows E24265-04
Oracle® Java Micro Edition Software Development Kit Developer's Guide Release 3.2 for Windows E24265-04 September 2012 This document describes how to use the Java ME SDK plugin for NetBeans. Oracle Java Micro Edition Software Development Kit, Release 3.2 for Windows E24265-04 Copyright © 2009, 2012, Oracle and/or its affiliates. All rights reserved. This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following notice is applicable: U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs.
[Show full text]
An Introduction to Security in a CSM 1.3 for AIX 5L Environment
Front cover An Introduction to Security in a CSM 1.3 for AIX 5L Environment Peek at the latest security mechanisms for pSeries clusters Practical security considerations included Security concepts and components explained Octavian Lascu Rashid Sayed Stuart Carroll Teresa Coleman Maik Haehnel Petr Klabenes Dino Quintero Rogelio Reyes, Jr. Mizuho Tanaka David Duy Truong ibm.com/redbooks International Technical Support Organization An Introduction to Security in a CSM 1.3 for AIX 5L Environment December 2002 SG24-6873-00 Note: Before using this information and the product it supports, read the information in “Notices” on page ix. First Edition (December 2002) This edition applies to Version 1, Release 3, of IBM Cluster Systems Management for use with the AIX operating system Version 5, Release 2. © Copyright International Business Machines Corporation 2002. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Figures . vii Notices . .ix Trademarks . x Preface . .xi The team that wrote this redbook. .xi Become a published author . xiii Comments welcome. xiii Chapter 1. Introduction . 1 1.1 Security overview . 2 1.1.1 System security. 2 1.1.2 Network security basics . 3 1.1.3 Data transmission security . 4 1.2 Cluster Systems Management security basics . 5 1.2.1 Reliable Scalable Cluster Technology (RSCT) . 6 1.2.2 Resource Monitoring and Control (RMC) . 6 1.2.3 Resource managers (RM). 7 1.2.4 Cluster Security Services (CtSec). 7 1.2.5 Group Services and Topology Services . 8 Chapter 2.
[Show full text]