Intel Cluster Ready and Platform Open Cluster Stack: Clusters Made Simple

Total Page:16

File Type:pdf, Size:1020Kb

Intel Cluster Ready and Platform Open Cluster Stack: Clusters Made Simple HIGH-PERFORMANCE COMPUTING INTEL CLUSTER READY AND PLATFORM OpEN CLUSTER STACK: CLUSTERS MADE SIMPLE The Intel® Cluster Ready program is designed to provide a common standard for high-performance computing (HPC) clusters, helping organizations design and build seamless, compatible configurations. Integrating the standards and tools provided by this program with Platform™ Open Cluster Stack and certified Dell™ clusters can help significantly simplify the deployment and management of HPC clusters. s the cost of high-performance computing (HPC) The source of this complexity is the cluster architecture falls, a growing range of problems have become eco- itself. Cluster servers are fundamentally different from tra- Anomical to solve with compute clusters based on ditional symmetric multiprocessing servers, because they commodity hardware components. These problems range comprise individual units for processing and storage. In from risk management for insurance portfolios to design opti- addition, using clusters for either multiple jobs or to run mization and durability studies on automobile and aerospace software applications that can use multiple processors Related Categories: components, which can now be solved with levels of compute simultaneously requires high-speed interconnects as well High-performance power formerly reserved for only the biggest and most costly as workload management middleware. Because of these computing (HPC) problems. Even in areas where non-cluster HPC systems have differences, clusters also require different approaches to Parallel systems been in use for some time, such as oil and gas reservoir simu- specification, installation, and management than are used Platform Computing lation and life sciences clinical studies, commodity hardware in traditional IT environments. Visit DELL.COM/PowerSolutions enables analysis of larger problem sizes with more fidelity Until recently, it was difficult to ensure that HPC clusters for the complete category index. than was previously economically feasible. met a minimum set of standards: each cluster may have had According to IDC, the HPC technical server market, which different hardware and software components, and the is 50 percent compute clusters, is growing dramatically in resulting combinations may or may not have functioned in the divisional, departmental, and workgroup areas. This the same way. To help avoid this problem, Intel has collab- growth is largely attributable to the use of commodity com- oratively developed the Intel Cluster Ready program and ponents, which have dramatically reduced the price/ technology package with original equipment manufacturers, performance ratio of these systems. However, IDC still expresses channel members, and independent software vendors concern that adoption may be limited, because “clusters are (ISVs). By taking advantage of the standards and tools pro- increasingly complex to deploy and manage” and require vided by this program, and combining them with Platform advanced or specialist skill sets for IT personnel.1 Open Cluster Stack (OCS) software and certified Dell HPC This product includes software developed by the Rocks™ Cluster Group at the San Diego Supercomputer Center at the University of California, San Diego, and its contributors. 1 “Intel Cluster Ready,” by IDC, Doc #207312, June 2007. /PowerSolutions Reprinted from Dell Power Solutions, November 2007. Copyright © 2007 Dell Inc. All rights reserved. DELL.COM 109 HIGH-PERFORMANCE COMPUTING Platform OCS helps simplify the deployment and running tests periodically on the cluster and “ comparing the results with those of previous management of Intel Cluster Ready–certified clusters tests. Doing so helps detect deviations from the by installing and configuring Intel Cluster Ready soft- original cluster certification to help ensure that the cluster remains certified. ware components on Dell HPC platforms.” Combining Platform OCS and certified Dell HPC clusters Platform OCS is a pre-integrated, vendor- certified, modular software stack designed to clusters, organizations can help significantly increasing flexibility and helping reduce total streamline the deployment and management simplify the deployment and management of cost of ownership. of clusters running the Linux® OS. Backed by HPC clusters. The Intel Cluster Ready specification is a available global 24/7 enterprise support, it key part of the program, but the program con- transparently integrates open source and com- Introducing Intel Cluster Ready and Intel sists of more than just documentation. The Intel mercial software into a single consistent clus- Cluster Checker Cluster Checker, a script-based tool that per- ter operating environment. Platform OCS helps Intel and its partners have created the Intel forms direct computational tests and measure- simplify the deployment and management of Cluster Ready program to help simplify the ments, helps both vendors and IT organizations Intel Cluster Ready–certified clusters by definition, acquisition, installation, and man- ensure conformance to the specification, pro- installing and configuring Intel Cluster Ready agement of HPC clusters for organizations vides an objective measure of system perfor- software components on Dell HPC platforms without prior experience in cluster computing mance, and can assist in troubleshooting. (see Figure 2). and those working to increase their technical Figure 1 illustrates the architecture of the Intel Platform Load Sharing Facility (LSF®) HPC, computing capacity. By incorporating certified Cluster Checker engine. This tool is designed to a powerful, comprehensive, policy-driven work- hardware, cluster system software, applica- significantly reduce deployment time while load management application for engineering tion software, and cluster-ready configura- increasing uptime for certified clusters. and scientific distributed computing environ- tions, this program helps reduce both If a cluster passes all of the Cluster Checker ments, works in conjunction with Platform OCS deployment time and total cost of ownership— tests, it is considered Intel Cluster Ready certi- to intelligently schedule parallel and serial both of which can be critical in environments fied. Organizations can also use this tool to help workloads, helping maximize available comput- where HPC applications are delivering essen- ensure that the cluster continues operating ing resources. By utilizing hardware-specific tial competitive and strategic advantages. properly and within the specification simply by integrations, Platform LSF HPC and Platform Intel Cluster Ready provides a reference specification for ISVs and system builders to help validate HPC clusters as well as a set of configurations describing in detail how to com- Cluster definition and Standard output and log file configuration XML file (pass/fail results and diagnostics) bine components to create an Intel Cluster Ready–certified cluster. For IT organizations, Intel Cluster Checker engine the key feature of this program is that it speci- Application fies a common basis for clusters, allowing them Configuration Output Result Configuration Output Result programming to select from a variety of hardware and soft- interface ware components based on their cluster’s pur- Test module Test module pose and helping ensure that ISV applications Parallel Parallel operations Check operations Check that work on one certified cluster can also run Node Node Node Node Node Node Node Node reliably on a different certified cluster. This common basis significantly simplifies the pro- cesses of designing, building, acquiring, and deploying clusters based on Intel components, Figure 1. Intel Cluster Checker engine architecture This product includes software developed by the Rocks™ Cluster Group at the San Diego Supercomputer Center at the University of California, San Diego, and its contributors. DELL POWER SOLUTIONS | November 2007 110 Reprinted from Dell Power Solutions, November 2007. Copyright © 2007 Dell Inc. All rights reserved. • One 16-port Dell PowerConnect™ switch • KVM (keyboard, video, mouse) over IP switch, Intel Cluster Checker cables, server rack, cable management Application registered with Intel Cluster Ready program system, and power distribution system Intel C++ Compiler and Intel MPI Library Intel Fortran Compiler Runtime Environment Building a standard for simplified Cluster Intel Runtime Library Direct Access Programming Library cluster deployment installer Linux OS and Intel Cluster Math Kernel Library OpenFabrics Enterprise Distribution The Intel Cluster Ready program is designed to let organizations easily deploy and manage OS kernel Distributed Message file system fabric HPC clusters, helping eliminate the need to Cluster hardware node architecture create custom implementations in which they must individually install and configure each application while modifying the cluster hard- Figure 2. Intel Cluster Ready components installed and configured by Platform OCS ware and system software to meet these appli- cations’ requirements. This program also OCS also enable organizations to take advan- cluster management tools. They can use helps significantly simplify the work required tage of the high-performance network intercon- Platform OCS to quickly install and configure by commercial and noncommercial applica- nects available on clustered systems and Intel Cluster Ready–certified Dell HPC clusters tion vendors, who can focus on certifying their supercomputers. with the following components: applications
Recommended publications
  • IBM Platform Computing Solutions Reference Architectures and Best Practices
    Front cover IBM Platform Computing Solutions Reference Architectures and Best Practices Helps with the foundation to manage enterprise environments Delivers reference architectures and best practices guides Provides case scenarios Dino Quintero Luis Carlos Cruz Ricardo Machado Picone Dusan Smolej Daniel de Souza Casali Gheorghe Tudor Joanna Wong ibm.com/redbooks International Technical Support Organization IBM Platform Computing Solutions Reference Architectures and Best Practices April 2014 SG24-8169-00 Note: Before using this information and the product it supports, read the information in “Notices” on page v. First Edition (April 2014) This edition applies to RedHat 6.4, IBM Platform Cluster Manager Standard Edition (PCM-SE) 4.1.1, IBM Platform Symphony Advanced Edition 6.1.1, GPFS FPO 3.5.0.13, Hadoop 1.1.1. © Copyright International Business Machines Corporation 2014. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Notices . .v Trademarks . vi Preface . vii Authors. vii Now you can become a published author, too! . ix Comments welcome. ix Stay connected to IBM Redbooks . .x Chapter 1. Introduction. 1 1.1 Why IBM Platform Computing?. 2 1.2 High performance clusters . 2 1.3 IBM Platform HPC implementation scenario. 3 1.4 Big Data implementation on an IBM high performance computing cluster . 3 1.5 IBM Platform Computing solutions and products . 5 Intel . 7 Chapter 2. High performance clusters . 9 2.1 Cluster management. 10 2.1.1 IBM Platform HPC. 12 2.1.2 IBM Platform Cluster Manager Standard Edition . 19 2.1.3 IBM Platform Cluster Manager Advanced Edition.
    [Show full text]
  • Sun[Tm] Based Beowulf Cluster
    Sun[tm] Based Beowulf Cluster By Börje Lindh - Sun Microsystems AB, Sweden Sun BluePrints™ OnLine - December 2001 http://www.sun.com/blueprints Sun Microsystems, Inc. 901 San Antonio Road Palo Alto, CA 94303 USA 650 960-1300 fax 650 969-9131 Part No.: 816-3636-10 Revision 1.0, 12/03/01 Edition: December 2001 Copyright 2001 Sun Microsystems, Inc. 901 San Antonio Road, Palo Alto, California 94303 U.S.A. All rights reserved. This product or document is protected by copyright and distributed under licenses restricting its use, copying, distribution, and decompilation. No part of this product or document may be reproduced in any form by any means without prior written authorization of Sun and its licensors, if any. Third-party software, including font technology, is copyrighted and licensed from Sun suppliers. Parts of the product may be derived from Berkeley BSD systems, licensed from the University of California. UNIX is a registered trademark in the U.S. and other countries, exclusively licensed through X/Open Company, Ltd. Sun, Sun Microsystems, the Sun logo, Sun Blade, Sun BluePrints, Sun Fire, Sun HPC ClusterTools, SunVTS, Forte, Prism, Netra, JumpStart, and Solaris are trademarks, registered trademarks, or service marks of Sun Microsystems, Inc. in the U.S. and other countries. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the U.S. and other countries. Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems, Inc. The Energy Star logo is a registered trademark of EPA The OPEN LOOK and Sun™ Graphical User Interface was developed by Sun Microsystems, Inc.
    [Show full text]
  • OGF Director, NSF Cloud and Autonomic Computing Center Senior Scientist, High Performance Computing Center Adjunct Professor of Physics Texas Tech University
    The Open Grid Forum: History, Introduction and Process Alan Sill VP of Standards, OGF Director, NSF Cloud and Autonomic Computing Center Senior Scientist, High Performance Computing Center Adjunct Professor of Physics Texas Tech University Open Grid Forum 44, May 21-22, 2015 EGI Conference, Lisbon, Portugal ©2015 Open Grid Forum 1 About the Open Grid Forum: Open Grid Forum (OGF) is a global organization operating in the areas of cloud, grid and related forms of advanced distributed computing. The OGF community pursues these topics through an open process for development, creation and promotion of relevant specifications and use cases. OGF actively engages partners and participants throughout the international arena through an open forum with open processes to champion architectural blueprints related to cloud and grid computing. The resulting specifications and standards enable pervasive adoption of advanced distributed computing techniques for business and research worldwide. © 2015 Open Grid Forum OGF 44 - EGI Conference Lisbon, Portugal May 18-22, 2015 2 History and Background • Began in 2001 as an organization to promote the advancement of distributed computing worldwide. • Grid Forum --> Global Grid Forum --> GGF + Enterprise Grid Alliance --> formation of OGF in 2005. • Mandate is to take on all forms of distributed computing and to work to promote cooperation, information exchange, and best practices in use and standardization. • OGF is best known for a series of important computing, security and network standards that form the basis for major science and business-based distributed computing (BES, GridFTP, DRMAA, JSDL, RNS, GLUE, UR, etc.). • We also develop cloud, networking and data standards (OCCI, DFDL, WS-Agreement, NSI/NML, etc.) in wide use.
    [Show full text]
  • IBM Platform Computing Integration Solutions
    Front cover IBM Platform Computing Integration Solutions Learn about solutions that integrate IBM Platform Symphony and IBM InfoSphere BigInsights for analytics See how the solutions use IBM System x and IBM GPFS Understand details about IBM Platform LSF implementation Dino Quintero Ricardo Dobelin Barros Ashraf Gomaa José Higino Archana Kumar Majid Ouassir Adam Parker Joanna Wong ibm.com/redbooks International Technical Support Organization IBM Platform Computing Integration Solutions March 2013 SG24-8081-00 Note: Before using this information and the product it supports, read the information in “Notices” on page vii. First Edition (March 2013) This edition applies to IBM Platform Symphony 6.1, IBM Platform Symphony 5.2, IBM Platform LSF 8.3, IBM InfoSphere BigInsights 1.4, Red Hat Enterprise Linux (RHEL) 6.2 x86_64, and RHEL 6.3 x86_64. © Copyright International Business Machines Corporation 2013. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Notices . vii Trademarks . viii Preface . ix The team who wrote this book . ix Now you can become a published author, too! . xi Comments welcome. xi Stay connected to IBM Redbooks . xi Chapter 1. Products and portfolio for IBM Platform Computing solutions . 1 1.1 IBM Platform LSF . 2 1.1.1 IBM Platform LSF editions . 2 1.1.2 IBM Platform LSF add-ons . 4 1.1.3 New in the IBM Platform LSF V9.1 family. 4 1.2 IBM Platform HPC. 4 1.3 IBM Platform Symphony . 5 1.3.1 IBM Platform Symphony edition features .
    [Show full text]
  • A Networking Approach to Grid Computing
    TEAM LinG - Live, Informative, Non-cost and Genuine! ffirs.qxd 10/11/2004 3:44 PM Page i A NETWORKING APPROACH TO GRID COMPUTING TEAM LinG - Live, Informative, Non-cost and Genuine! ffirs.qxd 10/11/2004 3:44 PM Page ii TEAM LinG - Live, Informative, Non-cost and Genuine! ffirs.qxd 10/11/2004 3:44 PM Page iii A NETWORKING APPROACH TO GRID COMPUTING DANIEL MINOLI Managing Director Leading-Edge Networks Incorporated A JOHN WILEY & SONS, INC., PUBLICATION TEAM LinG - Live, Informative, Non-cost and Genuine! ffirs.qxd 10/11/2004 3:44 PM Page iv Copyright © 2005 by John Wiley & Sons, Inc. All rights reserved. Published by John Wiley & Sons, Inc., Hoboken, New Jersey. Published simultaneously in Canada. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008. Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representation or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose.
    [Show full text]
  • IBM Platform Computing Solutions
    Front cover IBM Platform Computing Solutions Describes the offering portfolio Shows implementation scenarios Delivers add-on value to current implementations Dino Quintero Scott Denham Rodrigo Garcia da Silva Alberto Ortiz Aline Guedes Pinto Atsumori Sasaki Roger Tucker Joanna Wong Elsie Ramos ibm.com/redbooks International Technical Support Organization IBM Platform Computing Solutions December 2012 SG24-8073-00 Note: Before using this information and the product it supports, read the information in “Notices” on page vii. First Edition (December 2012) This edition applies to IBM Platform High Performance Computing (HPC) v3.2, IBM Platform Cluster Manager Advanced Edition v3.2, IBM Platform Symphony v5.2, IBM Platform Load Sharing Facility (LSF) v8.3, RedHat Enterprise Linux v6.2/v6.3, Hadoop v1.0.1, Sun Java Development Kit (JDK) v1.6.0_25, and Oracle Database Express Edition v11.2.0. © Copyright International Business Machines Corporation 2012. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Notices . vii Trademarks . viii Preface . ix The team who wrote this book . ix Now you can become a published author, too! . xi Comments welcome. xi Stay connected to IBM Redbooks . xii Chapter 1. Introduction to IBM Platform Computing . 1 1.1 IBM Platform Computing solutions value proposition . 2 1.1.1 Benefits . 3 1.1.2 Cluster, grids, and clouds . 4 1.2 History . 5 1.2.1 IBM acquisition . 6 Chapter 2. Technical computing software portfolio. 7 2.1 IBM Platform Computing product portfolio . 8 2.1.1 Workload management.
    [Show full text]