DOCSLIB.ORG

XC User Guide Version 4.0

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
XC User Guide Version 4.0 HP XC System Software XC User Guide Version 4.0 HP Part Number: A-XCUSR-40a Published: February 2009 © Copyright 2003, 2005, 2006, 2007, 2008, 2009 Hewlett-Packard Development Company, L.P. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. AMD and AMD Opteron are trademarks or registered trademarks of Advanced Micro Devices, Inc. Intel , the Intel logo, Itanium , and Xeon are trademarks or registered trademarks of Intel Corporation in the United States and other countries. Linux is a U.S. registered trademark of Linus Torvalds. LSF, Platform Computing, and the LSF and Platform Computing logos are trademarks or registered trademarks of Platform Computing, Inc. Lustre is a trademark of Cluster File Systems, Inc. Myrinet and Myricom are registered trademarks of Myricom, Inc. Quadrics and QSNet II are registered trademarks of Quadrics, Ltd. The Portland Group and PGI are trademarks or registered trademarks of The Portland Group Compiler Technology, STMicroelectronics, Inc. Red Hat is a registered trademark of Red Hat Inc. syslog-ng is a copyright of BalaBit IT Security. SystemImager is a registered trademark of Brian Finley. TotalView is a registered trademark of Etnus, Inc. UNIX is a registered trademark of The Open Group. Table of Contents About This Document.......................................................................................................11 Intended Audience................................................................................................................................11 New and Changed Information in This Edition...................................................................................11 Typographic Conventions.....................................................................................................................11 HP XC and Related HP Products Information.....................................................................................12 Related Information..............................................................................................................................13 Manpages..............................................................................................................................................16 HP Encourages Your Comments..........................................................................................................17 1 Overview of the User Environment.............................................................................19 1.1 System Architecture........................................................................................................................19 1.1.1 HP XC System Software..........................................................................................................19 1.1.2 Operating System....................................................................................................................19 1.1.3 Node Platforms.......................................................................................................................19 1.1.4 Node Specialization.................................................................................................................20 1.1.5 Storage and I/O........................................................................................................................21 1.1.6 File System...............................................................................................................................21 1.1.7 System Interconnect Network.................................................................................................22 1.1.8 Network Address Translation (NAT)......................................................................................23 1.2 Determining System Configuration Information............................................................................23 1.3 User Environment...........................................................................................................................23 1.3.1 LVS...........................................................................................................................................23 1.3.2 Modules...................................................................................................................................23 1.3.3 Commands..............................................................................................................................24 1.4 Application Development Environment.........................................................................................24 1.4.1 Parallel Applications...............................................................................................................24 1.4.2 Serial Applications..................................................................................................................25 1.5 Run-Time Environment...................................................................................................................25 1.5.1 SLURM....................................................................................................................................25 1.5.2 Load Sharing Facility (LSF).....................................................................................................25 1.5.3 Standard LSF...........................................................................................................................25 1.5.4 How LSF and SLURM Interact................................................................................................26 1.5.5 HP-MPI....................................................................................................................................26 1.6 Components, Tools, Compilers, Libraries, and Debuggers............................................................27 2 Using the System..........................................................................................................29 2.1 Logging In to the System.................................................................................................................29 2.1.1 LVS Login Routing..................................................................................................................29 2.1.2 Using the Secure Shell to Log In.............................................................................................29 2.2 Overview of Launching and Managing Jobs..................................................................................29 2.2.1 Introduction.............................................................................................................................30 2.2.2 Getting Information About Queues........................................................................................30 2.2.3 Getting Information About Resources....................................................................................30 2.2.4 Getting Information About System Partitions........................................................................31 2.2.5 Launching Jobs........................................................................................................................31 2.2.6 Getting Information About Your Jobs.....................................................................................31 2.2.7 Stopping and Suspending Jobs...............................................................................................31 2.2.8 Resuming Suspended Jobs......................................................................................................31 2.3 Performing Other Common User Tasks..........................................................................................31 Table of Contents 3 2.3.1 Determining the LSF Cluster Name and the LSF Execution Host..........................................32 2.4 Getting System Help and Information............................................................................................32 3 Configuring Your Environment with Modulefiles.......................................................33 3.1 Overview of Modules......................................................................................................................33 3.2 Supplied Modulefiles......................................................................................................................34 3.3 Modulefiles Automatically Loaded on the System.........................................................................36 3.4 Viewing Available Modulefiles.......................................................................................................36 3.5 Viewing Loaded Modulefiles..........................................................................................................36 3.6 Loading a Modulefile......................................................................................................................36 3.6.1 Loading a Modulefile for the Current Session........................................................................36
Load more

Recommended publications
  • Tree-Like Distributed Computation Environment with Shapp Library
    information Article Tree-Like Distributed Computation Environment with Shapp Library Tomasz Gałecki and Wiktor Bohdan Daszczuk * Institute of Computer Science, Warsaw University of Technology, 00-665 Warsaw, Poland; [email protected] * Correspondence: [email protected]; Tel.: +48-22-234-78-12 Received: 30 January 2020; Accepted: 1 March 2020; Published: 3 March 2020 Abstract: Despite the rapidly growing computing power of computers, it is often insufficient to perform mass calculations in a short time, for example, simulation of systems for various sets of parameters, the searching of huge state spaces, optimization using ant or genetic algorithms, machine learning, etc. One can solve the problem of a lack of computing power through workload management systems used in local networks in order to use the free computing power of servers and workstations. This article proposes raising such a system to a higher level of abstraction: The use in the .NET environment of a new Shapp library that allows remote task execution using fork-like operations from Portable Operating System Interface for UNIX (POSIX) systems. The library distributes the task code, sending static data on which task force is working, and individualizing tasks. In addition, a convenient way of communicating distributed tasks running hierarchically in the Shapp library was proposed to better manage the execution of these tasks. Many different task group architectures are possible; we focus on tree-like calculations that are suitable for many problems where the range of possible parallelism increases as the calculations progress. Keywords: workload management; remote fork; distributed computations; task group communication 1.
    [Show full text]
  • Administration Guide Administration Guide SUSE Linux Enterprise High Availability Extension 15 SP1 by Tanja Roth and Thomas Schraitle
    SUSE Linux Enterprise High Availability Extension 15 SP1 Administration Guide Administration Guide SUSE Linux Enterprise High Availability Extension 15 SP1 by Tanja Roth and Thomas Schraitle This guide is intended for administrators who need to set up, congure, and maintain clusters with SUSE® Linux Enterprise High Availability Extension. For quick and ecient conguration and administration, the product includes both a graphical user interface and a command line interface (CLI). For performing key tasks, both approaches are covered in this guide. Thus, you can choose the appropriate tool that matches your needs. Publication Date: September 24, 2021 SUSE LLC 1800 South Novell Place Provo, UT 84606 USA https://documentation.suse.com Copyright © 2006–2021 SUSE LLC and contributors. All rights reserved. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or (at your option) version 1.3; with the Invariant Section being this copyright notice and license. A copy of the license version 1.2 is included in the section entitled “GNU Free Documentation License”. For SUSE trademarks, see http://www.suse.com/company/legal/ . All other third-party trademarks are the property of their respective owners. Trademark symbols (®, ™ etc.) denote trademarks of SUSE and its aliates. Asterisks (*) denote third-party trademarks. All information found in this book has been compiled with utmost attention to detail. However, this does not guarantee complete accuracy. Neither SUSE
    [Show full text]
  • Introduction to Linux Virtual Server and High Availability
    Outlines Introduction to Linux Virtual Server and High Availability Chen Kaiwang [email protected] December 5, 2011 Chen Kaiwang [email protected] LVS-DR and Keepalived Outlines If you don't know the theory, you don't have a way to be rigorous. Robert J. Shiller http://www.econ.yale.edu/~shiller/ Chen Kaiwang [email protected] LVS-DR and Keepalived Outlines Misery stories I Jul 2011 Too many connections at zongheng.com I Aug 2011 Realserver maintenance at 173.com quiescent persistent connections I Nov 2011 Health check at 173.com I Nov 2011 Virtual service configuration at 173.com persistent session data Chen Kaiwang [email protected] LVS-DR and Keepalived Outlines Outline of Part I Introduction to Linux Virtual Server Configuration Overview Netfilter Architecture Job Scheduling Scheduling Basics Scheduling Algorithms Connection Affinity Persistence Template Persistence Granularity Quirks Chen Kaiwang [email protected] LVS-DR and Keepalived Outlines Outline of Part II HA Basics LVS High Avaliablity Realserver Failover Director Failover Solutions Heartbeat Keepalived Chen Kaiwang [email protected] LVS-DR and Keepalived LVS Intro Job Scheduling Connection Affinity Quirks Part I Introduction to Linux Virtual Server Chen Kaiwang [email protected] LVS-DR and Keepalived LVS Intro Job Scheduling Configuration Overview Connection Affinity Netfilter Architecture Quirks Introduction to Linux Virtual Server Configuration Overview Netfilter Architecture Job Scheduling Scheduling Basics Scheduling Algorithms Connection Affinity Persistence Template Persistence Granularity Quirks Chen Kaiwang [email protected] LVS-DR and Keepalived LVS Intro Job Scheduling Configuration Overview Connection Affinity Netfilter Architecture Quirks A Linux Virtual Serverr (LVS) is a group of servers that appear to the client as one large, fast, reliable (highly available) server.
    [Show full text]
  • Keepalived User Guide Release 1.4.3
    Keepalived User Guide Release 1.4.3 Alexandre Cassen and Contributors March 06, 2021 Contents 1 Introduction 1 2 Software Design 3 3 Load Balancing Techniques 11 4 Installing Keepalived 13 5 Keepalived configuration synopsis 17 6 Keepalived programs synopsis 23 7 IPVS Scheduling Algorithms 27 8 IPVS Protocol Support 31 9 Configuring SNMP Support 33 10 Case Study: Healthcheck 37 11 Case Study: Failover using VRRP 43 12 Case Study: Mixing Healthcheck & Failover 47 13 Terminology 51 14 License 53 15 About These Documents 55 16 TODO List 57 Index 59 i ii CHAPTER 1 Introduction Load balancing is a method of distributing IP traffic across a cluster of real servers, providing one or more highly available virtual services. When designing load-balanced topologies, it is important to account for the availability of the load balancer itself as well as the real servers behind it. Keepalived provides frameworks for both load balancing and high availability. The load balancing framework relies on the well-known and widely used Linux Virtual Server (IPVS) kernel module, which provides Layer 4 load balancing. Keepalived implements a set of health checkers to dynamically and adaptively maintain and manage load balanced server pools according to their health. High availability is achieved by the Virtual Redundancy Routing Protocol (VRRP). VRRP is a fundamental brick for router failover. In addition, keepalived implements a set of hooks to the VRRP finite state machine providing low-level and high-speed protocol interactions. Each Keepalived framework can be used independently or together to provide resilient infrastructures. In this context, load balancer may also be referred to as a director or an LVS router.
    [Show full text]
  • Ubuntu Server Guide Basic Installation Preparing to Install
    Ubuntu Server Guide Welcome to the Ubuntu Server Guide! This site includes information on using Ubuntu Server for the latest LTS release, Ubuntu 20.04 LTS (Focal Fossa). For an offline version as well as versions for previous releases see below. Improving the Documentation If you find any errors or have suggestions for improvements to pages, please use the link at thebottomof each topic titled: “Help improve this document in the forum.” This link will take you to the Server Discourse forum for the specific page you are viewing. There you can share your comments or let us know aboutbugs with any page. PDFs and Previous Releases Below are links to the previous Ubuntu Server release server guides as well as an offline copy of the current version of this site: Ubuntu 20.04 LTS (Focal Fossa): PDF Ubuntu 18.04 LTS (Bionic Beaver): Web and PDF Ubuntu 16.04 LTS (Xenial Xerus): Web and PDF Support There are a couple of different ways that the Ubuntu Server edition is supported: commercial support and community support. The main commercial support (and development funding) is available from Canonical, Ltd. They supply reasonably- priced support contracts on a per desktop or per-server basis. For more information see the Ubuntu Advantage page. Community support is also provided by dedicated individuals and companies that wish to make Ubuntu the best distribution possible. Support is provided through multiple mailing lists, IRC channels, forums, blogs, wikis, etc. The large amount of information available can be overwhelming, but a good search engine query can usually provide an answer to your questions.
    [Show full text]
  • HP/ISV Confidential
    Advanced Cluster Software for Meteorology Betty van Houten 1 Frederic Ciesielski / Gavin Brebner / Ghislain de Jacquelot 2 Michael Riedmann 3 Henry Strauss 4 1 HPC Division Richardson 2 HPC Competency Centre Grenoble 3 EPC Boeblingen 4 HPC PreSales Munich ECMWF workshop “Use of HPC in Meteorology” -- Reading/UK, November 2nd, 2006 © 2006 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice 2 Nov 2, 2006 Advanced Cluster Software for Meteorology Outline • requirements in met • HP‘s ways to address those − XC – a cluster environment for real-world applications − HP-MPI – a universal approach − some thoughts on checkpoint/restart and alternatives − future work • LM on HP – status & results • conclusions 3 Nov 2, 2006 Advanced Cluster Software for Meteorology Requirements • both capability and capacity • reliability & turn-around times • data management! • visualization? 4 Nov 2, 2006 Advanced Cluster Software for Meteorology Cluster Implementation Challenges • Manageability • Scalability • Integration of Data Mgmt & Visualization • Interconnect/Network Complexity • Version Control • Application Availability 5 Nov 2, 2006 Advanced Cluster Software for Meteorology XC Cluster: HP’s Linux-Based Production Cluster for HPC • A production computing environment for HPC built on Linux/Industry standard clusters − Industrial-hardened, scalable, supported − Integrates leading technologies from open source and partners • Simple and complete product for real world usage − Turn-key, with single
    [Show full text]
  • Parallel Processing.Pptx
    Compute Grid: Parallel Processing RCS Lunch & Learn Training Series Bob Freeman, PhD Director, Research Technology Operations HBS 8 November, 2017 Overview • Q&A • Introduction • Serial vs parallel • Approaches to Parallelization • Submitting parallel jobs on the compute grid • Parallel tasks • Parallel Code Serial vs Parallel work Serial vs Multicore Approaches Traditionally, software has been written for serial computers • To be run on a single computer having a single Central Processing Unit (CPU) • Problem is broken into a discrete set of instructions • Instructions are executed one after the other • One one instruction can be executed at any moment in time Serial vs Multicore Approaches In the simplest sense, parallel computing is the simultaneous use of multiple compute resources to solve a computational problem: • To be run using multiple CPUs • A problem is broken into discrete parts (either by you or the application itself) that can be solved concurrently • Each part is further broken down to a series of instructions • Instructions from each part execute simultaneously on different CPUs or different machines Serial vs Multicore Approaches Many different parallelization approaches, which we won't discuss: Shared memory Distributed memory 6 Hybrid Distributed-Shared memory Parallel Processing… So, we are going to briefly touch on two approaches: • Parallel tasks • Tasks in the background • gnu_parallel • Pleasantly parallelizing • Parallel code • Considerations for parallelizing • Parallel frameworks & examples We will not discuss parallelized frameworks such as Hadoop, Apache Spark, MongoDB, ElasticSearch, etc Parallel Jobs on the Compute Grid… Nota Bene!! • In order to run in parallel, programs (code) must be explicitly programmed to do so. • And you must ask the scheduler to reserve those cores for your program/work to use.
    [Show full text]
  • Protectv Installation Guide (AWS)
    ProtectV Installation Guide (AWS) Technical Manual Template 1 Release 1.0, PN: 000-000000-000, Rev. A, March 2013, Copyright © 2013 SafeNet, Inc. All rights reserved. Document Information Product Version 1.7 Document Part Number 007-011532-001, Rev R Release Date March 2014 Trademarks All intellectual property is protected by copyright. All trademarks and product names used or referred to are the copyright of their respective owners. No part of this document may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, chemical, photocopy, recording, or otherwise, without the prior written permission of SafeNet, Inc. • Linux® is a registered trademark of Linus Torvalds. Linux Foundation, Linux Standard Base, LSB, LSB Certified, IAccessible2, MeeGo are registered trademarks of the Linux Foundation. Copyright © 2010 Linux Foundation. All rights reserved. • Windows is a registered trademark of Microsoft Corporation in the United States and other countries. • Amazon Web Services™ and AWS™ are registered trademarks of Amazon.com, Inc. or its affiliates in the United States and other countries. • Red Hat® Linux® is a registered trademark of Red Hat, Inc. in the United States and other countries. • Ubuntu™ is a registered trademark of Canonical Ltd. Disclaimer SafeNet makes no representations or warranties with respect to the contents of this document and specifically disclaims any implied warranties of merchantability or fitness for any particular purpose. Furthermore, SafeNet reserves the right to revise this publication and to make changes from time to time in the content hereof without the obligation upon SafeNet to notify any person or organization of any such revisions or changes.
    [Show full text]
  • Pash: Light-Touch Data-Parallel Shell Processing
    PaSh: Light-touch Data-Parallel Shell Processing Nikos Vasilakis∗ Konstantinos Kallas∗ Konstantinos Mamouras MIT University of Pennsylvania Rice University [email protected] [email protected] [email protected] Achilles Benetopoulos Lazar Cvetković Unaffiliated University of Belgrade [email protected] [email protected] Abstract Parallelizability Parallelizing Runtime Classes §3 Transformations §4.3 Primitives §5 Dataflow This paper presents PaSh, a system for parallelizing POSIX POSIX, GNU §3.1 Regions shell scripts. Given a script, PaSh converts it to a dataflow Annotations §3.2 § 4.1 DFG § 4.4 graph, performs a series of semantics-preserving program §4.2 transformations that expose parallelism, and then converts Seq. Script Par. Script the dataflow graph back into a script—one that adds POSIX constructs to explicitly guide parallelism coupled with PaSh- Fig. 1. PaSh overview. PaSh identifies dataflow regions (§4.1), converts provided Unix-aware runtime primitives for addressing per- them to dataflow graphs (§4.2), applies transformations (§4.3) based onthe parallelizability properties of the commands in these regions (§3.1, §3.2), formance- and correctness-related issues. A lightweight an- and emits a parallel script (§4.4) that uses custom primitives (§5). notation language allows command developers to express key parallelizability properties about their commands. An accompanying parallelizability study of POSIX and GNU • Command developers, responsible for implementing indi- commands—two large and commonly used groups—guides vidual commands such as sort, uniq, and jq. These de- the annotation language and optimized aggregator library velopers usually work in a single programming language, that PaSh uses. PaSh’s extensive evaluation over 44 unmod- leveraging its abstractions to provide parallelism when- ified Unix scripts shows significant speedups (0.89–61.1×, ever possible.
    [Show full text]
  • Virtually Linux Virtualization Techniques in Linux
    Virtually Linux Virtualization Techniques in Linux Chris Wright OSDL [email protected] Abstract ware1 or software [16, 21, 19], may include any subset of a machine’s resources, and has Virtualization provides an abstraction layer a wide variety of applications. Such usages mapping a virtual resource to a real resource. include machine emulation, hardware consol- Such an abstraction allows one machine to be idation, resource isolation, quality of service carved into many virtual machines as well as resource allocation, and transparent resource allowing a cluster of machines to be viewed redirection. Applications of these usage mod- as one. Linux provides a wealth of virtual- els include virtual hosting, security, high avail- ization offerings. The technologies range in ability, high throughput, testing, and ease of the problems they solve, the models they are administration. useful in, and their respective maturity. This It is interesting to note that differing virtual- paper surveys some of the current virtualiza- ization models may have inversely correlated tion techniques available to Linux users, and proportions of virtual to physical resources. it reviews ways to leverage these technologies. For example, the method of carving up a sin- Virtualization can be used to provide things gle machine into multiple machines—useful such as quality of service resource allocation, in hardware consolidation or virtual hosting— resource isolation for security or sandboxing, looks quite different from a single system im- transparent resource redirection for availability age (SSI) [15]—useful in clustering. This pa- and throughput, and simulation environments per primarily focuses on providing multiple for testing and debugging. virtual instances of a single physical resource, however, it does cover some examples of a sin- 1 Introduction gle virtual resource mapping to multiple phys- ical resources.
    [Show full text]
  • GNU Astronomy Utilities
    GNU Astronomy Utilities Astronomical data manipulation and analysis programs and libraries for version 0.7, 8 August 2018 Mohammad Akhlaghi Gnuastro (source code, book and webpage) authors (sorted by number of commits): Mohammad Akhlaghi ([email protected], 1101) Mos`eGiordano ([email protected], 29) Vladimir Markelov ([email protected], 18) Boud Roukema ([email protected], 7) Leindert Boogaard ([email protected], 1) Lucas MacQuarrie ([email protected], 1) Th´er`eseGodefroy ([email protected], 1) This book documents version 0.7 of the GNU Astronomy Utilities (Gnuastro). Gnuastro provides various programs and libraries for astronomical data manipulation and analysis. Copyright c 2015-2018 Free Software Foundation, Inc. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled \GNU Free Documentation License". For myself, I am interested in science and in philosophy only because I want to learn something about the riddle of the world in which we live, and the riddle of man's knowledge of that world. And I believe that only a revival of interest in these riddles can save the sciences and philosophy from narrow specialization and from an obscurantist faith in the expert's special skill, and in his personal knowledge and authority; a faith that so well fits our `post-rationalist' and `post- critical' age, proudly dedicated to the destruction of the tradition of rational philosophy, and of rational thought itself.
    [Show full text]
  • OSS Alphabetical List and Software Identification
    Annex: OSS Alphabetical list and Software identification Software Short description Page A2ps a2ps formats files for printing on a PostScript printer. 149 AbiWord Open source word processor. 122 AIDE Advanced Intrusion Detection Environment. Free replacement for Tripwire(tm). It does the same 53 things are Tripwire(tm) and more. Alliance Complete set of CAD tools for the specification, design and validation of digital VLSI circuits. 114 Amanda Backup utility. 134 Apache Free HTTP (Web) server which is used by over 50% of all web servers worldwide. 106 Balsa Balsa is the official GNOME mail client. 96 Bash The Bourne Again Shell. It's compatible with the Unix `sh' and offers many extensions found in 147 `csh' and `ksh'. Bayonne Multi-line voice telephony server. 58 Bind BIND "Berkeley Internet Name Daemon", and is the Internet de-facto standard program for 95 turning host names into IP addresses. Bison General-purpose parser generator. 77 BSD operating FreeBSD is an advanced BSD UNIX operating system. 144 systems C Library The GNU C library is used as the C library in the GNU system and most newer systems with the 68 Linux kernel. CAPA Computer Aided Personal Approach. Network system for learning, teaching, assessment and 131 administration. CVS A version control system keeps a history of the changes made to a set of files. 78 DDD DDD is a graphical front-end for GDB and other command-line debuggers. 79 Diald Diald is an intelligent link management tool originally named for its ability to control dial-on- 50 demand network connections. Dosemu DOSEMU stands for DOS Emulation, and is a linux application that enables the Linux OS to run 138 many DOS programs - including some Electric Sophisticated electrical CAD system that can handle many forms of circuit design.
    [Show full text]