Compliant Cloud+Campus Hybrid HPC Infrastructure

Total Page:16

File Type:pdf, Size:1020Kb

Compliant Cloud+Campus Hybrid HPC Infrastructure Compliant Cloud+Campus Hybrid HPC Infrastructure Brenden Judson Matt Vander Werf, Paul Brenner Computer Science and Engineering Center for Research Computing University of Notre Dame University of Notre Dame [email protected] fmvanderw,[email protected] Abstract—We present a hybrid cloud+campus model for implement an information security and protection program. deploying secure research cyberinfrastructure in line with a On November 4, 2010, President Barack Obama issued Ex- growing number of federally funded scientific research security ecutive Order 13556 with the goal of establishing an “open requirements. While the security mechanisms are inline with many regulations, we specifically focus on the NIST 800-171 CUI and uniform program for managing” controlled unclassified compliance that is now required by many US DARPA funded information (CUI) [4]. Federal agencies often contract with contracts. We discuss our architectural framework and rationale private organizations, resulting in some their sensitive data for leveraging shared ND services in AWS GovCloud in concert being housed externally. This led to the creation of NIST with CUI compliant HPC systems on our campus in a hybrid Special Publication (SP) 800-171 Rev. 1 in 2016, which set the fashion; allowing an individual CUI regulated research project to bridge two connected but distinct infrastructures. regulation standards for CUI [5] data resident in non federal Index Terms—CUI, FISMA, HPC, NIST, AWS GovCloud, IT infrastructure. Since then, a growing number of agencies NIST SP 800-171, Thin Client have been requiring companies and institutions housing their information to meet NIST SP 800-171 requirements, often I. INTRODUCTION referred to as CUI compliance. Specific to the research projects The widespread embrace of distributed and data-centric in this work, the Defense Advanced Research Projects Agency platforms such as the Internet of Things (IoT), cloud storage, (DARPA) requires CUI compliance for many of their projects. machine learning, etc. has been driving high performance Multiple federal grants at the University of Notre Dame (ND) computational research infrastructure, which has traditionally now have the requirement of CUI compliance. been computation-centric, toward data-centric models. World- As CUI compliance continues to become the norm when wide, the amount and diversity of data continues to grow working with certain federal agencies; many companies, uni- exponentially. A recent prediction, made by the International versities, and institutions look to make their information Data Corporation (IDC), claimed that by the year 2025, 163 systems CUI compliant. Recent work highlights this trend zettabytes of data will be created globally each year [1]. and how compliance has become such a relevant topic. For As such, it is getting increasingly difficult to contain and example, in the HPC Security & Compliance Workshop regulate. Recent news stories surrounding data privacy laws (PEARC18), Pennsylvania State University's Joseph Gridley such as Facebook's Zuckerburg hearing before congress [2], gave a presentation on the “Validation of CUI Environments “fake news” influence of democratic elections, and others like Using NIST 800-171A” [6] and Preston Smith outlined the them have brought the value, risk and power of leveraging University of Purdue's compliant environment in his presenta- data to the forefront of society's attention (including legisla- tion, “REED: from Service to Ecosystem” [7]. This workshop tors); making data regulation increasingly more important and featured many other relevant presentations that can be found commonplace. on their website [8]. Other work such as Kelly W. Bennett's While society's attention to data security has recently in- paper on a cloud-based security architecture [9] and the creased, the federal regulation of sensitive digital data is not University of Arizona's CUI compliant environment [10] are a new government focus. The Health Insurance Portability additional examples of other institution's work that pertains to and Accountability Act of 1996 (HIPAA) placed regulations compliance. Although there is a great deal of relevant work on medical information. Similarly, personally identifiable in- being done, there is very little documentation or published formation (PII) and classified information have been highly work on hybrid compliant environments. regulated for decades. Major strides towards regulation were As outlined by NIST SP 800-171, there are fourteen cate- taken in 2002 when the Federal Information Security Man- gories associated with being CUI complaint: agement Act (FISMA) became law, requiring federal agencies 1) Awareness and Training to develop, document, and implement an information security 2) Auditing and Accountability and protection program (revised on December 8, 2014) [3]. 3) Configuration Management FISMA tasked the National Institute of Standards and Tech- 4) Identification and Authentication nology (NIST) to create standards outlining how to properly 5) Incident Response 6) Maintenance can simplify your responsibility in meeting compliance, saving 7) Media Protection you time and money. Cloud services also help you avoid the 8) Personnel Security headaches that accompany having physical hardware on-site 9) Physical Protection (compliance audits, maintenance, deterioration, etc.). Instead, 10) Risk Assessment the cloud vendor assures you state-of-the-art hardware and 11) Security Assessment enhanced security measures. Large public clouds may also 12) System and Communication Protection have newer and more powerful individual servers than in the 13) System and Information Integrity average HPC environment (an optimized InfiniBand network is 14) Access Control a separate matter). One of the reasons for this is the fact that Each one of these categories is broken down into many many companies and organizations have been keeping their controls that specify the exact compliance requirements, for on-site HPC systems longer than ever before [16]. more details see [11]. Cloud technologies are also more elastic and scalable than In recognition of the expanding market for cloud services traditional on-campus HPC infrastructures. Being able to scale that meet various federal regulations, all three of the major with demand provides agility to your CUI environment that US public cloud vendors have taken strides to expand this has not been available in the past, providing users with aspect of their business [12] [13] [14]. The University of more options. For many customers, AWS allows them to Notre Dame's CUI environment utilizes the Amazon Web scale beyond what would be practical in their on-campus Services (AWS) GovCloud region to implement a compliant environments without the need for large capital investment. environment and therefore, AWS is the cloud vendor focused AWS also offers cost saving features such as spot instances, on for the remainder of this paper. AWS GovCloud (US) is which allow you to purchase spare compute capacity at steep an isolated AWS region designed to allow customers to move discounts. sensitive workloads into a cloud environment that addresses B. Campus CUI Environment their specific regulatory and compliance requirements. The region is operated exclusively by employees who are U.S. Today, most aspects of enterprise information technology citizens, only on U.S. soil, and is only accessible to vetted (IT) at ND have been widely migrated to the cloud. HPC is U.S. entities. While not all CUI regulated projects have a also gaining traction in cloud usage. Cloud spending by HPC US citizenship requirement, it is a common requirement to customers grew by 44 percent from 2016 to 2017 and it is meet certain federal regulations, including International Traf- projected to be the fastest-growing expense for HPC centers fic in Arms Regulations (ITAR) and Export Administration for the next five years, reaching nearly $3 billion by 2022 [16]. Regulations (EAR). These GovCloud features (along with All three major cloud providers now support some degree of others) allow AWS to help customers meet a variety of HPC [17] [18] [19] and the case for HPC in the cloud is compliance requirements such as HIPAA, ITAR, EAR, Federal growing ever stronger. However, all three lack the capability Risk and Authorization Management Program (FedRAMP), to cost effectively support the large scale (>1-10K tightly FISMA High Baselines and the DOD Security Requirements coupled cores) and high utilization rates typical of top-tier Guide. HPC centers operated by many research institutions such as Recent research grants that the University of Notre Dame Notre Dame. has become involved with, such as those sought by the HPC environments consistently have very high utilization ND Turbomachinery Laboratory (NDTL) [15], have provided percentages, much higher than general or enterprise IT sys- the unique opportunity to create a hybrid (cloud+campus) tems. Cloud services are priced based on their utilization and CUI environment that contains the necessary infrastructure thus, cloud cycles continue to be more expensive than in- to perform high performance computing (HPC). This paper house cycles for HPC [16]. Furthermore, hosting your secure continues by comparing the trade-offs between hosting a CUI infrastructure in the GovCloud region adds additional fees HPC environment in AWS GovCloud or with on-campus to AWS's standard fees. Many customers will find that their hardware. From there, we outline Notre Dame's hybrid CUI research computing
Recommended publications
  • Univa Grid Engine 8.0 Superior Workload Management for an Optimized Data Center
    Univa Grid Engine 8.0 Superior Workload Management for an Optimized Data Center Why Univa Grid Engine? Overview Univa Grid Engine 8.0 is our initial Grid Engine is an industry-leading distributed resource release of Grid Engine and is quality management (DRM) system used by hundreds of companies controlled and fully supported by worldwide to build large compute cluster infrastructures for Univa. This release enhances the Grid processing massive volumes of workload. A highly scalable and Engine open source core platform with reliable DRM system, Grid Engine enables companies to produce higher-quality products, reduce advanced features and integrations time to market, and streamline and simplify the computing environment. into enterprise-ready systems as well as cloud management solutions that Univa Grid Engine 8.0 is our initial release of Grid Engine and is quality controlled and fully allow organizations to implement the supported by Univa. This release enhances the Grid Engine open source core platform with most scalable and high performing advanced features and integrations into enterprise-ready systems as well as cloud management distributed computing solution on solutions that allow organizations to implement the most scalable and high performing distributed the market. Univa Grid Engine 8.0 computing solution on the market. marks the start for a product evolution Protect Your Investment delivering critical functionality such Univa Grid Engine 8.0 makes it easy to create a as improved 3rd party application cluster of thousands of machines by harnessing the See why customers are integration, advanced capabilities combined computing power of desktops, servers and choosing Univa Grid Engine for license and policy management clouds in a simple, easy-to-administer environment.
    [Show full text]
  • Design of an Accounting and Metric-Based Cloud-Shifting
    Design of an Accounting and Metric-based Cloud-shifting and Cloud-seeding framework for Federated Clouds and Bare-metal Environments Gregor von Laszewski1*, Hyungro Lee1, Javier Diaz1, Fugang Wang1, Koji Tanaka1, Shubhada Karavinkoppa1, Geoffrey C. Fox1, Tom Furlani2 1Indiana University 2Center for Computational Research 2719 East 10th Street University at Buffalo Bloomington, IN 47408. U.S.A. 701 Ellicott St [email protected] Buffalo, New York 14203 ABSTRACT and its predecessor meta-computing elevated this goal by not We present the design of a dynamic provisioning system that is only introducing the utilization of multiple queues accessible to able to manage the resources of a federated cloud environment the users, but by establishing virtual organizations that share by focusing on their utilization. With our framework, it is not resources among the organizational users. This includes storage only possible to allocate resources at a particular time to a and compute resources and exposes the functionality that users specific Infrastructure as a Service framework, but also to utilize need as services. Recently, it has been identified that these them as part of a typical HPC environment controlled by batch models are too restrictive, as many researchers and groups tend queuing systems. Through this interplay between virtualized and to develop and deploy their own software stacks on non-virtualized resources, we provide a flexible resource computational resources to build the specific environment management framework that can be adapted based on users' required for their experiments. Cloud computing provides here a demands. The need for such a framework is motivated by real good solution as it introduces a level of abstraction that lets the user data gathered during our operation of FutureGrid (FG).
    [Show full text]
  • Bright Cluster Manager 9.0 User Manual
    Bright Cluster Manager 9.0 User Manual Revision: 406b1b9 Date: Fri Oct 1 2021 ©2020 Bright Computing, Inc. All Rights Reserved. This manual or parts thereof may not be reproduced in any form unless permitted by contract or by written permission of Bright Computing, Inc. Trademarks Linux is a registered trademark of Linus Torvalds. PathScale is a registered trademark of Cray, Inc. Red Hat and all Red Hat-based trademarks are trademarks or registered trademarks of Red Hat, Inc. SUSE is a registered trademark of Novell, Inc. PGI is a registered trademark of NVIDIA Corporation. FLEXlm is a registered trademark of Flexera Software, Inc. PBS Professional, PBS Pro, and Green Provisioning are trademarks of Altair Engineering, Inc. All other trademarks are the property of their respective owners. Rights and Restrictions All statements, specifications, recommendations, and technical information contained herein are current or planned as of the date of publication of this document. They are reliable as of the time of this writing and are presented without warranty of any kind, expressed or implied. Bright Computing, Inc. shall not be liable for technical or editorial errors or omissions which may occur in this document. Bright Computing, Inc. shall not be liable for any damages resulting from the use of this document. Limitation of Liability and Damages Pertaining to Bright Computing, Inc. The Bright Cluster Manager product principally consists of free software that is licensed by the Linux authors free of charge. Bright Computing, Inc. shall have no liability nor will Bright Computing, Inc. provide any warranty for the Bright Cluster Manager to the extent that is permitted by law.
    [Show full text]
  • Study of Scheduling Optimization Through the Batch Job Logs Analysis 윤 준 원1 · 송 의 성2* 1한국과학기술정보연구원 슈퍼컴퓨팅본부 2부산교육대학교 컴퓨터교육과
    디지털콘텐츠학회논문지 Journal of Digital Contents Society JDCS Vol. 18, No. 7, pp. 1411-1418, Nov. 2017 배치 작업 로그 분석을 통한 스케줄링 최적화 연구 Study of Scheduling Optimization through the Batch Job Logs Analysis 1 2* 윤 준 원 · 송 의 성 1한국과학기술정보연구원 슈퍼컴퓨팅본부 2부산교육대학교 컴퓨터교육과 JunWeon Yoon1 · Ui-Sung Song2* 1Department of Supercomputing Center, KISTI, Daejeon 34141, Korea 2*Department of Computer Education, Busan National University of Education, Busan 47503, Korea [요 약] 배치 작업 스케줄러는 클러스터 환경에서 구성된 계산 자원을 인지하고 순서에 맞게 효율적으로 작업을 배치하는 역할을 수행 한다. 클러스터내의 한정된 가용자원을 효율적으로 사용하기 위해서는 사용자 작업의 특성을 분석하여 반영하여야 하는데 이를 위해서는 다양한 스케줄링 알고리즘을 파악하고 해당 시스템 환경에 맞게 적용하는 것이 중요하다. 대부분의 스케줄러 소프트웨 어는 전체 관리 대상의 자원 명세와 시스템의 상태뿐만 아니라 작업 제출부터 종료까지 다양한 사용자의 작업 수행 환경을 반영하 게 된다. 또한 작업 수행과 관련한 다양한 정보 가령, 작업 스크립트, 환경변수, 라이브러리, 작업의 대기, 시작, 종료 시간 등을 저 장하게 된다. 본 연구에서는 배치 스케줄러를 통한 작업 수행과 관련된 정보를 통해 사용자의 작업 성공률, 수행시간, 자원 규모 등의 스케줄러의 수행 로그를 분석하여 문제점을 파악하였다. 향후 이 연구를 바탕으로 자원의 활용률을 높임으로써 시스템을 최 적화할 수 있다. [Abstract] The batch job scheduler recognizes the computational resources configured in the cluster environment and plays a role of efficiently arranging the jobs in order. In order to efficiently use the limited available resources in the cluster, it is important to analyze and characterize the characteristics of user tasks. To do this, it is important to identify various scheduling algorithms and apply them to the system environment.
    [Show full text]
  • Workload Schedulers - Genesis, Algorithms and Comparisons
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE International Journal of Computer Science and Software Engineering (IJCSSE), Volume 4, Issue 6, June 2015 provided by WestminsterResearch ISSN (Online): 2409-4285 www.IJCSSE.org Page: 141-155 Workload Schedulers - Genesis, Algorithms and Comparisons Leszek Sliwko1 and Vladimir Getov2 1, 2 Faculty of Science and Technology, University of Westminster, 115 New Cavendish Street, United Kingdom [email protected], [email protected] ABSTRACT especially for online request scheduling [60]. However, In this article we provide brief descriptions of three classes of not taking into account additional factors such as current schedulers: Operating Systems Process Schedulers, Cluster server load, network infrastructure or storage availability Systems Jobs Schedulers and Big Data Schedulers. We describe may result in an inefficient utilization of available their evolution from early adoptions to modern implementations, machines and overall higher operational system costs. considering both the use and features of algorithms. In summary, More complex algorithms rely on the availability of static we discuss differences between all presented classes of schedulers and discuss their chronological development. In infrastructure data such as CPU speed, installed memory conclusion we highlight similarities in the focus of scheduling etc. As an example we find largest remaining processing strategies design, applicable to both local and distributed systems. time on fastest machine rule (LRPT-FM), where the job with the most remaining processing time is assigned to Keywords: Schedulers, Workload, Cluster, Cloud, Process, Big fastest machine [45]. LRPT_FM approach offers clear Data. advantages in heterogeneous environments, where the system is composed from machines with different configurations.
    [Show full text]
  • Materials Studio: Installation and Administration Guide
    INSTALLATION AND ADMINISTRATION GUIDE MATERIALS STUDIO 8.0 Copyright Notice ©2014 Dassault Systèmes. All rights reserved. 3DEXPERIENCE, the Compass icon and the 3DS logo, CATIA, SOLIDWORKS, ENOVIA, DELMIA, SIMULIA, GEOVIA, EXALEAD, 3D VIA, BIOVIA and NETVIBES are commercial trademarks or registered trademarks of Dassault Systèmes or its subsidiaries in the U.S. and/or other countries. All other trademarks are owned by their respective owners. Use of any Dassault Systèmes or its subsidiaries trademarks is subject to their express written approval. Acknowledgments and References To print photographs or files of computational results (figures and/or data) obtained using BIOVIA software, acknowledge the source in an appropriate format. For example: "Computational results obtained using software programs from Dassault Systèmes Biovia Corp.. The ab initio calculations were performed with the DMol3 program, and graphical displays generated with Materials Studio." BIOVIA may grant permission to republish or reprint its copyrighted materials. Requests should be submitted to BIOVIA Support, either through electronic mail to [email protected], or in writing to: BIOVIA Support 5005 Wateridge Vista Drive, San Diego, CA 92121 USA Contents Introduction to Materials Studio 1 Gateway administration 28 Materials Studio Updates 1 Server gateway information 28 Further Information 1 Configuring a gateway 30 System requirements and recommendations 2 Server scripts 31 Client system requirements 2 Port 31 Hardware minimum 2 Managing a gateway 31 Operating
    [Show full text]
  • HPC Job Sceduling Co-Scheduling in HPC Clusters
    HPC Job Sceduling Co-scheduling in HPC Clusters October 26, 2018 Nikolaos Triantafyllis PhD student School of Electrical and Computer Engineering - N.T.U.A. Categories of Schedulers The majority of Schedulers could be categorized in: • Operating Systems Process Schedulers • Cluster Systems Jobs Schedulers • Big Data Schedulers 1 Operating Systems Process Schedulers • During scheduling events, an algorithm has to assign CPU times to tasks • Focus on responsiveness and low overhead • Most notable process schedulers: ◦ Cooperative Scheduling (CS) ◦ Multi-Level Feedback Queue (MLFQ) ◦ O(n) Scheduler ◦ O(1) Scheduler ◦ Completely Fair Scheduler (CFS) ◦ Brain F Scheduler (BFS) • CFS: Each process should have equal share of CPU time. Current Linux kernels use CFS • BFS: Improves interactivity, but lowers performance. Proposed in 2009 2 Cluster Systems Jobs Schedulers • During scheduling events, an algorithm has to assign nodes to jobs • Focus on scalability and high throughput • Most notable jobs schedulers: ◦ Simple Linux Utility for Resource Management (SLURM) ◦ Maui Cluster Scheduler (Maui) ◦ Moab High-Performance Computing Suite (Moab) ◦ Univa Grid Engine (UGE) ◦ LoadLeveler (LL) ◦ Load Sharing Facility (LSF) ◦ Portable Batch System (PBS) [OpenPBS, TORQUE, PBS Pro] ◦ Globus toolkit ◦ GridWay ◦ HTCondor ◦ Mesos ◦ Open MPI ◦ TORQUE ◦ Borg and Omega 3 Big Data Schedulers • During scheduling events, an algorithm has to assign nodes to jobs • Jobs have storage and processing of large and complex data sets • Support specialized frameworks and to a very limited set of problems • Most notable process schedulers: ◦ Dryad ◦ MapRedure ◦ Hadoop ◦ HaLoop ◦ Spark ◦ CIEL “Big Data: New York Stock Exchange produces about 1 TB of new trade data per day.” 4 Cluster Structure Figure 1: Typical resource management system 5 Job Submission example in SLURM § ¤¨ 1 #!/bin/bash 2 # Example with 48 MPI tasks and 24 tasks per node.
    [Show full text]
  • A Simulated Grid Engine Environment for Large-Scale Supercomputers
    Ito et al. BMC Bioinformatics 2019, 20(Suppl 16):591 https://doi.org/s12859-019-3085-x SOFTWARE Open Access Virtual Grid Engine: a simulated grid engine environment for large-scale supercomputers Satoshi Ito1*, Masaaki Yadome1, Tatsuo Nishiki2, Shigeru Ishiduki2,HikaruInoue2, Rui Yamaguchi1 and Satoru Miyano1 From IEEE International Conference on Bioinformatics and Biomedicine (2018) Madrid, Spain, 3-6 December 2018 Abstract Background: Supercomputers have become indispensable infrastructures in science and industries. In particular, most state-of-the-art scientific results utilize massively parallel supercomputers ranked in TOP500. However, their use is still limited in the bioinformatics field due to the fundamental fact that the asynchronous parallel processing service of Grid Engine is not provided on them. To encourage the use of massively parallel supercomputers in bioinformatics, we developed middleware called Virtual Grid Engine, which enables software pipelines to automatically perform their tasks as MPI programs. Result: We conducted basic tests to check the time required to assign jobs to workers by VGE. The results showed that the overhead of the employed algorithm was 246 microseconds and our software can manage thousands of jobs smoothly on the K computer. We also tried a practical test in the bioinformatics field. This test included two tasks, the split and BWA alignment of input FASTQ data. 25,055 nodes (2,000,440 cores) were used for this calculation and accomplished it in three hours. Conclusion: We considered that there were four important requirements for this kind of software, non-privilege server program, multiple job handling, dependency control, and usability. We carefully designed and checked all requirements.
    [Show full text]
  • Avaliaç˜Ao Das Alteraç˜Oes Nos Tempos De Execuç˜Ao Em Aplicaç
    1 Avaliac¸ao˜ das alterac¸oes˜ nos tempos de execuc¸ao˜ em aplicac¸oes˜ paralelas executadas em cluster f´ısicos e virtuais Lucas Santana Carneiro, Autor. Angelo Amancioˆ Duarte, Orientador Resumo—Este artigo apresenta a avaliac¸ao˜ das alterac¸oes˜ para a construc¸ao˜ de clusters apareceu. Grandes empresas com dos tempos de execuc¸ao˜ de um benchmark paralelo em um servic¸os de Cloud Computing, como a Amazon [3], a IBM cluster quando este utiliza gerenciadores de trabalho, ou quando [4] e a Microsoft [5], passaram a oferecer soluc¸oes˜ de High o mesmo esta´ virtualizado em uma plataforma de nuvem, Performance Computing (HPC) sobre a plataforma da nuvem. buscando identificar os fatores que influenciam no desempenho As instanciasˆ de cluster EC2 (Elastic Compute Cloud) da das aplicac¸oes˜ em execuc¸ao,˜ quando os gestores sao˜ utilizados. Amazon possibilitam economia aos clientes, pois, o aluguel Para realizar as avaliac¸oes,˜ testes foram executados nos clusters utilizando o mesmo conjunto de computadores, mantendo dos recursos utilizados nos clusters e´ cobrado sob demanda as mesmas configurac¸oes˜ de hardware modificando apenas [6]. Outra vantagem oferecida por este tipo de servic¸o e´ que os gestores utilizados. Apos´ a realizac¸ao˜ dos testes os dados os clientes podem definir cargas de trabalho com recursos obtidos foram analisados, identificando os fatores envolvidos flex´ıveis [6]. Alem´ da reduc¸ao˜ dos custos e da flexibilizac¸ao˜ de nas alterac¸oes˜ dos tempos de execuc¸ao˜ entre as diferentes recursos ao utilizar as instanciasˆ de cluster EC2, os clientes configurac¸oes˜ testadas. tambem´ nao˜ precisam esperar para submeter seus trabalhos, como ocorre em clusters compartilhados, pois os recursos sao˜ Palavras-chave: Cluster virtual, Cloud computing, Gestor de disponibilizados ao serem alocados [6].
    [Show full text]
  • Scheduling David King, Sr
    Linux Clusters Institute: Scheduling David King, Sr. HPC Engineer National Center for Supercomputing Applications University of Illinois August 2017 1 About me • Worked in HPC since 2007 • Started at Purdue as a Student • Initially fixing nodes • Moved on to bigger projects • Sysadmin at Idaho National Laboratory for a year • Sr. HPC Engineer at Northwestern University for 5 years • Scheduling with Moab and Torque • Condo Cluster of 1000 nodes • Sr. HPC Engineer at NCSA for about a year • Scheduling on Blue Waters • 27k node system using Torque/Moab August 2017 2 Scheduling Schedule • Introduction • Break • Workload Management Overview • Slurm Deep Dive • What is the goal? • Components • Review of Fundamentals of Workload Managers • Interface • Queues • Limiting Resources • Priority • Overview • FIFO • Cgroups • Fairshare • Processor affinity • QOS • Containers • Reservations • Cloud and Scheduling • Feature Set Survey • Accounting • PBS Pro • Xdmod • Torque • Gold/Mam • Moab/Maui • Slurm • LSF • Lunch • Other Open Source Tools • Slurm Hands-on • Cloud and Scheduling August 2017 3 Workload Management Overview • What is the goal of workload management? • Efficient, effective use of all resources by the user community under normal use • Ability to accommodate unusual circumstances and special requests August 2017 4 Workload Management Overview • What it looks like in practice to the admin: August 2017 5 Workload Management • Workload management is software that “fairly” manages jobs running on an HPC system. • Most can apply many different
    [Show full text]
  • Digital Program Book
    Welcome To CUG 2016 I am delighted to welcome all of you to London, for this 59th edition of the Cray User Group conference. Though ECMWF is a European organisation rather than a British one, the UK has been our host nation since the creation of the Centre in 1975, and as such we do consider it to be home to ECMWF. As members of the Cray User Group for the past couple of years, we were delighted to be asked to host this edition, and choosing Scalability as a theme took us all of half a minute. Progress in numerical weather prediction is intimately connected with progress in super- computing. Over the years, more computing power has enabled us to increase the skill and detail of our forecasts. This has brought huge value to society, not least through early warn- ings of severe weather. But as the forecasting system becomes more complex, with current computing architectures it will soon be impossible to issue forecasts within schedule and at a reasonable cost. The Scalability Programme that ECMWF has been leading for a few years is about making numerical weather prediction coding more efficient. We strongly believe that it is one of the biggest challenge our community has ever had to face. What makes it such a perfect fit for this gathering is that, like the Cray User Group, Scalability is about col- laboration between users, computer scientists, software developers and hardware designers. It is about learning from each other, and about academia, operational centres and commer- cial sectors working together to ensure that technology and science can together achieve the best for society.
    [Show full text]
  • Grid Engine Users's Guide
    Univa Corporation Grid Engine Documentation Grid Engine Users’s Guide Author: Version: Univa Engineering 8.5.3 July 19, 2017 Copyright ©2012–2017 Univa Corporation. All rights reserved. Contents Contents 1 Overview of Basic User Tasks1 2 A Simple Workflow Example1 3 Displaying Univa Grid Engine Status Information5 3.1 Cluster Overview....................................5 3.2 Hosts and Queues...................................5 3.3 Requestable Resources.................................8 3.4 User Access Permissions and Affiliations....................... 10 4 Submitting Batch Jobs 13 4.1 What is a Batch Job?................................. 13 4.2 How to Submit a Batch Job.............................. 13 4.2.1 Example 1: A Simple Batch Job....................... 13 4.2.2 Example 2: An Advanced Batch Job..................... 14 4.2.3 Example 3: Another Advanced Batch Job.................. 14 4.2.4 Example 4: A Simple Binary Job....................... 15 4.3 Specifying Requirements................................ 15 4.3.1 Request Files.................................. 16 4.3.2 Requests in the Job Script........................... 17 5 Using Job Classes to Prepare Templates for Jobs 17 5.1 Examples Motivating the Use of Job Classes..................... 18 5.2 Defining Job Classes.................................. 19 5.2.1 Attributes describing a Job Class....................... 20 5.2.2 Example 1: Job Classes - Identity, Ownership, Access........... 22 5.2.3 Attributes to Form a Job Template...................... 22 5.2.4 Example 2: Job Classes - Job Template................... 25 5.2.5 Access Specifiers to Allow Deviation..................... 26 5.2.6 Example 3: Job Classes - Access Specifiers................. 28 5.2.7 Different Variants of the same Job Class................... 29 5.2.8 Example 4: Job Classes - Multiple Variants................. 30 5.2.9 Enforcing Cluster Wide Requests with the Template Job Class.....
    [Show full text]