Altair PBS Professional 2020.1.1 User's Guide You are reading the Altair PBS Professional 2020.1.1 User’s Guide (UG) Updated 9/30/20 Copyright © 2003-2020 Altair Engineering, Inc. All rights reserved. ALTAIR ENGINEERING INC. Proprietary and Confidential. Contains Trade Secret Information. Not for use or disclo- sure outside of Licensee’s organization. The software and information contained herein may only be used internally and are provided on a non-exclusive, non-transferable basis. Licensee may not sublicense, sell, lend, assign, rent, distribute, publicly display or publicly perform the software or other information provided herein, nor is Licensee permitted to decompile, reverse engineer, or disassemble the software. Usage of the software and other information provided by Altair (or its resellers) is only as explicitly stated in the applicable end user license agreement between Altair and Licensee. In the absence of such agreement, the Altair standard end user license agreement terms shall govern. Use of Altair’s trademarks, including but not limited to “PBS™”, “PBS Professional®”, and “PBS Pro™”, “PBS Works™”, “PBS Control™”, “PBS Access™”, “PBS Analytics™”, “PBScloud.io™”, and Altair’s logos is subject to Altair's trademark licensing policies. For additional information, please contact [email protected] and use the wording “PBS Trademarks” in the subject line. For a copy of the end user license agreement(s), log in to https://secure.altair.com/UserArea/agreement.html or contact the Altair Legal Department. For information on the terms and conditions governing third party codes included in the Altair Software, please see the Release Notes. This document is proprietary information of Altair Engineering, Inc. Contact Us For the most recent information, go to the PBS Works website, www.pbsworks.com, select "My PBS", and log in with your site ID and password. Altair Altair Engineering, Inc., 1820 E. Big Beaver Road, Troy, MI 48083-2031 USA www.pbsworks.com Sales [email protected] 248.614.2400 Please send any questions or suggestions for improvements to [email protected]. Technical Support Need technical support? We are available from 8am to 5pm local times: Location Telephone e-mail Australia +1 800 174 396 [email protected] China +86 (0)21 6117 1666 [email protected] France +33 (0)1 4133 0992 [email protected] Germany +49 (0)7031 6208 22 [email protected] India +91 80 66 29 4500 [email protected] +1 800 208 9234 (Toll Free) Italy +39 800 905595 [email protected] Japan +81 3 6225 5821 [email protected] Korea +82 70 4050 9200 [email protected] Malaysia +91 80 66 29 4500 [email protected] +1 800 425 0234 (Toll Free) North America +1 248 614 2425 [email protected] Russia +49 7031 6208 22 [email protected] Scandinavia +46 (0)46 460 2828 [email protected] Singapore +91 80 66 29 4500 [email protected] +1 800 425 0234 (Toll Free) South Africa +27 21 831 1500 [email protected] South America +55 11 3884 0414 [email protected] UK +44 (0)1926 468 600 [email protected] Contents About PBS Documentation ix 1 Getting Started with PBS 1 1.1 Why Use PBS? . 1 1.2 PBS Tasks and Components . 1 1.3 Interfaces to PBS . 3 1.4 Setting Up Your Environment . 5 2 Submitting a PBS Job 11 2.1 Introduction to the PBS Job. 11 2.2 The PBS Job Script. 14 2.3 Submitting a PBS Job . 18 2.4 Job Submission Recommendations and Advice . 23 2.5 Job Submission Options . 24 2.6 Job Submission Caveats. 30 3 Job Input & Output Files 31 3.1 Introduction to Job File I/O in PBS . 31 3.2 Input/Output File Staging. 31 3.3 Managing Output and Error Files . 39 4 Allocating Resources & Placing Jobs 49 4.1 What is a Vnode? . 49 4.2 PBS Resources. 49 4.3 Requesting Resources . 51 4.4 How Resources are Allocated to Jobs . 59 4.5 Limits on Resource Usage . 61 4.6 Viewing Resources . 63 4.7 Specifying Job Placement. 64 4.8 Backward Compatibility. 70 5 Multiprocessor Jobs 77 5.1 Submitting Multiprocessor Jobs . 77 5.2 Using MPI with PBS . 81 5.3 Using PVM with PBS. 102 5.4 Using OpenMP with PBS . 103 5.5 Hybrid MPI-OpenMP Jobs. 105 PBS Professional 2020.1.1 User’s Guide UG-v Contents 6 Controlling How Your Job Runs 107 6.1 Using Job Exit Status . 107 6.2 Using Job Dependencies . 107 6.3 Adjusting Job Running Time . 110 6.4 Using Checkpointing . 113 6.5 Holding and Releasing Jobs . 115 6.6 Allowing Your Job to be Re-run. 118 6.7 Controlling Number of Times Job is Re-run . 119 6.8 Deferring Execution. 119 6.9 Setting Priority for Your Job . 120 6.10 Making qsub Wait Until Job Ends. 120 6.11 Running Your Job Interactively . 121 6.12 Using Environment Variables . 126 6.13 Specifying Which Jobs to Preempt . 127 6.14 Releasing Unneeded Vnodes from Your Job . 128 6.15 Running Your Job in a Container . 130 6.16 Running Your Job in the Cloud . 132 6.17 Allowing Your Job to Tolerate Vnode Failures . 132 7 Reserving Resources 135 7.1 Glossary . 135 7.2 Quick Explanation of Reservations for Jobs . 136 7.3 Prerequisites for Reserving Resources. 136 7.4 Advance and Standing Reservations . 136 7.5 Job-specific Reservations . 140 7.6 Getting Confirmation of a Reservation . 142 7.7 Modifying Reservations. 142 7.8 Deleting Reservations . 142 7.9 Viewing the Status of a Reservation . 143 7.10 Submitting a Job to a Reservation . 146 7.11 Reservation Caveats and Errors . ..