H2-for-the-Arts Documentation Release 2020 Raffaella D’Auria Apr 22, 2020 Using the Hoffman2 cluster: 1 Connecting/Logging in 3 1.1 Connecting via a terminal........................................3 1.2 Connecting via NX clients........................................6 2 Unix command line 101 11 2.1 Navigation................................................ 11 2.2 Environmental variables......................................... 13 2.3 Working with files............................................ 14 2.4 Miscellaneous commands........................................ 15 3 Data transfer 17 3.1 Data transfer nodes............................................ 17 3.2 Tools................................................... 17 3.3 Cloud storage services.......................................... 18 3.4 Globus.................................................. 19 3.5 rclone................................................... 22 3.6 scp.................................................... 35 3.7 sftp.................................................... 36 3.8 rsync................................................... 37 4 Rendering 39 4.1 Getting an interactive-session...................................... 39 4.2 Submitting batch jobs.......................................... 44 4.3 GPU-access................................................ 49 5 Indices and tables 51 i ii H2-for-the-Arts Documentation, Release 2020 This page will guide you on how to get started on H2. How-to use this documentation Please use the table of contents contained in the menu to navigate to the needed page, or use the search box, also in the page menu, to look up a topic in the documentation by keywords. About the Hoffman2 Cluster The Hoffman2 Cluster is a project of the Institute for Digital Research and Education (IDRE). It opened to users on January 28, 2008. The Hoffman2 Cluster is managed and operated by the IDRE Research Technology Group under the direction of Lisa Snyder. Using the Hoffman2 cluster: 1 H2-for-the-Arts Documentation, Release 2020 2 Using the Hoffman2 cluster: CHAPTER 1 Connecting/Logging in 1.1 Connecting via a terminal In order to be able to connect to the Hoffman2 Cluster and open graphical user interfaces such as Maya you need to have an SSH-client and an X Window System installed on your computer. Step by step instructions on how to acquire and use these tools are given below for Linux, Mac and Windows operating systems. All the connections to the cluster are based on the SSH secure protocol that requires authentication. Depending on the operating system that your computer is running, you have several choices for which command line application to use when connecting with the cluster. 1.1.1 Connectiong from Linux An X Window System server is generally available and running with the default installation (or can be readily installed via the OS package manager). If you are connecting from a Linux box, you can use the standard SSH-client generally installed with the OS and available via any terminal application. Open a terminal window and type the following: ssh-X login_id @hoffman2.idre.ucla.edu where login_id is replaced by your cluster user name. 1.1.2 Connecting from a MacOS On Mac OS X, the X windows system is called XQuartz. Mac OS X 10.5 10.6 and 10.7 installed it by default, but as of 10.8 Apple has dropped dedicated support and directs users to the open source XQuartz page. You can install XQuartz from the OS distribution media or download it from XQuartz page. MacOS, the operating system running on a Mac computer, comes equipped with a fully functional Termainal ap- plication and an SSH-client. The Terminal application can be located using the Spotlight Search or searching the Applications folder in Finder. 3 H2-for-the-Arts Documentation, Release 2020 Note: To enable indirect GLX and to allow remote visualization on the Hoffman2 Cluster, open the MacOS Termianl application and issue at its command prompt: defaults write org.macosforge.xquartz.X11 enable_iglx -bool true Warning: You will need to reboot your machine before being able to open GUI applications on Hoffman2. Note: You will need to perform the operation above only once To connect to the Hoffman2 Cluster, open the MacOS Terminal application and issue at its command prompt: ssh-Y login_id @hoffman2.idre.ucla.edu where login_id is replaced by your cluster user name. 1.1.3 Connecting from Windows To connect to the Hoffman2 Cluster from your Windows computer Install one of any of the following free ssh client programs. Note: Only one of the following options is needed to connect to the cluster via SSH. • MobaXterm a free X server for Windows with SSH terminal, SFTP and more. • XMing and Xming fonts provides a X Window System Server for Microsoft to be used in conjunction with GitBash an emulation shell for Windows. • Cygwin a free Linux-like environment for Windows. To add Cygwin/X server, select the xinit package from the X11 category. • Xshell a commercial option. Instructions are given for the first two of the options described above. Instructions for MobaXterm Download and install the MobaXterm Home Edition, either the Portable or Installer edition will work. To log into the Hoffman2 Cluster open MobaXterm and click on the Session button (the Sessions button is circled in green in MobaXterm application with the Sessions button circled in green.). A new window will pop-up to allow you to choose a type of session. Choose SSH by clicking on the SSH button (upper left corner) and fill the Remote host field with the Hoffman2 Cluster address, that is: hoffman2.idre.ucla.edu save the session by pressing the OK button. You can also create a new session type which you will use to transfer files back and from the cluster to your local machine, by selecting the Sessions button again and choosing SFTP as the session type and fill the Remote host field with the Hoffman2 Cluster data transfer node address, that is: 4 Chapter 1. Connecting/Logging in H2-for-the-Arts Documentation, Release 2020 Fig. 1: MobaXterm application with the Sessions button circled in green. Fig. 2: MobaXterm application with the SSH session set for Hoffman2. 1.1. Connecting via a terminal 5 H2-for-the-Arts Documentation, Release 2020 dtn1.hoffman2.idre.ucla.edu save the session by pressing the OK button. To access the cluster double click on the save SSH session. Instructions for XMing and GitBash Download and install XMing and Xming fonts following the installer instructions. Download and install GitBash following the installer instructions. Start Xming from the programs, open GibBAsh from the Start Menu, at its command promopt issue: export DISPLAY-localhost:0 Connect to the Hoffman2 Cluster by typing: ssh-XY login_id @hoffman2.idre.ucla.edu where login_id is replaced by your cluster user name. 1.2 Connecting via NX clients NX is a free, secure, compressed protocol for remote X Window System connections for Windows, Linux, Mac OSX, and Solaris. We currently support connecting to the Hoffman2 cluster via the NoMachine client as well as the X2Go client. 1.2.1 NoMachine client Download NoMachine Client for your operating system A free NoMachine client is available from NoMachine for Windows, Mac OSX, and most Linux distributions. • Download NoMachine from the website. • Use the download link for your operating system (Windows, Mac OSX, or Linux Distributions). Install • Once downloaded, double click on the downloaded package file. • Click the Run/Install button. • When the NoMachine Client Setup Wizard starts, accept all the setup defaults. Configure • Open the NoMachine client and continue to the connection page. • Click on the New link to create a connection to the Hoffman2 Cluster. • Specify the protocol as SSH and press Continue. • Provide the host name (nx.hoffman2.idre.ucla.edu). 6 Chapter 1. Connecting/Logging in H2-for-the-Arts Documentation, Release 2020 • Leave the port field set to 22 and press the Continue button. • Select Use the system login and press the Continue button. • Select Password and press the Continue button. • Select Don’t use a proxy and press the Continue button. • Name your connection and press the Done button. Please notice that if you are planning to suspend and reconnect to an NX session, you will need to repeat the configu- ration steps described above for the hosts: nx1.hoffman2.idre.ucla.edu nx2.hoffman2.idre.ucla.edu Run • Open the NoMachine client and continue to the connection page. • Click on the saved connection that you previously configured. • Enter your username and password and your NX session with a virtual desktop on the cluster will start. How to get a terminal window After you have logged into the Hoffman2 remote desktop, click on the Applications menu. Then select System Tools and Terminal. Note: To pin the terminal to the panel, right click on the Terminal menu item and select Add this launcher to panel. How to terminate your NX session From the gnome desktop, select: System -> Log Out login_id . A small window will open. Click the Log Out button. How to suspend an NX session Make note of the actual NX node where your session is running (typically NX1 or NX2) to allow for future reconnec- tions. To terminate the NX client on your computer, close the application or close the virtual desktop window. How to reconnect to a suspended NX session If you are planning to reconnect to a session, make note of the actual NX node where your session is running (typically NX1 or NX2). Make sure to configure your NX client for the NX node where your session is running. You will need to create a configuration using either nx1.hoffman2.idre.ucla.edu or nx2.hoffman2.idre.ucla.edu. See the Configuration section for more detailed instructions. 1.2. Connecting via NX clients 7 H2-for-the-Arts Documentation, Release 2020 Note: Reconnecting to a suspended NX session is not always possible. Lengthy computations should be run as batch jobs. We recommend terminating NX sessions (logout) instead of suspending them. On fast networks, a preferred alternative to NX is to run an X server on your platform (e.g., X11 packages for linux platforms, XQuartz for mac OSX, XMing and Xming fonts for Windows), and connect to the cluster via a terminal (or via PuTTY on Windows platforms) by setting up X11 forwarding.