Viewing and editing text files

cat file Print entire content of file ​ ​ less file Similar to more, but with additional features Lighthouse Cheat Sheet ​ Guide to general Linux (Bash) and Slurm commands head file Print first 10 lines of file ​ ​ ​ ​ ​ ​ ​

tail file Print last 10 lines of file ​ ​ ​ Accessing Lighthouse nano Simple, easy to use text editor

Logging in from a terminal (Duo required) vim Minimalist yet powerful text editor ssh [email protected] ​ ​ emacs Extensible and customizable text editor

Transferring files between Lighthouse and your system scp source [email protected]:target ​ ​ ​ Advanced file management scp -r source [email protected]:target ​ ​ ​ scp [email protected]:source target chmod Change read/write/execute permissions ​ ​ ​ ​ ​

which cmd List the full file path of a command GUI Clients ​ whereis cmd List all related file paths (binary, source, PuTTY SSH client for Windows ​ manual, etc.) of a command WinSCP SCP client for Windows du dir List size of directory and its subdirectories ​ FileZilla FTP client for Windows, Mac, and Linux find Find file in a directory

Basic Linux file management Aliases and system variables man command Display the manual page for command ​ ​ alias Create shortcut to command pwd Print out the present working directory env Lists all environment variables ls List the files in the current directory export var=val Create environment variable $var with value ​ ​ ​ ​ ​ ls -lh Show long, human-readable listing val ls dir List files inside directory dir echo $var Print the value of variable $var ​ ​ ​ ​ rm file Delete file .bashrc File that defines user aliases and variables ​ ​

mkdir dir Create empty directory called dir ​ ​ Input and output redirection rmdir dir Remove empty directory dir ​ ​ $(command) Runs command first, then inserts output to the rm -r dir Remove directory dir and all contents ​ ​ ​ ​ ​ ​ ​ ​ rest of the overall command cd dir Change working directory to dir ​ ​ < Standard input redirection cd .. Change working directory to parent > Standard output redirection cd Change working directory to home 2> Standard error redirection ls List the files in the current directory 2>&1 Standard error to standard output redirection cp file1 file2 Copy file1 as file2 cmd1 | cmd2 Pipe the output of cmd1 to cmd2 ​ ​ ​ ​ ​ ​ ​ ​ ​ cp file1 dir Copy file1 into directory dir ​ ​ ​ ​ Filters mv file1 file2 Rename file1 as file2 ​ ​ ​ ​ Word, line, and character count mv file1 dir Move file1 into directory dir wc ​ ​ ​ ​ Find and print text matching a regular ~ (tilde) Home directory grep expression . (period) Current (working) directory sort Sort input .. (2 periods) Parent directory uniq Filter duplicate lines wget URL Download a file from Internet URL ​ ​ cut Cut specific fields or columns unzip file.zip Extract a ZIP file ​ ​ sed Stream editor for search and replace tar xzf file Extract a gzip compressed tarball (common ​ Extensive tool for complex filtering tasks extensions, .tar.gz and .tgz) awk ​ ​ ​ ​ arc-ts.umich.edu/lighthouse Revised 1/2020 ​ Lighthouse directories #SBATCH directives and #PBS counterparts

/home/uniqname For use with running jobs, 80 GB quota #SBATCH #PBS Description ​ --job-name=name -N name Job name /tmp Small file reads/writes, deleted after 10 days ​ ​ --account=name -A name Account to charge /scratch Large file reads/writes, purged periodically ​ ​ --partition=name -q name Submit to partition /afs Only on login node, 10 GB backed up ​ ​ (different for each PI)

--time=dd-hh:mm:ss -l walltime=time Time limit (walltime) Lmod ​ ​ --nodes=count -l nodes=count Number of nodes module keyword string Search for module names or ​ ​ ​ descriptions matching string --tasks-per-node=count -l ppn=count Processes per node ​ ​ ​ module spider string Search for modules matching string --cpus-per-task=count n/a CPU cores per process ​ ​ ​ module avail Show modules that can be loaded --mem=count -l mem=count RAM per node (e.g. ​ ​ now 1000M, 1G)

module load module Load module in the environment --mem-per-cpu=count -l pmem=count RAM per CPU core ​ ​ ​ ​ ​ module show module Show the help and variables set by --gres=gpu:count -l gpus=count GPUs per node ​ ​ ​ module --nodelist=nodes -l nodes=nodes Request nodes ​ ​ module list List currently loaded modules --array=arrayspec -t arrayspec Define job array ​ ​ module unload module Remove module from environment --output=%x-%j.log -o filepath Standard output in run ​ ​ ​ ​ ​ directory, formatted: module purge Remove all modules from jobName-jobID.log environment --error=%x-%j-E.log -e filepath Standard error log module save collection Save all currently loaded modules to ​ ​ ​ collection --export=ALL -V Copy environment

--export=var=val -v var=val Copy env variable module savelist Return all saved module collections ​ ​ ​ ​ ​ ​ --depend=var:jobid -W Job dependency states module describe Return all modules in collection ​ ​ ​ ​ depend=var:jobid (var): after, afterok, collection ​ ​ ​ ​ ​ afterany, afternotok module restore Restore all modules from collection ​ --mail-user=email -M email Email for job alerts collection ​ ​ --mail-type=type -m type Email alert types: BEGIN, ​ ​ END, NONE, FAIL, Slurm REQUEUE

sbatch filename Submit a job script filename --exclude=nodes n/a Nodes to avoid ​ ​ ​ squeue -u uniqname Show job queue for user uniqname ​ ​ scancel jobid Delete job jobid ARC-TS custom commands ​ ​ scontrol hold jobid Hold job jobid my_usage Usage in CPU minutes ​ ​ scontrol release jobid Release job jobid my_accounts Show account membership and ​ ​ resource limits sinfo Cluster status home-quota Show user disk quota and usage srun Launch parallel job step maxwalltime Show walltime available for jobs sacct Display job accounting information (including upcoming maintenance)

Slurm Environment Variables ARC-TS Documentation & Support

SLURM_JOBID Job ID Lighthouse User Guide: https://arc-ts.umich.edu/lighthouse/user-guide SLURM_SUBMIT_DIR Job submission directory Email [email protected] for further Lighthouse support SLURM_SUBMIT_HOST Host from which job was submitted ​ ​ ​ Sensitive data should not be stored or processed on Lighthouse SLURM_JOB_NODELIST Node names allocated to job ​ ​

SLURM_ARRAY_TASK_ID Task ID within job array SLURM_JOB_PARTITION Job partition

arc-ts.umich.edu/lighthouse Revised 1/2020 ​