Lighthouse Cheat Sheet Guide to General Linux (Bash) and Slurm Commands Head File Print First 10 Lines of File
Total Page：16
File Type：pdf, Size：1020Kb
Viewing and editing text files cat file Print entire content of file less file Similar to more, but with additional features Lighthouse Cheat Sheet Guide to general Linux (Bash) and Slurm commands head file Print first 10 lines of file tail file Print last 10 lines of file Accessing Lighthouse nano Simple, easy to use text editor Logging in from a terminal (Duo required) vim Minimalist yet powerful text editor ssh [email protected] emacs Extensible and customizable text editor Transferring files between Lighthouse and your system scp source [email protected]:target Advanced file management scp -r source [email protected]:target scp [email protected]:source target chmod Change read/write/execute permissions which cmd List the full file path of a command GUI Clients whereis cmd List all related file paths (binary, source, PuTTY SSH client for Windows manual, etc.) of a command WinSCP SCP client for Windows du dir List size of directory and its subdirectories FileZilla FTP client for Windows, Mac, and Linux find Find file in a directory Basic Linux file management Aliases and system variables man command Display the manual page for command alias Create shortcut to command pwd Print out the present working directory env Lists all environment variables ls List the files in the current directory export var=val Create environment variable $var with value ls -lh Show long, human-readable listing val ls dir List files inside directory dir echo $var Print the value of variable $var rm file Delete file .bashrc File that defines user aliases and variables mkdir dir Create empty directory called dir Input and output redirection rmdir dir Remove empty directory dir $(command) Runs command first, then inserts output to the rm -r dir Remove directory dir and all contents rest of the overall command cd dir Change working directory to dir < Standard input redirection cd .. Change working directory to parent > Standard output redirection cd Change working directory to home 2> Standard error redirection ls List the files in the current directory 2>&1 Standard error to standard output redirection cp file1 file2 Copy file1 as file2 cmd1 | cmd2 Pipe the output of cmd1 to cmd2 cp file1 dir Copy file1 into directory dir Filters mv file1 file2 Rename file1 as file2 Word, line, and character count mv file1 dir Move file1 into directory dir wc Find and print text matching a regular ~ (tilde) Home directory grep expression . (period) Current (working) directory sort Sort input .. (2 periods) Parent directory uniq Filter duplicate lines wget URL Download a file from Internet URL cut Cut specific fields or columns unzip file.zip Extract a ZIP file sed Stream editor for search and replace tar xzf file Extract a gzip compressed tarball (common Extensive tool for complex filtering tasks extensions, .tar.gz and .tgz) awk arc-ts.umich.edu/lighthouse Revised 1/2020 Lighthouse directories #SBATCH directives and #PBS counterparts /home/uniqname For use with running jobs, 80 GB quota #SBATCH #PBS Description --job-name=name -N name Job name /tmp Small file reads/writes, deleted after 10 days --account=name -A name Account to charge /scratch Large file reads/writes, purged periodically --partition=name -q name Submit to partition /afs Only on login node, 10 GB backed up (different for each PI) --time=dd-hh:mm:ss -l walltime=time Time limit (walltime) Lmod --nodes=count -l nodes=count Number of nodes module keyword string Search for module names or descriptions matching string --tasks-per-node=count -l ppn=count Processes per node module spider string Search for modules matching string --cpus-per-task=count n/a CPU cores per process module avail Show modules that can be loaded --mem=count -l mem=count RAM per node (e.g. now 1000M, 1G) module load module Load module in the environment --mem-per-cpu=count -l pmem=count RAM per CPU core module show module Show the help and variables set by --gres=gpu:count -l gpus=count GPUs per node module --nodelist=nodes -l nodes=nodes Request nodes module list List currently loaded modules --array=arrayspec -t arrayspec Define job array module unload module Remove module from environment --output=%x-%j.log -o filepath Standard output in run directory, formatted: module purge Remove all modules from jobName-jobID.log environment --error=%x-%j-E.log -e filepath Standard error log module save collection Save all currently loaded modules to collection --export=ALL -V Copy environment --export=var=val -v var=val Copy env variable module savelist Return all saved module collections --depend=var:jobid -W Job dependency states module describe Return all modules in collection depend=var:jobid (var): after, afterok, collection afterany, afternotok module restore Restore all modules from collection --mail-user=email -M email Email for job alerts collection --mail-type=type -m type Email alert types: BEGIN, END, NONE, FAIL, Slurm REQUEUE sbatch filename Submit a job script filename --exclude=nodes n/a Nodes to avoid squeue -u uniqname Show job queue for user uniqname scancel jobid Delete job jobid ARC-TS custom commands scontrol hold jobid Hold job jobid my_usage Usage in CPU minutes scontrol release jobid Release job jobid my_accounts Show account membership and resource limits sinfo Cluster status home-quota Show user disk quota and usage srun Launch parallel job step maxwalltime Show walltime available for jobs sacct Display job accounting information (including upcoming maintenance) Slurm Environment Variables ARC-TS Documentation & Support SLURM_JOBID Job ID Lighthouse User Guide: https://arc-ts.umich.edu/lighthouse/user-guide SLURM_SUBMIT_DIR Job submission directory Email [email protected] for further Lighthouse support SLURM_SUBMIT_HOST Host from which job was submitted Sensitive data should not be stored or processed on Lighthouse SLURM_JOB_NODELIST Node names allocated to job SLURM_ARRAY_TASK_ID Task ID within job array SLURM_JOB_PARTITION Job partition arc-ts.umich.edu/lighthouse Revised 1/2020 .