Introduction to Linux/Unix Xiaoge Wang [email protected] How does this class works • We are going to cover some basics with hands on exampes. • Exercises are denoted by the following icon in this presents: • Examples will be showed and code will be provided. Please try to follow the demo. The history of demo will be provided at the end of lecture. Green and Red Sticky • Use the sDcky notes provided to help me help you. – No S%cky = I am working – Green = I am done and ready to move on (yea!) – Red = I am stuck and need more Dme and/or some help Agenda • IntroducDon • Linux/Shell Tour – Commands • Set 1: look around • Set 2: Do something • Summary – Pipeline • Standard input and output and redirecDon • Pipeline of commands – Scripts • Grammar(key words and special characters) • Scope of script and recursive • Variables, expression, expansion • Control flow • ExecuDon model (source vs. direct run, ) • Summary Agenda • Linux/Shell Tour – Commands • Set 1: look around • Set 2: Do something • Summary – Pipeline • Standard input and output and redirecDon • Pipeline of commands – Scripts • Grammar(key words and special characters) • Scope of script and recursive • Variables, expression, expansion • Control flow • ExecuDon model (source vs. direct run, ) • Summary Introduction • Get ready – Ticket? Equipment? • Big Map – Linux/Unix – Shell • Overview of trail – Commands – Shell script – Get ready for HPC and more. Exercise 1: ssh ssh [email protected] Connect to HPCC: go to any develop node. – Windows users MobaXterm – Mac users Terminal – Linux users? • Show ‘mod’ and ‘ssh dev-X’ Example of terminal • Mac Show screen MobaXterm Big Picture Shell Shell Big Picture • Shell – CLI ✔ – GUI OS Shell Example Overview of the Tour – Commands • Set 1: Look around • Set 2: Do something • Summary – Pipeline • Standard input and output and redirecDon • Pipeline of commands – Scripts • Grammar(key words and special characters) • Scope of script and recursive • Variables, expression, expansion • Control flow • ExecuDon model (source vs. direct run, ) Linux/Shell Tour Ready? GO! Agenda • IntroducDon – Pipeline • Standard Input and Output • RedirecDon >, >>, .& • Pipeline of commands – Scripts • Grammar(key words and special characters) • Scope of script and recursive • Variables, expression, expansion, special charactors ($, backDck, =, !, …) • Control flow • ExecuDon model (source vs. direct run, ) – Help and self-explore (man, --help, google, sites, examples, ) • Summary Set 1: Look Around • What are in your home directory? • What you can tell about an item in the directory? • Where are commands I can see? • What is the content of a file? • What sodware ready for use? • How many jobs are running? • Do I have jobs running? • How many jobs are waiDng in the queue? • Who are currently on this node? • Who are they? Who am I? • Why machine is so slow? Who is using it? • …... Demo 1 ls, pwd, cd, more, less, cat, head, tail, env, set, module list/spider, export, echo, ps, top, mod, grep, ….... • (1) starDng from home directory: – pwd: absolute path and relave path – ls: permission, ownership, others • (2) Look into file content – cat, more, less, head, tail, find • (3) Look into command repository (PATH) – echo $PATH – Module list/spider/show – Powertools – man, whas, which • (4) Look into what is running on the nodes – top, ps, mod, showq, qstat – who, whoami, finger, userinfo • (5) Look into users – who, whoami, finger, userinfo Exercise 2 • Find the number of files in your home directory • Find the CPU load and memory usage of the dev node. Set 2: Do Something • AddiDon – Bring in data/files – Create new files – Load something from HPCC • SubtracDon – Delete file or folder – Unload modules • Change/move things Demo 2 Touch, cp, mv, rm, mkdir, module load, scp, curl, wget, vi, nano, > and >>, chmod, export, • (1) Add something – a) create locally: mkdir, cp mv, vi/vim/nano >, >> – b) cp from PC: scp, (not cover disk drive mapping) – c) down load from Internet site (not involve the GUI) – d) by “getexample” , getexample, module load • (2) Remove something, – rm, mv, module purge, module unload • (3) Change something – chmod, mv (with careful), touch, export Exercise 2 • Copy example files to from /mnt/scratch/ class0/intro2linux.tar to you home directory. • Untar it to a directory by $ tar –xvf tarfilename • Or copy directory intro2linux to your home directory Summary • Command format – Command name – OpDons: strings, integers, filenames – Input/output: stdin, stdout, stderr, redirecon • Command locaon – PATH and how to set • Tricks – Tab (Key board short cut) – ! – Arrows Agenda • IntroducDon r – Commands • Set 1: look around • Set 2: Do something • Summary – Scripts • Grammar(key words and special characters) • Scope of script and recursive • Variables, expression, expansion • Control flow • ExecuDon model (source vs. direct run, ) • Summary I/O and Redirection • Standard input and output • >, >>, >&, < Demo 3 • Use input file to replace the stdin • Stdout to create a output Pipeline • Task need more commands to complete • One command’s output is used for next command’s input • Format: Commd1 |commd2 • Commonly used – grep, wc, sort, less, more, uniq, head, tail,… Examples of pipeline • Filtering output showq |grep userid • SorDng the output ls –l |sort • Counng ls |wc • Beer viewing ps |more Exercise 4 • Use pipeline to find out how many people are login. • Find out how many jobs in job queue. Among them, how many are running. Let’s Take a Break Agenda • IntroducDon – Commands • Set 1: look around • Set 2: Do something • Summary – Pipeline • Standard Input and Output and redirecDon • Pipeline of commands • Summary Shell Script Combine more commands together for more sophisDcate/complicated task. • Grammar • Environment and scope • Parameters, expressions, expansion, • Control flow (branch, loops) • ExecuDon (source vs. direct run, ) Environment and Scope • Variables • IniDalizaon: • /etc/profile • ~/.profile • ~/.bashrc • Local variable Demo 4 • Set iniDal environment such that module “powertools” is loaded in Exercise 5 Add ~/intro2linux to your PATH by add following line into .bashrc file: export PATH=~/intro2linux:$PATH To check the result, use echo $PATH To see if it is added into the PATH Control Flow • if • exit • for • while • unl • case • break • connue if • First form if condi&on ; then commands Fi • Second form if condi&on ; then commands else commands fi • Third form if condi&on ; then commands elif condi&on ; then commands fi What is a "condition"? • Test – First form test expression – Second form [ expression ] • Return value Expression examples • -d file True if file exists and is a directory. • -e file True if file exists (regardless of type). • -f file True if file exists and is a regular file. • file1 -nt file2 True if file1 exists and is newer than file2. • s1 = s2 True if the strings s1 and s2 are idenDcal. • s1 < s2 True if string s1 comes before s2 based on the ASCII value of their characters. • n1 -eq n2 True if the integers n1 and n2 are algebraically equal. • expression1 -a expression2 True if both expression1 and expression2 are true. NOTE: use “man test” to get more details of expression Example $ more tesDf $ ./tesDf file1 file2 Case– More branching • Form case word in paerns ) statements ;; esac • Example (in case_ex) #!/bin/bash echo -n "Type a digit or a leqer > " read character case $character in # Check for leqers [a-z] | [A-z] ) echo "You typed the leqer $character" ;; # Check for digits [0-9] ) echo "You typed the digit $character" ;; # Check for anything else * ) echo "You did not type a leqer or a digit" esac Loops: while, for, until • Example 1 (in while_ex): #!/bin/bash number=0 while [ $number –lt 5 ]; do echo "Number = $number” # do things here number=$((number + 1)) done • Example 2 (in unDl_ex): #!/bin/bash number=0 unl [ $number –ge 5 ]; do echo "Number = $number” # do things here number=$((number + 1)) done Loops: • for loop: for variable in words; do statements done • Example (in for_ex): #!/bin/bash for number in 1 2 3 4 5; do echo “Number = $((number-1))” # do things here # do not need this: number=$((number + 1)) done Quotes and Escaping There are three recognized types: • per-character escaping using a backslash: \ $stuff • weak quo%ng with double-quotes: "stuff" • strong quong with single-quotes: 'stuff' All three forms have the very same purpose: They give you general control over parsing, expansion and expansion results. NOTE: Bash contains ANSI C-like escape sequences Example: quotes and escaping In “quote_escape_ex” #/bin/bash # first $HOME is not expand, so as double quote echo \$HOME is set to \"$HOME\" # weak quote echo "My PATH is: $PATH" # strong quote echo 'My PATH is: $PATH’ Expansions and substitutions • Parameter expansion – $WORD, ${STUFF…} • Pathname expansion – *.txt, page_1?.html • ArithmeDc expansion – $(( EXPRESSION )) • Command subsDtuDon – $( COMMAND ), ` COMMAND ` • Tilde expansion – ~, ~+, ~- • Brace expansion – {X,Y,z}, {X..Y}, {X..Y..z} • Process subsDtuDon – <( COMMAND ), >( COMMAND ) Parameter Expansion • PosiDonal parameters • Special parameters • Named variables – System defined variables – Script defined variables Position Parameters • The arguments given to your scripts when it is invoked (command line). • It could be from $1 to $N. • It has total number of $# • $@ expands the list of all • Note: When N consists of more than a single digit, it must be enclosed in a braces like ${N}. Special Parameters variable Descripon $# The number of arguments supplied to a script. $* All the arguments are double quoted.