Unix Crash Course Subjects (after lunch) ● Networking and the X windowing system ● Shell scripting concepts ● Regular expressions ● sed and awk ● Miscelanious (editing and programming) Unix Crash Course Slide 1 Unix Crash Course Slide 4 $ who am i Mumbo Jumbo? ● Rob Wolfram ● Unix System administrator ● E-mail: [email protected] ● PGP Key: 0xF7A0F7A0 Unix Crash Course Slide 2 Unix Crash Course Slide 5 Subjects (before lunch) Unix History ● A short history of Unix and Linux ● AT&T abandoned MULTICS ● Structure and philosophy of Unix ● Ken Thompson & Dennis Ritchie developed UNICS on a PDP7 ● Files and filesystems ● UNIX ported to Ritchie's C language ● Shell variables and globbing ● BSD released, many ports followed ● Processes and jobs ● POSIX effort to unify Unices ● GNU project / Linux kernel Unix Crash Course Slide 3 Unix Crash Course Slide 6 Unix History Unix philosophy ● Everything is a file ● Combine small tools to build your requirement Ritchie and Thompson working on a PDP11/20 at Bell Labs Unix Crash Course Slide 7 Unix Crash Course Slide 10 Unix History Unix structure Unix has an onion-like structure, the hardware is handled by the kernel Unix Crash Course Slide 8 Unix Crash Course Slide 11 (Gnu/) Linux distros Kernel provisions The kernel provides: ● CPU scheduling of processes ● Accessing hardware ● System calls for user-land programs ● The filesystem ● Privilege separation etc. Unix Crash Course Slide 9 Unix Crash Course Slide 12 Multi-user environment File permissions ● Unix runs programs from multiple users concurrently, even interactive programs ● User identified by numeric id, user name provided for verbosity ● User is member of one or more groups, identified by group id ● Interactive sessions need a TTY for input and output Unix Crash Course Slide 13 Unix Crash Course Slide 16 The filesystem Lab 1 A single unified tree of multiple filesystems ● ls -l ● cat README ● chmod a+r README ● ls -l ● cat README Unix Crash Course Slide 14 Unix Crash Course Slide 17 Inodes The Shell ● There are various shells available: – sh: the Bourne shell – csh: the C shell – ksh: the Korn shell – tcsh: the TENEX C shell – bash: the Bourne-Again shell – zsh: the Z shell Unix Crash Course Slide 15 Unix Crash Course Slide 18 The Shell Special variables ● The shell interprets entered commands Syntax: command argument_list $PATH $0 $MANPATH $1 - $9 ● Arguments are separated by white space $LD_LIBRARY_PATH $# Certain arguments (usually single characters $HOME $* preceded by a dash change the program's $USER $@ $PWD $? behaviour and are called “options” $SHELL $$ ● The shell will perform command substitution, $PS1 $! variable expansion and globbing and execute $PS2 the command with modified command line. All these steps depend on quoting. Unix Crash Course Slide 19 Unix Crash Course Slide 22 Frequent commands Quoting Variables expand in double quotes, not in single man chmod wc quotes. Backslashes “escape” a single echo chown more read chgrp date character: ls umask time $ HW="Hello, World." cp cat tar $ echo "$HW" mv head gzip Hello, World. ln tail compress $ echo '$HW' rm cut xargs $HW pwd grep tee $ echo \$HW cd sed expr $HW mkdir sort awk $ echo \\$HW rmdir uniq find \Hello, World. $ Unix Crash Course Slide 20 Unix Crash Course Slide 23 Shell variables Command substitution Using shell variables: Use “backquotes” for command substitution: $ echo $HW $ wc log.`date +%Y%m%d` $ HW="Hello, World." 1 8 54 log.20131102 $ echo $HW $ Hello, World. $ On modern shells, $( ) is allowed. This enables nesting: Export variables to make them visible in a sub- bash$ wc log.$(expr $(date +%Y%m%d) – 100) shell: 630 1616 40744 log.20131002 $ export VARIABLE bash$ Use curly braces to disambiguate variables: $ echo ${VARIABLE}more_text Unix Crash Course Slide 21 Unix Crash Course Slide 24 Globbing Redirection ● The asterisk (*) expands to zero or more ● Three file descriptors: characters (e.g. “ls foo*”) – FD0 is standard input (stdin) ● The question mark expands to exactly one – FD1 is standard output (stdout) character (e.g. “ls /etc/?asswd”) – FD2 is standard error (stderr) ● Characters in square brackets expand to one ● More are available for advanced use character from the list. Ranges are allowed. ● Redirect output with and input with (e.g. (“ ”). Negate the list with an > < ls foo.[abc0-9] “ ” or “ ” (FD is exclamation mark (“ ”). command 1> file command 0< file ls foo.[!abc0-9] optional for stdin and stdout) Unix Crash Course Slide 25 Unix Crash Course Slide 28 Globbing Redirection ● On modern Unices, the tilde (~) expands to the ● Connect file descriptors with >& construct: users homedirectory and “~foo” to the (command > file 2> &1) homedirectory of user “foo” ● > overwrites the output file or create a new ● Globbing is handled by the shell, the executed one, command doesn't know if globbing occurred >> will append to the file instead – Notice that this can cause an error of an oversized ● << is called a “here document” and used in argument list scripts. Unix Crash Course Slide 26 Unix Crash Course Slide 29 Lab 2 Pipes ● A pipe connects stdout of a command to stdin of the next ● File & directory management This is central to the Unix philosophy, i.e. ● Variable substitution create small but powerful tools and connect ● Command substitution them Example: ● Globbing ls /tmp | wc -l ● stderr can't be piped alone, only with stdout Unix Crash Course Slide 27 Unix Crash Course Slide 30 tee and xargs Forking ● Two commands used a lot with pipes: ● The shell will start a “child process”. The tee and xargs. Examples: command will be executed in this process. ● Save log output and count entries: ● After the child exits, it signals the “parent”. grep 10.1.2.3 /var/log/apache/access.log \ ● | tee /tmp/rogueclient.txt | wc -l Changed environment in child is not visible in the parent (variables, current directory) ● Search for text in files that are less than 4 days old: ● Parent variables should be “exported” to be find /var/log -mtime -4 -print | xargs \ visible in the child (export VARNAME) grep -l 'kernel error' ● Executing a shell in current process is called “sourcing” (. scriptfile) Unix Crash Course Slide 31 Unix Crash Course Slide 34 Grouping Processes Combine stdout of multiple commands with () or ● Every process has a “process id” {}. Parentheses work in sub-shell, braces in ● Use ps to retrieve information of processes current shell: ● Use kill to send processes a signal $ echo Foo ; echo Bar | wc Some signals (e.g. SIGINT) are handled by the Foo program, others (e.g. SIGKILL) are handled by 1 1 4 the kernel $ { echo Foo ; echo Bar ;} | wc 2 2 8 $ (echo Foo ; echo Bar) | wc 2 2 8 $ Unix Crash Course Slide 32 Unix Crash Course Slide 35 Lab 3 Jobs ● A process can be started in the background by appending an ampersand (&) to the CL ● Redirection and piping ● A program is suspended by sending it a SIGSTOP (e.g. by pressing CTRL-Z) ● jobs gives a list of all suspended and backgrounded processes ● fg and bg continue running a process in the foreground or background respectively (job id may be appended with % sign Unix Crash Course Slide 33 Unix Crash Course Slide 36 Scheduling Networking ● Run a program unattended later with at: ● Interactive shells and file sharing can be echo "find /tmp -mtime +30 | xargs rm -f"\ started from networked hosts | at 20:08 tomorrow ● Common tools: ● Schedule regularly with cron. Syntax: min hou dom mon dow command [arguments] – telnet Example: – ftp 5 * * 3,6 2 echo foo >> /tmp/myfile – “rsh” tools (rsh, rlogin, rcp) – secure shell suite (ssh, scp, sftp) ● Pseudo-TTY's are assigned to interactive networked shells Unix Crash Course Slide 37 Unix Crash Course Slide 40 Shell Initialization The X Windowing System ● The shell will source files on login or other ● The standard Unix GUI (X) is networked startup. based. Consists of an “X server” (which can – sh, ksh: /etc/profile, $HOME/.profile (on display graphics and handle keyboard and login) mouse) and an “X client” (a program requesting graphical output. – bash: /etc/profile, $HOME/.bash_profile, $HOME/.profile (on login) ● The X server is identified by the $DISPLAY /etc/bash.bashrc, $HOME/.bashrc (interactive) variable (e.g. myscreen.example.com:0.0) Unix Crash Course Slide 38 Unix Crash Course Slide 41 Lunch The X Windowing System See you in half an hour Unix Crash Course Slide 39 Unix Crash Course Slide 42 X server access Shell flow: case ● Host based access: (dis)allow all users access ● Syntax: to the X server. Syntax: xhost +|- [hostname] case string in valuelist1) ● Cookie based access. List cookie on X server command list and add it to the .Xauthority file from the user ;; running the X client. xauth is used for cookie valuelist2) command list management ;; ● ssh can automate the xauth process and pass ... valuelistn) X traffic via encrypted tunnel. command list ;; esac Unix Crash Course Slide 43 Unix Crash Course Slide 46 Shell scripting Shell flow: case ● A “shebang” is needed to tell the OS what ● Valuelists consist of one or more patterns to script language is used. Syntax: match against the string, separated by pipes #!/bin/sh (|) ● Functions are “named groupings” and are not ● Shell globbing syntax is allowed when executed at time of declaration. Syntax: matching the string shfunc() { commandlist ; } ● Only the first matching entry is executed ● Here document redirects stdin from the script: command << WORD first line of stdin last line of stdin WORD Unix Crash Course Slide 44 Unix Crash Course Slide 47 Shell flow: if Shell flow: while ● Syntax: ● Repeat a block of commands as long as the if command constraint is valid. Syntax: then while command command list do elif command command list then done command list or else until command command list do fi command list ● Alternative: done command1 && command2 || command3 ● Exit or restart the loop with break or continue Unix Crash Course Slide 45 Unix Crash Course Slide 48 Shell flow: for Lab 4 ● Repeat a loop a number of times while assigning a value to a variable.