Shells and Shell scripting What is a Shell? • A shell is a command line interpreter that is the interface between the user and the OS. • A “program launcher” of sorts. • The shell: o analyzes each command o determines what actions are to be performed o performs the actions • Example: wc –l file1 > file2 Which shell? • sh – Bourne shell o Most common, other shells are a superset o Good for programming • csh or tcsh – default for command line on CDF o C-like syntax o Best for interactive use. Not good for programming. • bash – default on Linux (Bourne again shell) o Based on sh, with some csh features. • korn – written by David Korn o Based on sh – Some claim best for programming. o Commercial product. Common shell facilities Shell startup When a shell is invoked, it does the following: 1. Read a special startup file (usually in home directory) 2. display prompt and wait for command 3. Ctrl-D on its own line terminates shell, otherwise, goto step 2. Shell startup files used to set shell options, set up environment variables, alias sh – executes .profile if it’s there. ksh – executes .profile if in interactive mode. Executes $ENV (usually $HOME/.kshrc) csh – executes .cshrc if it exists. If a login shell, executes .login bash – executes .bashrc, if a login shell, executes .bash_profile instead Executables vs. built-in commands Most commands you run are other compiled programs. Found in /bin Example: ls – shell locates ls binary in /bin directory and launches it Some are not compiled programs, but built into the shell: cd, echo Input-output redirection prog < infile > outfile ls > outfile 2>&1 # sh stdout and stderr Pipelining commands send the output from one command to the input of the next: ls -l | wc ps –aux | grep reid | sort Before a program is executed, the shell recognizes the special characters such as <, >, |, and rewires the standard input, output, or error file descriptors of the program about to be executed to point to the right files (or the standard input of another program). Once everything is setup, the shell executes the program. The program is blissfully ignorant of where its input is coming from. Job Control • A job is a program whose execution has been initiated by the user. • At any moment, a job can be running or suspended. • Foreground job: o a program which has control of the terminal • Background job: o runs concurrently with the parent shell and does not take control of the keyboard. • Start a job in the background by appending & • Commands: ^Z (stop signal), jobs, fg, bg, kill Because the shell is essentially a program launcher, it has control over the processes it creates (they are child processes of the shell). Through the command line interface, you could move a job to the foreground, background, stop it, or kill it. File Name Expansion ls *.c rm file[1-6].? cd ~/bin ls ~reid ls *.[^oa] - ^ in csh, ! in sh When the shell sees a wild card character, it attempts to match the pattern to existing files. It then replaces the wild card expression with each file that matched it. If there were 2 files a1.c and a2.c then ls *.c actually becomes ls a1.c a2.c * stands in for 0 or more characters ? stands in for exactly one character [1-6] stands in for one of 1, 2, 3, 4, 5, 6 [^oa] stands in for any char except o or a ~/ stands in for your home directory ~reid stands in for reid’s home directory Sequences Enter series of simple commands or pipelines separated by semicolons (;) Commands will be executed in sequence Each process the shell executes has return value – shell checks for it conditional execution with &&,|| && executes next command if return value is 0 (success) || executes next command if return value is nonzero (failure) example: g++ myprog.cpp || echo compile failed Scripts A bunch commands stored inside a text file Must add execute permission with chmod provides all the programming essentials: variables, logic, control flow, loops powerful way to automate tasks • First line of script should always be #! Pathname • Pathname will be used to interpret script • If #! Not there, sh (Bourne shell) is used Should provide explicit return value by using exit Subshell A copy of the current shell (subshell is a child of current shell) Created when: • grouped commands with ; are executed. Parent waits until child (subshell) finishes • Script is executed. Parent sleeps until script is done • A background job is executed. Parent runs concurrently with sub-shell In 1st two cases, parent doesn’t wait if jobs are in background Every shell contains 2 areas: an environment space and a local variable space. A child shell inherits a copy of the parent’s environment space and a clean local variable space. Shell Programming (Bourne shell) • file starts with #!/bin/sh o absolute path to the shell program o not the same on every machine. • can also write programs interactively by starting a new shell at the command line. • Tip: this is a good way to test your shell programs The 1st line #!/bin/sh tells Unix what shell should interpret the current script (because the syntax & features for each shell are different). If you specify none, it defaults to using the Bourne shell to interpret the commands in the script. Example • In a file: #! /bin/sh echo “Hello World!” Running the above script is equivalent to: penguin% sh bash$ echo “Hello World!” Hello World! bash$ exit penguin% Commands • You can run any program in a shell by calling it as you would on the command line. • When you run a program like grep or ls in a shell program, a new process is created. • There are also some built-in commands where no new process is created (eg. cd, echo) Variables • local shell variables vs. environment variables • environment variables copied to subshell, local shell variables are not. • Use export to make local shell variables into environment variables: sh-2.05a$ pet1=fuzzoo sh-2.05a$ pet2=moomoo sh-2.05a$ echo $pet1 $pet2 fuzzoo moomoo sh-2.05a$ export pet1 sh-2.05a$ sh sh-2.05a$ echo $pet1 $pet2 fuzzoo sh-2.05a$ exit sh-2.05a$ echo $pet1 $pet2 fuzzoo moomoo Common Predefined Environment Variables • $HOME – full path of your home directory • $PATH – list of directories to search for files • $MAIL – full path of mailbox • $USER – your user name • $SHELL – full path of your login shell For the Bourne shell, the initial values of the environment variables are set by looking at the .profile file in your home directory. Other Built-in Variables • $$ – process id of the current shell • $! – process id of last background command • $0 – name of shell script • $1...$9 – commandline arguments • $* – list of all commandline args • $? – return value of the previously executed command • $# – number of commandline params. Example: sh-2.05a$ cat > shelldemo.sh #! /bin/sh echo $HOME echo $PATH echo commandline=$* echo 1st arg=$1 echo $? gcc hello.cpp echo $? gcc noexist.cpp echo $? sh-2.05a$ chmod ugo+x shelldemo.sh sh-2.05a$ shelldemo.sh arg1 arg2 arg3 /u/kenxu /u/kenxu/bin/Linux:/u/kenxu/bin:/local/bin/X11:/usr/X11R6/bin:/local/bin:/usr/ucb:/bin:/usr/bin:.:/u/kenxu commandline=arg1 arg2 arg3 1st arg=arg1 0 0 gcc: noexist.cpp: No such file or directory gcc: no input files 1 Notice the first 2 echo of $? has value 0 (success), while the last has value 1 ( failed). Variable Access $name – replaced by value of name ${name} – replaced by value of name. Useful if ${name} is followed by alphanumerical characters: verb=play Doesn’t work: echo I like $verbing Works: echo I like ${verb}ing Output: I like playing Local Shell Variables • name=value – assignment (spaces matter) • $name – replaced by value of name • variables can have a single value or list of values. • Single value: bindir=“/usr/bin” • List of values (separated by spaces): searchdirs=“~/tests $HOME/test2 .” Example: ($ is the default sh prompt) $ bindir=“/usr/bin” $ searchdirs=“~/tests $HOME/test2 .” $ echo $searchdirs ~/tests /u/reid/test2 . $ echo $bindir /usr/bin Quoting • Double quotes inhibit wildcard replacement only. • Single quotes inhibit wildcard replacement, variable substitution and command substitution. • Back quotes cause command substitution. Quoting example $ echo Today is date Today is date $ echo Today is `date` Today is Thu Sep 19 12:28:55 EST 2002 $ echo ”Today is `date`” Today is Thu Sep 19 12:28:55 EST 2002 $ echo ’Today is `date`’ Today is `date` Another Quoting Example • What do the following statements produce if the current directory contains the following non- executable files? a b c $ echo * $ echo ls * $ echo `ls *` $ echo ”ls *” $ echo ’ls *’ $ echo `*` Command substitution causes another process to be created. Which of the following is better? src=`ls *.c` or src=*.c Here Documents • Easy way to provide input to commands without using auxiliary files. sh-2.05a$ cat << ENDOFTEXT > type some text and > have cat repleat > it! > ENDOFTEXT type some text and have cat repleat it! sh-2.05a$ It doesn’t have to be ENDOFTEXT like we have above. In fact, it could be an arbitrary token. Once the token is detected, everything entered up until that point is stored in a buffer, and that buffer is provided as the standard input to some command. Arithmetic Bourne Shell doesn’t support arithmetic natively. Use expr command instead. expr • Since shell scripts work by text replacement, we need a special function for arithmetic. • expr command can be used for arithmetic and some string manipulation • spaces are very important! We’ll show how to use expr by example.