sign in sign up
<<
Home , Bourne shell, Command (Unix), Exit (command)

Shells and scripting

What is a Shell? • A shell is a line interpreter that is the interface between the user and the OS. • A “program launcher” of sorts. • The shell: o analyzes each command o determines what actions are to be performed o performs the actions • Example:

–l file1 > file2

Which shell? • sh – o common, other shells are a superset o Good for programming • csh or – default for command line on CDF o -like syntax o Best for interactive use. Not good for programming. • – default on (Bourne again shell) o Based on sh, with some csh features. • korn – written by David Korn o Based on sh – Some claim best for programming. o Commercial product.

Common shell facilities Shell startup When a shell is invoked, it does the following: 1. Read a special startup (usually in home directory) 2. display prompt and for command 3. Ctrl-D on its own line terminates shell, otherwise, goto step 2.

Shell startup files used to set shell options, set up environment variables, sh – executes .profile if it’s there. ksh – executes .profile if in interactive mode. Executes $ (usually $HOME/.kshrc) csh – executes .cshrc if it exists. If a login shell, executes .login bash – executes .bashrc, if a login shell, executes .bash_profile instead

Executables vs. built-in commands

Most commands you run are other compiled programs. Found in /bin Example: – shell locates ls binary in /bin directory and launches it Some are not compiled programs, but built into the shell: ,

Input-output prog < infile > outfile ls > outfile 2>&1 # sh stdout and stderr

Pipelining commands send the output from one command to the input of the next: ls -l | wc –aux | reid |

Before a program is executed, the shell recognizes the special characters such as <, >, |, and rewires the standard input, output, or error file descriptors of the program about to be executed to point to the right files (or the standard input of another program). Once everything is setup, the shell executes the program. The program is blissfully ignorant of where its input is coming from.

Job Control • A job is a program whose execution has been initiated by the user. • any moment, a job can be running or suspended. • Foreground job: o a program has control of the terminal • Background job: o runs concurrently with the parent shell and does not take control of the keyboard. • a job in the background by appending & • Commands: ^Z (stop ), jobs, fg, bg,

Because the shell is essentially a program launcher, it has control over the processes it creates (they are child processes of the shell). Through the command line interface, you could a job to the foreground, background, stop it, or kill it.

File Name Expansion ls *.c file[1-6].? cd ~/bin ls ~reid ls *.[^oa] - ^ in csh, ! in sh

When the shell sees a wild card character, it attempts to match the pattern to existing files. It then replaces the wild card expression with each file that matched it. If there were 2 files a1.c and a2.c then ls *.c actually becomes ls a1.c a2.c

* stands in for 0 or characters ? stands in for exactly one character [1-6] stands in for one of 1, 2, 3, 4, 5, 6 [^oa] stands in for any char except o or a ~/ stands in for your home directory ~reid stands in for reid’s home directory

Sequences

Enter series of simple commands or pipelines separated by semicolons (;) Commands will be executed in sequence Each process the shell executes has return value – shell checks for it conditional execution with &&,|| && executes next command if return value is 0 (success) || executes next command if return value is nonzero (failure) example: g++ myprog.cpp || echo compile failed

Scripts

A bunch commands stored inside a text file Must add execute permission with provides all the programming essentials: variables, logic, , loops powerful way to automate tasks

• First line of should always be #! Pathname • Pathname will be used to interpret script • If #! Not there, sh (Bourne shell) is used Should provide explicit return value by using

Subshell

A of the current shell (subshell is a child of current shell) Created when: • grouped commands with ; are executed. Parent waits until child (subshell) finishes • Script is executed. Parent sleeps until script is done • A background job is executed. Parent runs concurrently with sub-shell In 1st two cases, parent doesn’t wait if jobs are in background

Every shell contains 2 areas: an environment space and a local variable space. A child shell inherits a copy of the parent’s environment space and a clean local variable space.

Shell Programming (Bourne shell)

• file starts with #!/bin/sh o absolute to the shell program o not the same on every machine. • can also programs interactively by starting a new shell at the command line. • : this is a good way to your shell programs

The 1st line #!/bin/sh tells Unix what shell should interpret the current script (because the syntax & features for each shell are different). If you specify none, it defaults to using the Bourne shell to interpret the commands in the script.

Example

• In a file: #! /bin/sh echo “Hello World!”

Running the above script is equivalent to: penguin% sh bash$ echo “Hello World!” Hello World! bash$ exit penguin%

Commands • You can run any program in a shell by calling it as you would on the command line. • When you run a program like grep or ls in a shell program, a new process is created. • There are also some built-in commands where no new process is created (eg. cd, echo)

Variables

• local shell variables vs. environment variables • environment variables copied to subshell, local shell variables are not. • Use export to local shell variables into environment variables: sh-2.05a$ pet1=fuzzoo sh-2.05a$ pet2=moomoo sh-2.05a$ echo $pet1 $pet2 fuzzoo moomoo sh-2.05a$ export pet1 sh-2.05a$ sh sh-2.05a$ echo $pet1 $pet2 fuzzoo sh-2.05a$ exit sh-2.05a$ echo $pet1 $pet2 fuzzoo moomoo

Common Predefined Environment Variables

• $HOME – full path of your home directory • $PATH – list of directories to search for files • $ – full path of mailbox • $USER – your user name • $SHELL – full path of your login shell

For the Bourne shell, the initial values of the environment variables are set by looking at the .profile file in your home directory.

Other Built-in Variables

• $$ – process id of the current shell • $! – process id of last background command • $0 – name of • $1...$9 – commandline arguments • $* – list of all commandline args • $? – return value of the previously executed command • $# – number of commandline params.

Example: sh-2.05a$ > shelldemo.sh #! /bin/sh echo $HOME echo $PATH echo commandline=$* echo 1st arg=$1 echo $? gcc hello.cpp echo $? gcc noexist.cpp echo $? sh-2.05a$ chmod ugo+x shelldemo.sh sh-2.05a$ shelldemo.sh arg1 arg2 arg3 /u/kenxu /u/kenxu/bin/Linux:/u/kenxu/bin:/local/bin/X11:/usr/X11R6/bin:/local/bin:/usr/ucb:/bin:/usr/bin:.:/u/kenxu commandline=arg1 arg2 arg3 1st arg=arg1 0 0 gcc: noexist.cpp: No such file or directory gcc: no input files 1

Notice the first 2 echo of $? has value 0 (success), while the last has value 1 ( failed).

Variable Access

$name – replaced by value of name ${name} – replaced by value of name. Useful if ${name} is followed by alphanumerical characters: verb=play Doesn’t work: echo I like $verbing Works: echo I like ${verb}ing Output: I like playing

Local Shell Variables

• name=value – assignment (spaces matter) • $name – replaced by value of name • variables can have a single value or list of values. • Single value: bindir=“/usr/bin” • List of values (separated by spaces): searchdirs=“~/tests $HOME/test2 .”

Example: ($ is the default sh prompt) $ bindir=“/usr/bin” $ searchdirs=“~/tests $HOME/test2 .” $ echo $searchdirs ~/tests /u/reid/test2 . $ echo $bindir /usr/bin

Quoting

• Double quotes inhibit wildcard replacement only. • Single quotes inhibit wildcard replacement, variable substitution and . • Back quotes cause command substitution.

Quoting example $ echo Today is date Today is date $ echo Today is `date` Today is Thu Sep 19 12:28:55 EST 2002 $ echo ”Today is `date`” Today is Thu Sep 19 12:28:55 EST 2002 $ echo ’Today is `date`’ Today is `date`

Another Quoting Example • What do the following statements produce if the current directory contains the following non- executable files? a b c $ echo * $ echo ls * $ echo `ls *` $ echo ”ls *” $ echo ’ls *’ $ echo `*`

Command substitution causes another process to be created. Which of the following is better? src=`ls *.c` or src=*.c

Here Documents

• Easy way to provide input to commands without using auxiliary files. sh-2.05a$ cat << ENDOFTEXT > some text and > have cat repleat > it! > ENDOFTEXT type some text and have cat repleat it! sh-2.05a$

It doesn’t have to be ENDOFTEXT like we have above. In fact, it could be an arbitrary token. Once the token is detected, everything entered up until that point is stored in a buffer, and that buffer is provided as the standard input to some command.

Arithmetic

Bourne Shell doesn’t support arithmetic natively. Use command instead. expr • Since shell scripts work by text replacement, we need a special function for arithmetic. • expr command can be used for arithmetic and some string manipulation • spaces are very important!

We’ll show how to use expr by example. The rest of the cases are analogous to what’s described below: sh-2.05a$ x=1 sh-2.05a$ echo $x 1 sh-2.05a$ x=`expr $x + 2` sh-2.05a$ echo $x 3 sh-2.05a$ echo `expr \( 2 + 4 \) \* 5` 30 sh-2.05a$ echo `expr length "cat"` 3 sh-2.05a$ echo `expr substr "donkey" 4 3` key sh-2.05a$ echo `expr index "donkey" "key"` 4 sh-2.05a$ echo `expr match "smalltalk" '.*lk'` 9 sh-2.05a$ echo `expr match "transputer" '.*lk'` 0 sh-2.05a$ echo `expr \( 4 \> 5 \)` 0 sh-2.05a$ echo `expr \( 4 \> 5 \) \| \( 6 \< 7 \)` 1

Boolean conditions

• test is an utility that gives 0 if boolean condition is true and non-zero status otherwise • used by control statements such as if. • syntax: test expression or [ expression ]

sh-2.05a$ if test 2 -eq 3 > then > echo equal > else > echo not equal > fi not equal sh-2.05a$ if [ 2 -eq 2 ]; then echo equal; else echo not equal; fi equal sh-2.05a$

test can be used on a wide variety of conditions, listed below: (these are just copied from man test)

! EXPRESSION EXPRESSION is false

EXPRESSION1 -a EXPRESSION2 both EXPRESSION1 and EXPRESSION2 are true

EXPRESSION1 -o EXPRESSION2 either EXPRESSION1 or EXPRESSION2 is true

-z STRING the length of STRING is zero

STRING1 = STRING2 the are equal

STRING1 != STRING2 the strings are not equal

INTEGER1 -eq INTEGER2 INTEGER1 is equal to INTEGER2

-ge , -gt ,-le, -lt, -ne (≥, >, ≤, <, ≠)

-d FILE FILE exists and is a directory -e FILE FILE exists -f FILE FILE exists and is a regular file -r FILE FILE exists and is readable - FILE FILE exists and is writable -x FILE FILE exists and is executable

There are many more. Look at man test.

Control Statements if

The general syntax for if is:

if command1 then command2 command3 ...etc elif command4 command5 ...etc else command6 fi if makes its decision based on the exit status of commands. In the above pseudo code, if looks at the exit status of command1. If it’s 0, then command2...etc. are executed, otherwise, it executes command4 and looks at its exit status, and so on. The elif part is optional and may be repeated. null command

Does nothing Consider an “unload” program: ls –a | cpio –o > /dev/rmt0 if [ $? –eq 0 ] then rm * else echo problem occurred with cpio fi How to this to a backup program?

To convert the unload program to the backup program, simply the rm * command with : (null command), which does nothing if the backup was ok. case case takes a string and tries to match it to one or more patterns. If a pattern matches, then the associated statements are executed. The general syntax is: case value in pattern1) command command;; pattern2) command command;; ... patternn) command; esac

The patterns may contain wild cards, much like the commandline interpreter. Here’s an example: sh-2.05a$ =2 sh-2.05a$ case $choice in > "1") > echo choice was 1 > ;; > "2"|"3") > echo choice was 2 or 3 > ;; > *) > echo unknown choice > ;; > esac choice was 2 or 3

Notice you can use the | for multiple patterns and wild cards like *.

Loops

For

The for loop takes a list of arguments and executes the loop one for each word in the list. The general syntax is: for variable in arg1 arg2 ...argn do command ... command done example for color in red green blue pink do echo The sky is $color done

While

Unlike the for loop, the while loop tests on a boolean condition to decide whether or not to continue the loop. The general form of the while loop is: while command do command command ... command done

Like if, while tests the exit status to see if it’s equal to 0. If so, the loop is continued. example sh-2.05a$ count=1 sh-2.05a$ while [ $count -lt 10 ] > do > echo $count > count=`expr $count + 1` > done 1 2 3 4 5 6 7 8 9 sh-2.05a$ until until is analogous to while, and has the form: until command do command command ... command done

Functions

Define function with: funcname () { command ... command; }

Below is a sample where a function just prints the parameters passed to it: sh-2.05a$ myfunc() { echo $1 $2 ; } sh-2.05a$ myfunc param1 param2 param1 param2 sh-2.05a$ set and shift set – assigns positional parameters to its arguments. $ set `date` $ echo “The date today is $2 $3, $6” The date today is Aug 27, 2001 shift – change the meaning of the positional parameters. Causes $2..$n to be renamed $1..$(n-1) and $1 to be lost. read

• Used when input is required from user. • read one line from standard input and assigns successive words to the specified variables. Leftover words are assigned to the last variable. sh-2.05a$ read a1 a2 value1 value2 sh-2.05a$ echo $a1 $a2 value1 value2 sh-2.05a$

Debugging set –vx (v: echo commands as they are read, x: echo as commands as they are executed) Set – to turn off tracing

$ cat testscript2 #!/bin/sh set -vx a=`ls -l` b=5 ls -l | '{ $1;}' $ testscript2 a=`ls -l` ls -l ++ ls -l + a=total 1 -rw------1 kenxu instrs 0 May 25 10:26 myfile1 -rw------1 kenxu instrs 0 May 25 10:26 myfile2 -rwx--x--x 1 kenxu instrs 58 May 25 10:19 testscript2 b=5 + b=5 …