PROGRAMMING SHELL SCRIPTS
Introduction
One of the nice things about the C-shell is that it provides access to the individual UNIX utilities (i.e., UNIX commands) that can be run from the command prompt. For example, a complex command using pipes, such as
who | cut –c-8 | sort –u | pr –l1 -8 –w78 –t
provides a compact list of all users who are currently logged on to Hercules. If the command is entered properly, a formatted list of users should be reported that looks similar to the following:
abraham appleton charlie frizzie howard jerry kosmo linda peterson root sutton
While this is one form of shell programming, the command would be more useful if it did not have to be entered at the command prompt each time it was needed. That is, the command would be more useful if it was contained in a shell script (i.e., a file that contains shell commands). For example, the following could be entered into an executable file called showUsers:
#! /usr/bin/tcsh -f who | cut -c-8 | sort -u | pr -l1 -8 -w78 -t exit 0
It would then merely require that the showUsers command be entered at the command prompt to obtain a compact list of current users.
In this section, we focus on the creation and execution of shell script files.
Shell Variables
1. Shell variables hold values which are actually lists containing zero or more elements (called words). - Words are just character strings.
- If a word contains only digits, without a leading zero and with no embedded spaces, it can be used as an operand in an arithmetic operation.
- Any shell variable can be used in a manner similar to a Boolean variable. If the variable is set (i.e., it has been given a value), it represents the value true. If the variable is unset (i.e., it has not been given a value), it represents the value false.
2. While there are numerous built-in shell variables (enter set at the command prompt to see them all or echo $
- prompt: Contains the string which prints as the command prompt when you log on to Hercules. The default value is %. You can change it by using the set command.
- Example
set prompt = ‘your name here>’ (or “your name here>”)
This command will change the command prompt to your name followed by >. If you entered the name jimbo, the resulting prompt would be
jimbo>
- Example
set prompt = ‘your name here{\!}>’
This command will also show the command number, so the resulting prompt could be jimbo{23}>
- history: Controls the number of previously executed commands maintained in the history list.
- Example
set history = 50
- path: Contains a list of directories that tcsh will search for the commands (i.e., programs) named in the command line.
- Example
set path = ( /bin /usr/bin /usr/ucb . )
Sets the first three elements of path to system directories where standard UNIX commands are usually located. The fourth element (i.e., the period) represents the current directory.
- status: Contains the completion status of the last command executed (i.e., usually zero and non- zero mean successful and unsuccessful completion, respectively.)
- $$: Contains the process id of the shell.
3. In a UNIX command, we can use characters called magic characters. Specifically, whenever the characters *, ?, [, and { are encountered, expansions are performed. The * character is a wild card character. For example, assume the command to be executed is
CC *.c The shell will use *.c as a pattern, attempt to match it against all of the file names in your directory, and compile all those files ending in .c (known as file name expansion). Similarly, the ? character is a wild card character for matching a single character. For example, assume the command to be executed is
CC file?.c
The shell will attempt to match file?.c against all of the file names in your directory, and compile all those whose names begin with file, followed by any single character, followed by .c. An alternative to the ? character uses square brackets to enumerate particular single characters to match. For example, assume the command to be executed is
CC file[134].c
The shell will compile file1.c, file3.c, and file4.c. Similarly, curly braces can be used to form a string product. For example, assume the command to be executed is
CC file{1,3,4}.c
The shell will compile file1.c, file3.c, and file4.c.
4. Example
Given what we now know from above in item 3, let’s consider the use of single quotes versus double quotes in UNIX commands. Consider the command
echo *** WAKEUP!! ***
This command has serious unintended problems because the shell will perform both history substitution (!! re-executes the most recent command) and file name substitution (for every asterisk encountered). The command could be written as
echo \*\*\* WAKEUP\!\! \*\*\* to resolve the problems. However, a better way to write the command is
echo ‘*** WAKEUP\!\! ***’ or echo “*** WAKEUP\!\! ***”
Single quotes protect the enclosed text from being split into words, suppress all forms of expansion, prevent the substitution of words that begin with $, and prevent the substitution of the history characters (i.e., !9, !! and !$). Double quotes protect the enclosed text from being split into words and suppress all forms of expansion. However, double quotes allow substitution. The bottom line: you have to be careful in using single and double quotes when magic characters and substitution characters are involved.
5. The set command is used to assign values to variables. It has several forms:
- set
- Example
set arguments
- set
- Example
set arguments = abc
Sets the variable arguments to the string “abc”. More precisely, the variable arguments contains a list with one element whose value is the string “abc”.
- set
set arguments = ( abc 123 def 456 )
Sets the variable arguments to a four-element list containing the values shown above.
- set
- Example
set arguments[2] = 789
Sets (changes) the value of the second word in the variable arguments (note that the indexes start at 1). That is, the variable arguments now contains the value
abc 789 def 456
- set: Generates a list of all currently set variables.
- @: An alternative form of the set command used exclusively for integer arithmetic and assignment.
- Example
@ i = 10 @ j = $i * 2 + 5 @ i++ @ j = ( $i – 5 ) / 2 Note: The first assignment is equivalent to set i = 10. However, none of the other assignment statements can be expressed using the set command.
6. The value of a shell variable is accessed by placing $ (i.e., the dollar symbol) before the name of the variable. It also has several forms:
- $
- Example
set arguments = ( abc 789 def 456 )
We can use the echo command to see the value of a variable.
echo $arguments
Outputs the value
abc 789 def 456
- ${
- Example
set arguments = ( abc 789 def 456 )
The command
echo $arguments outputs the value
abc 789 def 456
The command
echo $arguments xyz
outputs the value
abc 789 def 456 xyz
The command
echo $argumentsxyz
results in an undefined variable error message.
The command
echo ${arguments}xyz
outputs the value
abc 789 def 456xyz
- $
- Example The command
echo $arguments[2]
outputs the value of the second element of arguments, namely
789
- $
- Example
The command
echo $arguments[2-4]
outputs the value
789f 456
- $
- Example
The command
echo $arguments[2-] outputs the value
789f 456
7. Shell variables can also be tested in a couple of ways:
- $#
- Example
set arguments = ( abc 789 def 456 )
$#arguments returns the value 4.
- $?
Shell Expressions
Shell expressions are formed by combining variables and constants with operators that are similar to those in the C and C++ programming languages.
1. Arithmetic operators.
- *: Multiplication.
- /: Division.
- %: Modulo division. - +: Addition.
- -: Subtraction.
Note: Operator precedence is the same as in C/C++. Enclose expressions within parentheses to alter the precedence.
2. Assignment operators.
- =: Assign value.
- +=: Assign value after addition.
- -=: Assign value after subtraction.
- *=: Assign value after multiplication.
- /=: Assign value after division.
- %=: Assign value after modulo division.
- ++: Increment value.
- --: Decrement value.
3. Logical operators.
- &&: Logical AND.
- ||: Logical OR. - !: Logical negation.
4. Comparision operators.
- == and =~: Equality.
- != and !~: Inequality.
- <=: Less than or equal.
- >=: Greater than or equal.
- <: Less than.
- >: Greater than.
5. File inquiry operators.
- -d
- -e
- -f
- -o
- -r
- -w
- -z
- !: Negates the result of each of the above tests.
Shell Control Structures
Shell control structures are similar to those found in the C and C++ programming languages.
1. if Command
- One-line form: The
if (
- Example
if ( -w $file1 ) mv $file1 $file2
- Example
if ( $?file ) rm $file
- Multi-line form: The if and endif lines must be contained on one line (however, the if line can be split into multiple lines using a backslash (i.e., \) character).
if (
- Example
set file1 = $< set file2 = $< if ( -o $file1 && ! -e $file2 ) then mv $file1 $file2 endif
- Multi-line forms using else: Provide facilities for multi-way branching.
if (
if (
2. while Command
- The general form: while (
- The while and end lines must be contained on one line.
- Within the loop body, break and continue commands may be used to terminate and short- circuit execution, respectively (i.e., they work the same as the like-named statements found in C and C++).
3. foreach Command
- The general form:
foreach
- The loop index variable
- Example
foreach file ( *.c ) set size = `wc –l $file` if ( $size > 1000 ) then echo “$file skipped: Too large to print” continue endif echo “$file printing” pr $file | lpr -Pdept2 end Note: The expression *.c is expanded to the list of words containing all file names in the current directory whose extension is .c. Any command enclosed within a pair of single back quotes (i.e., `wc –l $file`) will be executed and its output treated as a list of words. However, in this case, the result is a list containing a single numeric value representing the number of lines in the file whose name is stored in the variable file.
4. switch Command
- The general form:
switch (
- The default clause is optional.
- The commands associated with a case must not be on the same line as the case. For example, case
will result in an error.
- Example
switch ( $#arguments ) case 0: zero or more lines containing shell commands breaksw case 1: zero or more lines containing shell commands breaksw case 2: zero or more lines containing shell commands breaksw default: zero or more lines containing shell commands endsw
- The value between the parentheses in the switch command does not need to be an integer, it can be a word.
- Example
switch ( $argv[$i] ) case end: breaksw case list: @ k = $i + 1 ls $argv[$k] breaksw case delete: case erase: @ k = $i + 1 rm $argv[$k] breaksw endsw
5. goto Command
- The general form:
- A label can be placed anywhere in a shell script file, but it must be on a line by itself.
- The goto command can be placed anywhere before or after the label.
Other Shell Variables
The following are a few special shell variables you might find useful.
1. $<: Indicates that the value of a variable should be read from standard input.
2. $0 … $9: These are special variable names for up to 10 arguments in a command line. 3. $argv and $*: Contain a list of command line arguments.
4. The shift command can be used as a convenient way to loop over all command line arguments.
- shift: Discards argv[1] and moves the remaining arguments one position to the left.
- shift
- It is an error for argv not to be set or to have less than one argument.
5. $#argv and $#: Contain the number of command line arguments.
Examples
You can copy and paste each shell script in this example and the following examples into a file and run it. They demonstrate most of the useful features and characteristics of shell script programming.
1. SCRIPT = argDemo
#! /usr/bin/tcsh -f
if ( $#argv == 0 ) then echo “argDemo: usage is argDemo
switch ( $argv[1] ) case [yY][eE][sS]: echo “argv[1] is ‘yes’” breaksw case [nN][oO]: echo “argv[1] is ‘no’” breaksw default: echo “Enter yes or no in any sequence of upper- and lower-case” exit 1 endsw
exit 0
Note: Comments in a script file are lines that start with a #. However, due to a historical artifact in the development of the C- shell, it was decided that lines beginning with the directive #! (which looks like it should be a comment) would make a script file directly executable. That is, the kernel can start this program, even though it is not machine code. By default, if the first line does not contain such a directive, the script file would be executed by the (more primitive) Bourne shell (i.e., /bin/sh). Any options that would normally be supplied to the shell must then be given in the first-line shell directive. For example, the -f option in the example above, ignores the ~.tcshrc file resulting in faster execution time.
Background on ~.tcshrc: The ~.tcshrc file is a file in your home directory that is read and its contents executed every time tcsh is invoked. In addition, if a particular invocation of tcsh is the shell created at the time you login, a file named ~.login will be read and its contents executed immediately following the ~.tcshrc file. These files are typically used to set variables and aliases that you would like to use (basically, they “customize” your shell session). The ~.tcshrc file should be kept as short as possible because new shell instances are created fairly often. So, put the stuff that needs to be done once in the ~.login file (e.g., most shell variables like path and prompt, environment variables, and most aliases).
Note: Scripts should be terminated by an exit command. Optionally, but typically, exit should return an error code to the caller (e.g., exit 0 meaning no error and exit
2. SCRIPT = ifElseDemo #! /usr/bin/tcsh -f
set class set number = $argv[1]
if ( $number < 0 ) then @ class = 0 else if ( 0 <= $number && $number < 10 ) then @ class = 1 else @ class = 2 endif
echo “The number $number is in class $class”
exit 0
3. SCRIPT = forEachDemo1
#! /usr/bin/tcsh -f
foreach file ( `ls -1` ) echo $file wc $file end
exit 0
4. SCRIPT = foreachDemo2
#! /usr/bin/tcsh -f foreach i ( 10 15 20 40 ) echo $i end
foreach i ( a b c 17 ) echo $i end
exit 0
5. SCRIPT = zap
#! /usr/bin/tcsh -f
if ( $#argv != 1 ) then echo "zap: usage is zap
set ps_output = "`ps -u $argv[1]`" @ i = 2
set ps_test while ( $i <= $#ps_output ) set line = ( $ps_output[$i] ) if ( $line[4] != "ps" && $line[4] != "tcsh" && $line[4] != "zap" ) then set ps_test = ( $ps_test $i ) endif @ i ++ end foreach i ( $ps_test ) set line = ( $ps_output[$i] ) set process_no = $line[1] RETRY: echo echo -n "zap: Kill CMD = ${line[4]} (PID = ${line[1]})? " set response = $< if ( $response =~ n* || $response =~ N* ) then continue else if ( $response =~ q* || $response =~ Q* ) then break else if ( $response =~ y* || $response =~ Y* ) then kill -9 $process_no continue else echo "zap: Invalid response: Enter y (yes), n (no), or q (quit)" goto RETRY endif end
exit 0
6. SCRIPT = saveFiles
#! /usr/bin/tcsh -f
if ( $#argv != 2 ) then echo “saveFiles: usage $0 dir1 dir2” exit 1 endif set dir1 = $argv[1] set dir2 = $argv[2] if ( ! -e $dir1 ) then echo “saveFiles: $dir1 does not exist” exit 1 endif if ( ! -d $dir1 ) then echo “saveFiles: $dir1 is not a directory” exit 1 endif if ( -e $dir2 ) then echo “saveFiles: $dir2 already exists” exit 1 endif mkdir $dir2 if ( $status != 0 ) exit 1 foreach file ( $dir1/* ) if ( -d $file ) then echo “saveFiles: $file (directory) not copied” else echo “saveFiles: $file (file) copied” cp $file $dir2 endif end exit 0 Debugging Shell Scripts
In what follows, assume the name of the script file being debugged is scriptFile.
7. To check the script file for syntax errors:
$ tcsh –n scriptFile
8. To output each line of the script file as it is checked for syntax errors:
$ tcsh –nv scriptFile
9. To output each line of the script file before it is executed:
$ tcsh –v scriptFile
10. To output each line of the script file after variable and command substitutions have occurred and before the line is executed:
$ tcsh –vx scriptFile
11. Use echo commands in a shell script to:
- Output statements or flags that indicate where the script is currently executing (like a trace).
- Output the values of variables.
- Prevent disasters by placing the echo command at the start of a line. - Example
Assuming that fileName contains junkola and trash contains /home/venus/hilder, then
echo mv $fileName $trash/fileName
will only print
mv junkola /home/venus/hilder/fileName
to the screen. That is, it does any expansions required, but does not actually execute the instruction. Note that what was probably intended was to have a $ sign at the beginning of the second fileName.
12. Use an exit command to prevent a shell script from executing passed a particular point in the script.