SHELL PROGRAMMING

GODFREY C. MUGANDA

1. Introduction

A Unix shell is a program that interprets user’s commands typed at the keyboard. Unix systems come with thousands of commands in the form of programs. These programs implement all kinds of functionality and can be invoked at the shell prompt. A Unix shell functions as a command intepreter, executing these programs at the behest of the user. In addition to executing commands typed by users, Unix shells support a program- ming language with decision making and looping control structures. Instead of typing commands at the keyboard, you can edit a sequence of commands into a file called a shell script. Your shell script can use sequencing, decision making, and looping control structures to tie together the commands available and perform useful tasks. There are several types of shell programs available for Unix systems. Here we cover bash, the Bourne-Again Shell. This is the most popular shell on Unix systems.

2. Some Unix commands

Let’s cover a few of the many Unix commands available. You can simply type these at the shell prompt in a terminal to see what they do, or you can use the man pages to learn more about them.

(1) who (2) cal (3) whoami (4) date (5) cat (6) grep (7) mv (8) printf (9) echo (10) ls (11) uname (12) wc

Notice that information for these commands can be found in section 1 of the man pages. Unix manual pages are divided into 8 sections: the first few are

1 Commands available to users 2 Unix and C system calls 3 C library routines for C programs 1 2 GODFREY C. MUGANDA

So for example, printf is both a user command and a C library function. The command man 1 printf returns information about the user command, while man 3 printf returns information about the printf library function.

3. Trying Out the echo command

Here is the information on echo returned by the man pages. NAME echo - display a line of text SYNOPSIS echo [SHORT-OPTION]... [STRING]... echo LONG-OPTION

DESCRIPTION Echo the STRING(s) to standard output.

-n do not output the trailing newline -e enable interpretation of backslash escapes -E disable interpretation of backslash escapes (default)

If -e is in effect, the following sequences are recognized:

\\ backslash \a alert (BEL) \b backspace \c produce no further output \e escape \f form feed \n new line Basically, the echo command outputs a line of text. The command takes a list of 0 or more arguments, where the arguments are separated by spaces. Each argument is printed to standard output, with spaces being used to separate the arguments. If an argument is a string containing a space, you can put quotes around it to have echo treat it as a single argument. This practice of putting quotes around a single argument to echo is widespread. It is routinely used even when it is not needed. Try the following commands echo echo hello world echo hello world echo "hello world" echo -e "my \bmother" echo "get\nout" echo -e "get\nout" SHELL PROGRAMMING 3

4. Hello World

Let us do the obligatory “Hello World!” in a Shell program. Start your favourite text editor, say gedit, and use is to create a text file named hello.sh. The .sh extension is not necessary, but it is conventional to use it for a shell script. gedit hello.sh

Edit a single line into the file echo "Hello World!" save the file, and exit the editor. Before we can run the program, we must make its file executable. Type the com- mand chmod a+x hello.sh

This command changes the file mode of hello.sh to grant eXecute permissin to All. Now you can execute the file with the command.

./hello.sh

That is it: we have now written our first shell script.

5. Variables and Assignment

The shell does not require variables to be declared. The variable is created the first time it is used. To assign a value to a variable, use the assignment operator = with no spaces around it. To access the value in a variable, prefix the variable with a $. Try these commands at the shell prompt: greeting="Hello out there!" echo greeting echo $greeting number=12 echo number echo $number number1=$number echo $number1 echo "$number1" echo "$number days of Christmas"

Notice how the value in a variable is evaluated even when it is part of a string. It is not uncommon for shell variables to be enclosed in {} when being evaluated. For example, greeting="Hello out there!" echo greeting echo ${greeting}

Enclosing with {} is optional, and supposedly, aids readability. 4 GODFREY C. MUGANDA

6. Reading Input

You typically use echo or printf to do output in a shell script. To input values and store them in a variable, Use read. Use your text editor to create a file input.sh with the following contents echo -n "Enter a number: " read x echo "You entered $x" echo -n "Enter a string: " read str echo "You entered $str"

Make it executable and run it to see how it works.

7. Comments

Comments start with # and end at the end of line. You can find many examples of comments in shell scripts online, or in any book on Shell programming.

8. Command Substitution

The string that a program outputs to its standard output can be captured and used in a Shell program. To capture the ouput of a command, enclose the invocation of the command in backticks. The backtick is the backward-leaning single quote that occupies the same key as the tilde at the top left of your keyboard. For example, try typing d=‘date‘ echo "The day and time is $d" at the command line. You will see that the output of the date command is captured, assigned to the variable d, and then output as part of the argument to echo. This mechanism is called command substitution, or sometimes, command expansion. There are two alternative forms (syntax) for command substitution.

‘command‘ and

$(command)

The shell performs the expansion by executing COMMAND and replacing the com- mand substitution with the standard output of the command, with any trailing newlines deleted. Thus echo ‘date‘ and echo $(date) SHELL PROGRAMMING 5 do the same thing. Exercise: Use the uname command to print the name of the operating system, name of the network host, and the the name of the CPU processor that the shell script is running on.

9. Arithmetic Expansion

There are several ways to evaluate arithmetic expressions. Let us try something that we would expect to work. echo 2+3 d=17+2 echo d echo $d When you try this, you quickly realize that the Shell does not evaluate arithmetic expressions written like this. This is because evaluation of arithmetic expressions requires a mechanism called arithmetic expansion. The syntax of arithmetic is

$((expression)) Try echo $((2+3)) d=17+2 echo $((d)) echo $((d+3)) You should get the expected results. Nearly all the operators you have come to know and love from C and C-derived languages are supported by the shell. For example echo $((1 ? 23 : 24)) is an example of the conditional operator ? :. The above command will ouput 23. If you use bash, you can also use the square brackets syntax

$[expression] in place of

$((expression))

10. The expr command

The expression command evaluates its arguments and writes the result to standard output. This is different from expression expansion, which yields the value of an expression but does not write it to standard output. The arguments of the expr command must be separated by whitespace. To learn more about expr, use the command man expr Here are examples of using the command expression 6 GODFREY C. MUGANDA expr 2 + 3 expr 2 \* 3 These two commands yield 5 and 6, the sum and product of 2 and 3. The * operator normally stands for wildcard; this usage has to be escaped. Notice the spaces around the arguments to expr. Here are more examples expr 2 + 2 = 4 + 1 expr 2 + 3 = 3 + 1 expr 2 != 4 These commands process boolean expression and output 1 for true and 0 for false. Note from the man pages that expr can operate on strings and regular expressions expr length "I love linux" expr index hello l The first of the above prints the number of characters in the string I love linux whereas the second prints the position that the substring l is found in the string hello. Note that positions of characters in strings are 1-based rather than 0-based. A return value of 0 for the index means the searched-for substring was not found.

11. Using pipes

You can form pipelines of processes in the shell in a way similar to how you can use pipes of processes on the command line. You use the | symbol to form a pipeline between two processes. For example, the command wc -w will count the number of words in the file used as its standard input. The command cat file will send the contents of the file passed on the command line to standard input. For example, to count the number of words in the file, just write cat file | wc -w The standard output of the cat command is piped into the standard input of the wc command. You can type the statement at the shell prompt, or put it in a shell script. The arithmetic facilities built into the shell are very weak. You can process integers, but not floats. Also, there is no power operator. For example, you cannot write expr 2 ^ 3 if you want to raise 2 to the power 3: the shell does not recognize the command and you get an error. There is a command called bc that can do more sophisticated calculations. bc is a calculator program with a complete programming language of its own. You can learn more about bc man bc SHELL PROGRAMMING 7

You can use bc on the command line. Simply type bc and then start typing expressions to be evaluated. For example, here is a sample interaction gcm@linuxmint ~ $ bc bc 1.06.95 Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006 Free Software Foundation, Inc. This is free software with ABSOLUTELY NO WARRANTY. For details type ‘warranty’. 4 + 5 9 2 * 4 8

Each time you type an expression, bc comes back with the value of the expression. When you are done, type CTRL D to indicate end of input. bc will then exit back to the shell. Here we will look at how to get the value 2.0 ^ 3 computed in the shell. Simply write echo 2.0 ^ 3 | bc You can try this at the shell prompt, or put it in a shell script. Basically, the standard output of the echo command is piped into the standard input of bc. You can mix pipes with command expansion. For example, if you want to know the number of words in the string output by the date command, just write date | wc -w or echo ‘date‘ |wc -w or echo $(date) | wc -w

12. Intro to Conditional Statements

The Shell supports several types of looping and decision statements. Let us start with a simple script that uses a loop to print 5 numbers. Copy the following program into a script, say whilescript.sh, make the script executable, and run it. Make sure you have the blank spaces after the [ and before the ] in the condition for the while loop. Also, follow the exact line structure. i=1 while [ $i -le 5 ] do echo $i i=$((i+1)) done 8 GODFREY C. MUGANDA

You should get

1 2 3 4 5 You get the same effect with i=1 while [ $i -le 5 ] do echo $i i=$[i + 1] done If you change the script to i=1 while [ $i -le 5 ] do echo -n $i " " i=$[i + 1] done You get

1 2 3 4 5

13. Line Structure in Shell programs

The shell is line oriented: it uses end of line to recognize the end of a command. If you want to put more than one command on the same line, you have to separate them with a semicolon. In the following modification of the whilescript.sh, we have moved the do command onto the same line as the while command. i=1 while [ $i -le 5 ]; do echo -n $i " " i=$[i + 1] done

14. Testing boolean expressions

While and if statements use command called test to test “boolean conditions”. Here is an excerpt of the documentation for test from the man pages.

NAME test - check file types and compare values

SYNOPSIS test EXPRESSION test SHELL PROGRAMMING 9

[ EXPRESSION ] [] We note from this that the test command serves two purposes (1) check file types (2) compare values We learn further that there you can use it with two different but equivalent syntaxes: test EXPRESSION and

[ EXPRESSION ] In using test to compare values. In the second form, not the presence of blank spaces surrounding the expression being tested. Those spaces are significant. To demonstrate the first form, here is a modification of whilescript.sh that prints the numbers 10 through 1, separated by spaces, on a single line. i=10 while test $i -gt 0 ; do echo -n $i " " i=$[i-1] done The results are

10 9 8 7 6 5 4 3 2 1

15. Using test to compare values

To help us use test to compare values, we have the following excerpt from the man pages.

INTEGER1 -eq INTEGER2 INTEGER1 is equal to INTEGER2 INTEGER1 -ge INTEGER2 INTEGER1 is greater than or equal to INTEGER2 INTEGER1 -gt INTEGER2 INTEGER1 is greater than INTEGER2 INTEGER1 -le INTEGER2 INTEGER1 is less than or equal to INTEGER2 INTEGER1 -lt INTEGER2 INTEGER1 is less than INTEGER2 INTEGER1 -ne INTEGER2 INTEGER1 is not equal to INTEGER2 Note that the values being compared must be integers. And, to test string values:

-n STRING the length of STRING is nonzero STRING equivalent to -n STRING -z STRING 10 GODFREY C. MUGANDA

the length of STRING is zero STRING1 = STRING2 the strings are equal STRING1 != STRING2 the strings are not equal

So a single string can be tested to see if its length is zero or nonzero, and two strings can be compared for equality or inequality.

16. Using test to check file types

Much of the time spent working with Shell involves working with files, so it is not surprising that one of the uses of test is to check file types. Here is the relevant information from the man pages. There is more than this in the man pages: this is to give an idea of what is available.

FILE1 -nt FILE2 FILE1 is newer (modification date) than FILE2 FILE1 -ot FILE2 FILE1 is older than FILE2 -b FILE FILE exists and is block special -c FILE FILE exists and is character special -d FILE FILE exists and is a directory -e FILE FILE exists -f FILE FILE exists and is a regular file

Here are some more testing options on files.

-L FILE FILE exists and is a symbolic link (same as -h) -O FILE FILE exists and is owned by the effective user ID -p FILE FILE exists and is a named pipe -r FILE FILE exists and read permission is granted -s FILE FILE exists and has a size greater than zero -S FILE FILE exists and is a socket -t FD file descriptor FD is opened on a terminal -u FILE FILE exists and its set-user-ID bit is set -w FILE FILE exists and write permission is granted -x FILE FILE exists and execute (or search) permission is granted SHELL PROGRAMMING 11

You might be wondering how a file can be a socket. One of the address families (also called protocol family) supported by the socket interface is the Unix domain protocol family. Using Unix domain sockets, two processed can use the socket interface to communicate with each other if they are both on the same machine. This is very much like using named pipes or fifos.

17. Using the boolean Operators with test

The simple tests coverd above can be combined with the equivalents of boolean operators.

( EXPRESSION ) EXPRESSION is true ! EXPRESSION EXPRESSION is false EXPRESSION1 -a EXPRESSION2 both EXPRESSION1 and EXPRESSION2 are true EXPRESSION1 -o EXPRESSION2 either EXPRESSION1 or EXPRESSION2 is true Apparently you can also use the more traditional && and || to indicate in place of -a and -o.

18. Structure of While Loop

The general structure of the while loop is while test-expression do statements done

19. If statement

The general structure of an if statement is if test-expression then statements fi Naturally, there is the double alternative variation if test-expression then statements else statements fi Generally, the then and else keywords have to be on a line by themselves, but you can use the semicolon before them to place them on a preceding line. The following script prints all positive integers less or equal to 20 that are either multiples of 5 or multiples of 3. 12 GODFREY C. MUGANDA i=20 while [ $i -gt 0 ] ; do if [ ‘expr $i % 3‘ -eq 0 -o ‘expr $i % 5‘ -eq 0 ]; then echo -n $i " " fi i=$[i-1] done The output is

20 18 15 12 10 9 6 5 3 The if statement is actually more general than what we have shown, but you can google the subject and learn more on your own.

20. The Until loop

This loop is similar to the while loop, except it reverses the sense of the test. until test-expression do statements done This will execute the enclosed statements as long as the test expression remains false. This is in contrast to the while statement, which executes the enclosed statements as long as the enclosed test expression remains true.

21. The For statement

The for statement has the form for var in argument-list do statements done It executes the statements in its body for each argument in its argument list. The following script for name in larry curly moe do echo $name done prints the output larry curly moe The following script for name in larry curly moe do echo $name " " $(expr length $name) SHELL PROGRAMMING 13 done prints each name in the list together with its length: larry 5 curly 5 moe 3 The last script, is of course, equivalent to for name in larry curly moe do echo $name " " ‘expr length $name‘ done Let us think about this a little, to enforce what we already know. First, you need the expr command to compute the length of the name: expr length $name You cannot, however, write echo $name " " expr length $name and get what you want, because the expr command sends its result to standard output. What we want to do is to use command substitution to capture the standard output of the expr command and use that output as a string argument to echo. Thus what we need is echo $name " " ‘expr length $name‘

22. The continue and break statements

These statements work in shell programs the same as they work in C and C++.

23. Command Line Arguments

You can write your scripts to take arguments on the command line. In C, you are accustomed to using arguments to the main function to access command line arguments as an array of strings. int main(int argc, char *argv[]} In shell, you use the expressions

$# $@ to denote the number of command line arguments, and the list of command line argument, respectively. Edit a script called cmdlinescript.sh, with the following contents: echo "The number of command line args is $#" echo "The list of command line args is $@" Now, if you run the command

./cmdlinescript.sh larry curly moe 14 GODFREY C. MUGANDA

You should see the output

The number of command line args is 3 The list of command line args is larry curly moe Note that the number of command line arguments does not include the name of the script itself. The individual arguments on the command line can be accessed by position, using the expressions

$0, $1, $2, ..., $9 The $0 is the name of the file that contains the script, and $1 is the name of the first argument to the script. Some shells may support a higher limit for command line arguments.

24. The Shift Command

The shift command works in a shell script. Each execution of shift shifts the list of command line arguments down by 1. The argument in position 1 drops off, all the other arguments shift down by one position, and the number of command line arguments decreases by 1. The following modification of the the previous script displays each command line argument in turn, together with its length. echo "The number of command line args is $#" echo "The list of command line args is $@" while [ $# -ne 0 ]; do echo $1 " " ‘expr length $1‘ shift done The results are

./cmdlinescript.sh larry curly moe The number of command line args is 3 The list of command line args is larry curly moe larry 5 curly 5 moe 3

25. Learning More

There is more to shell programming than we have covered in this short tutorial. Shell scripts support single dimesional arrays. Shell scripts can run any program, written in shell script, or some other language such as C. The shell script can pass these programs command line arguments, form pipelines of such processes, and capture their standard output for use in the shell script. A shell script can use the export command to add a variable to its environment, allowing such variables to be inherited by child processes spawned by the shell. SHELL PROGRAMMING 15

Having covered this material, you should be able to go online and find whatever other information you might need to write your shell script.