Bash Shell Scripts

Bash Shell Scripting: Getting Started

SELF2015 Beau Sanders Greenville Technical College https://beausanders.org/self15

150609 SELF2015 • 1 About me

• Academic Program Director Computer Technology Department Greenville Technical College Greenville, South Carolina • Instructor for 13 years • RHCE, RHCSA, RHCT • Certified Red Hat Academy Instructor • Teach Linux Essentials, Administration, Network Applications, Security, and Scripting

150609 SELF2015 • 2 Assumptions

• You can handle the Linux command prompt • You understand the need to make administration easier and more manageable • You know a little programming • You know some Linux commands • You understand command structure in Linux

150609 SELF2015 • 3 Some Basic Linux Commands

• cat • less • ls • more • ps • chmod • pwd • chown • cp • chgrp • mv • useradd • rm • usermod • mkdir • logger • head • wget • tail • mail

150609 SELF2015 • 4 Linux Command Structure

150609 SELF2015 • 5 Bash Shell Scripting: Getting Started Overview of BASH

150609 SELF2015 • 6 The bash shell

• The shell is the most commonly used program in Linux • The shell is what you see when you log in or open a terminal, and is what you use to start most every command • The most commonly used shell in Linux is the bash shell • Bourne Again SHell

150609 SELF2015 • 7 Interactive Shells vs. Shell Scripts

• The bash shell is an interactive shell • The bash shell is also designed to be a powerful scripting language – Bash shell scripts are short programs written using the same syntax used on the command line – Shell scripts allow users to automate often repeated actions by combining a series of commands – Many of the features of the bash shell provide programming logic (such as branches and loops) for writing sophisticated scripts

150609 SELF2015 • 8 An Introduction to Shell Scripts

• A bash shell script is just a text file containing a list of ordinal Linux commands • The commands are sent through a specified program, called an interpreter, which runs each command in turn • The interpreter will be the bash shell (/bin/bash or /bin/sh). • Other interpreters allow you to use more powerful programming languages like Perl, Python and Ruby

150609 SELF2015 • 9 An Introduction to Shell Scripts

• The first line of your script must specify which interpreter to send instructions to – This is done with a special string called a "shebang" which looks like this: #! – The shebang is followed by the name of the interpreter for this script – For example, to use bash as your interpreter you would use #!/bin/sh or #!/bin/bash

150609 SELF2015 • 10 An Introduction to Shell Scripts

• Before you can run a script, you must enable the executable permission on it – otherwise, it's just a text file • A typical command to enable execute permissions on a script is: chmod 775 scriptname

150609 SELF2015 • 11 Default Path

• A fixed default set of directories in which Linux looks for commands • These directories are referred to collectively as your PATH • For security reasons, your PATH seldom includes the current directory

150609 SELF2015 • 12 Default Path

• To solve this problem you have two choices: – You can explicitly specify the location of the script by typing ~/foo.sh or ./foo.sh ( . always refers to the current directory) – You can place the script in a directory that is part of your PATH – Non-root users will need to create their personal bin: $mkdir ~/bin

150609 SELF2015 • 13 Bash Shell Scripting: Getting Started Variables and Redirecting Data

150609 SELF2015 • 14 Shell Variable Basics

• The bash shell allows users to set and reference shell variables • A shell variable is simply a named value that the shell remembers • Shell variables • Can be used in commands • Can be used in shell scripts • Can also be referenced by programs as configuration options

150609 SELF2015 • 15 Shell Variable Basics

• There are two types of shell variables: local variables and environmental variables • A local variable exists only within the shell in which it is created • Environmental variables are inherited by child shells such as when a graphical terminal is launched after logging in • Setting local variables is quite simple

150609 SELF2015 • 16 Shell Variable Basics

• Make sure that you do not put any spaces on either side of the = sign • The shell will "hang on" to this association for as long as the shell exists or until it is unset

150609 SELF2015 • 17 Environmental Variables

• Environmental variables are a part of the process stored in the kernel, like the process id, user id, and current working directory are part of the process • Whenever a process starts another process, environmental variables are inherited by the child process • This allows users to use the bash shell to create or modify an environmental variable, and then all commands started by the shell will inherit that variable

150609 SELF2015 • 18 Environmental Variables

• How do we create environmental variables within the bash shell? – First, a shell variable is created, and then the shell variable is "promoted" to an environmental variable using the export command – The variable will then be exported to any future child processes

150609 SELF2015 • 19 Standard In and Standard Out

• Command line programs (terminal based programs) read information from one source, and write information to one destination • The source programs read from is referred to as Standard In (stdin), and is usually connected to a terminal's keyboard • The destination programs write to is referred to as Standard Out (stdout), and is usually connected to a terminal's display • When using the bash shell, stdout can be redirected using > or >>, and stdin can be redirected using <

150609 SELF2015 • 20 Redirecting stdout Writing Output to a File

• The bash shell uses > to redirect a process's stdout stream to a file •

150609 SELF2015 • 21 Appending Output to a File

• To append a command's output to a file, rather than clobbering it, bash uses >> • Often >> is used to create a log file in a script

150609 SELF2015 • 22 Redirecting stdin

• Bash uses < to read input from somewhere other than the keyboard • Often used to read in a file as input to script

150609 SELF2015 • 23 Bash Shell Scripting: Getting Started Commands and Tools

150609 SELF2015 • 24 echo

• The echo command takes whatever text is typed as part of the command and echoes it to stdout • The echo command, together with redirection, can be used to create text files

150609 SELF2015 • 25 echo

• Usage: echo [OPTIONS] [STRING...] • Echoes STRING(s) to standard output

150609 SELF2015 • 26 grep

• The name grep stands for general regular expression parser • A regular expression is simply a way of describing a pattern, or template, to match some sequence of characters

150609 SELF2015 • 27 grep

150609 SELF2015 • 28 grep

• The pattern argument supplies the template characters for which grep is to search • The pattern is expected to be a single argument – If the pattern contains any spaces, or other characters special to the shell, you must enclose the pattern in quotes to prevent the shell from expanding or word splitting it

150609 SELF2015 • 29 grep

• By default, grep shows only the lines matching the search pattern • Sometimes you are interested in the lines that do NOT match the pattern • The -v command line switch inverts grep's operation •

150609 SELF2015 • 30 Pipes

• When two commands are joined by a pipe, the stdout stream of the first process is tied directly to the stdin sequence of the second process • In order to create a pipe using bash, the two commands are joined with a vertical bar | • All processes that are joined in a pipe are referred to as a process group • Many commands in Unix are designed to operate as a filter, reading input from stdin and sending output to stdout

150609 SELF2015 • 31 Filtering output using grep

• The traditional Unix grep command is commonly used in pipes to reduce data to only useful data; sometimes called a filter • The grep command is used to search for and extract lines which contain a specified string of text

150609 SELF2015 • 32 Filtering output using grep

• The first argument to the grep command is the string of text to be searched for • Any remaining arguments are files to be searched for the text • If the grep command is called with only one argument, it looks to Standard In as its source of data on which to operate

150609 SELF2015 • 33 Extracting and Assembling Text: cut and paste

• The cut command extracts texts from text files, based on columns specified by bytes, characters, or fields • The paste command merges two text files line by line

150609 SELF2015 • 34 The cut Command - Extracting Text with cut

• The cut command extracts columns of text from a text file or data stream • The cut command interprets any command line arguments as names of files on which to operate, or operates on the standard in stream if none are provided • The cut command can extract by bytes, characters, or fields

150609 SELF2015 • 35 The cut Command - Extracting Text with cut

• The list arguments are actually a comma-separated list of ranges • Each range can take one of the following forms:

150609 SELF2015 • 36 Extracting text by Character Position with cut -c

• With the -c command line switch, the list specifies a character's position in a line of text • The first line character is character number 1

150609 SELF2015 • 37 Extracting Fields of Text with cut -f

• The cut command can also be used to extract text that is structured not by character position, but by some delimiter character, such as a TAB or :

150609 SELF2015 • 38 The paste Command

• The paste command is used to combine multiple files into a single output • The paste command expects a series of filenames as arguments • The paste command will read the first line from each file, join the contents of each line inserting a TAB character in between, and write the resulting single line to standard out • It then continues through each line from each files

150609 SELF2015 • 39 Return Values (also called Exit Status)

• Return values come in the form of integers which range from 0 to 255 • The Linux and Unix convention is that a program returns a 0 to indicate "success" when a command completes correctly • A non zero return value means the command failed

150609 SELF2015 • 40 Return Values

• The bash shell stores the return value of the previously executed command in a special variable named ? • The value of this variable can be examined with the echo $? command

150609 SELF2015 • 41 Running Multiple Commands Conditionally

• The bash shell uses && and || to join two commands conditionally • When commands are conditionally joined, the first will always execute • The second command may execute or not, depending on the return value of the first command

150609 SELF2015 • 42 Running Multiple Commands Conditionally

• By coupling two commands with && the second command will only run if the first command succeeded and the first command had a return value of 0 • This is similar to the "and" operation found in may programming languages

150609 SELF2015 • 43 Running Multiple Commands Conditionally

• Multiple commands can be combined with || • In this case bash will execute the second command only if the first command "fails" providing a non zero return value • This is similar to the "or" operator found in programming languages

150609 SELF2015 • 44 find

• The find command is used to search the filesystem for files that meet a specified criteria • A find command essentially consists of three parts: 1) a starting directory 2) a search criteria 3) an action

• find (staring directory) (criteria) (action)

150609 SELF2015 • 45 find

• Usually the find command is given criteria to refine its search, in the form of (non standard) command line switches

150609 SELF2015 • 46 find

150609 SELF2015 • 47 find

• Action • You can also specify what you would like done to files that meet the specified criteria • By default, if no criteria is specified, the file name is printed to standard out, one file per line

150609 SELF2015 • 48 find

• The -exec mechanism is powerful: rather than printing the names of matching files, commands can be run • The -exec mechanism is awkward, because the syntax for specifying the command to run is tricky • The command should be written after the -exec switch, using a literal {} as a placeholder for the file name • The command should be terminated with a ; (semi-colon)

150609 SELF2015 • 49 Positional Parameters

150609 SELF2015 • 50 Positional Parameters

• Can pass data to scripts • Use numbers 0 through 9 when referencing • $0 is the script itself • $1 through $9 represent data • Use the shift command to pass more than 9 variables to a script

150609 SELF2015 • 51 read

• Make scripts interactive with the user • Use the read command to prompt for user input: read variable-name

• The read command options: – -p prompt used to display a prompt – -s used to suppress characters entered – -t timeout used to expire the command once the timeout seconds value has been reached – -a array-name used to read data into an array

150609 SELF2015 • 52 read and readonly

• Protecting a Variable – You can protect a variable to ensure the contents do not change – Use the readonly command – Example:

read -p "Enter Zip Code: " Zip echo "Zip Code is: " $Zip readonly Zip read -p "Attempting to change Zip Code: " Zip echo "Zip Code is: " Zip

150609 SELF2015 • 53 crontab

150609 SELF2015 • 54 crontab

150609 SELF2015 • 55 Bash Shell Scripting: Getting Started Constructs

150609 SELF2015 • 56 Branches: if ... then ... [else ...] fi

• Bash syntax for branches • Branches are also called decision statements or decision structure • In programming, branches allow programs to choose between one of two (or more) alternate execution paths • The bash shell, like most programming languages, uses the word if to signify a branch

150609 SELF2015 • 57 Branches: if ... then ... [else ...] fi

150609 SELF2015 • 58 Branches: if ... then ... [else ...] fi

• Carriage returns are important (i.e., the if and then must occur on separate lines), but indentations are not

• What does bash expect as a condition? • Unlike most programming languages, bash has no internal syntax for making comparisons (such as $A == apple, or $B > 25) • Instead, bash focuses on what shells were designed to do: run commands

150609 SELF2015 • 59 Branches: if ... then ... [else ...] fi

• Any command can be used for the condition. • Sometimes condition is called list • The bash shell will execute the command, and examine its return value • If the command succeeds (returns a return value of 0), the first stanza of commands is executed • If the command fails (returns a return value not equal to zero), the second stanza of commands is executed (if any)

150609 SELF2015 • 60 150609 SELF2015 • 61 Branches: if ... then ... [else ...] fi

150609 SELF2015 • 62 test

• The test command will check for the existence of a file, and return the appropriate return code, without emitting any other messages • The test command was designed to be used as the conditional command in bash if ... then statements • The test command is designed to compare strings, integers, and file attributes • It never generates output, but instead communicates using its return value • The test command returns 0 if the expression it evaluates is true, and a non zero value otherwise

150609 SELF2015 • 63 test

150609 SELF2015 • 64 test

150609 SELF2015 • 65 Loops

• A loop is a condition that causes a specific set of statements to be repeated • Statements repeat until a condition terminates the loop • An iteration refers to a complete pass through the loop

150609 SELF2015 • 66 Why do you use loops for?

• Reading files line by line • Menu systems • Undefined tests where the programmer does not know how many tests to preform • To count items • There are many uses for a loop

150609 SELF2015 • 67 for Loop

• Allows you to perform a set number of iterations • Theoretical For statement: For variable goes from initial- value to ending-value Do Perform activity as long as variable not equal ending- value Increment or decrement variable End-For

150609 SELF2015 • 68 while Loop

• The theoretical while statement tests a condition for true or false • Theoretical While Logic: While true-condition Do Perform activity for true- condition End-While • If condition is true Do statements execute • If condition is false loop terminates and program flow continues with subsequent statement

150609 SELF2015 • 69 until Loop

• The theoretical until statement tests a condition for true or false • Theoretical Until Logic: Until false-condition Do Perform activity for false- condition End-While • If condition is false Do statements execute • If condition is true loop terminates

150609 SELF2015 • 70 Loops: for ... in ... do ... done

• Loops allow a series of commands to be repeated, usually with slight variations in each iteration • Iteration = cycle through the loop • Usually, these variations are implemented using a variable referred to as an iterator • For each iteration of the loop, the variable takes own a different value

150609 SELF2015 • 71 Loops: for ... in ... do ... done

• for ... in ... do ... done loops in bash use the following syntax:

• For each repetition of the loop, the variable iterator will evaluate to the individual words listed in the expression list

150609 SELF2015 • 72 Loops: for ... in ... do ... done

150609 SELF2015 • 73 Loops: for ... in ... do ... done

#!/bin/bash if test $1 -ge 10 then for (( i = 0; i <=$1; i++ )) do echo "As long as $1 is equal to or above 10, let's count: $i" done else echo $1 is not equal to or greater than 10. fi

150609 SELF2015 • 74 150609 SELF2015 • 75 How to read from a file?

while read myfile do echo $myfile done < inputfile

150609 SELF2015 • 76 How to read from a file?

150609 SELF2015 • 77 How to read from a file?

150609 SELF2015 • 78 Terminating a Loop

• Use the break command to end a loop prematurely • Example: ((count=1)) while true do echo $count ((count++)) if [[ $count –gt 3 ]] then break fi done

150609 SELF2015 • 79 Bash Shell Scripting: Getting Started Example Scripts

150609 SELF2015 • 80 SELF2015 • 81 150609 150609 SELF2015 • 82 SELF2015 • 83 150609 Bash Shell Scripting: Getting Started Questions

150609 SELF2015 • 84 Sources

• Red Hat Academy Many different courses over the years http://www.redhat.com/en/services/training/red-hat-academy

150609 SELF2015 • 85 Thank You!

• https://beausanders.org/self15 • Download slides • [email protected] • Resource list • [email protected] • My email

150609 SELF2015 • 86