INTRODUCTION TO LINUX PT2 Oliver Standford STOR-I, Lancaster University Introduction to Linux | Oliver Standford SHELL SCRIPTING Introduction to Linux | Oliver Standford Intro • SHELL • A program that interprets commands. • An intermediary between the user and system. • A programming language. • SCRIPTS • A Script is a shell program. • Easy to use tool for building applications by ‘gluing together’ different programs. • Well suited for repetitive tasks. Introduction to Linux | Oliver Standford Hello World #!/bin/bash echo "Hello World!" • #! /bin/bash - comment explaining which program to use. • echo “Hello World!” - making use of the echo program. • #!/bin/bash • tar -zcvf my-backup.tgz /home/guest00 • A very simple backup script • Create a tar backup of a user’s home directory. • NOT intended to be used. • Can use any text editor - nano, emacs, vim, gedit Introduction to Linux | Oliver Standford Executing a script • Execute your script. • bash your-script-name.sh • sh your-script-name.sh • ./your-script-name.sh – needs to be executable. • Set Executable permissions. • chmod +x your-script-name • chmod 755 your script name (owner - R,W,X, Group/Other - R,X) • Example. • chmod +x hello.sh • ./hello.sh Introduction to Linux | Oliver Standford Command Substitution • Command substation is to run a shell command store its output to a variable or display it back using the echo command. #!/bin/bash OF=my-backup-`date +%Y_%m_%d`.tgz tar -zcvf $OF /home/guest00 • A better backup script. • What is the function of: `date +%Y_%m_%d` • Why is it useful in this context? • For command substitution we use ` (backticks) not “” (quote). • Create a similar script. Introduction to Linux | Oliver Standford Variables • Scripts can use variables as in any programming language. • There are no data types. • Can contain a number, character or a string of characters. #!/bin/bash STR=“Hello!”; here=`pwd`; echo “$STR you are at $here” • What will the output of this script be? Introduction to Linux | Oliver Standford Passing Arguments bash myscript.sh arg1 arg2 arg3 echo There are $# arguments to $0: $* echo first argument: $1 echo second argument: $2 echo third argument: $3 • Arguments typed after the script name on a shell command line are passed to the script as variables. • $* - returns a single string containing all arguments. • $1,2 etc - referrers to a numbered argument. • $0 - referrers to the script name itself. • $# - referrers to the number of arguments in the command line. • What is the output of the above script? Introduction to Linux | Oliver Standford Setting variables interactively • Using the Read command. • Reads a line from standard input, and • Assigns each word to the corresponding shell variable. #!/bin/bash echo "What is your name?"; read MYNAME; echo "Hello $MYNAME - hope you’re well." Introduction to Linux | Oliver Standford Backup Script Revisited #!/bin/bash echo "Extension of files to back-up"; read extension; OF=my-backup-`date +%Y_%m_%d`.tgz tar -zcvf $OF `find ~/ -name "*.${extension}“ -print` • A more specialised backup script. • READ - asks the user what extension to backup. • FIND - searches for files matching that extension. Introduction to Linux | Oliver Standford Exercise • Create a backup script that has the following features. • When the script is run, the user will be prompted to specify the location of the files to be backed up. • The script will then ask the user where they want to save their backup. • The script will prompt the user to give the backup a name. • The script name will be appended with the date and time of the backup. • Modify the script so that the folder to be backed up is passed by an argument instead of prompting the user for the location. Introduction to Linux | Oliver Standford Conditionals • Decide whether or not to perform an action. • Decision taken by evaluating an expression. #!/bin/bash T1=“foo” T2="bar” if [ "$T1" = "$T2" ]; then echo expression evaluated as true else echo expression evaluated as false fi Introduction to Linux | Oliver Standford File test operators • -e file exists • -f file is a regular file (not a directory etc) • -s file is not zero size • -d file is a directory • -L file is a symbolic link • -r file has read permission • -w file has write permission • -x file has execute permission • f1 -nt f2 f1is newer than f2 • f1 -ot f2 f1 is older than f2 Introduction to Linux | Oliver Standford Other comparison operators Integer Comparison String Comparison -eq is equal to = is equal to -ne is not equal to != is not equal to -gt is greater than -z is null, zero length -ge is greater than or -n is not null equal to. -lt is less than -le is less than or equal to. Introduction to Linux | Oliver Standford For do Loops • Loops are used to perform actions over and over until a condition is met or until all data is has been processed. • The For Do loop iterates through a list, executing the body of the loop for each value in the list. for i in 0 1 2 3 4 5 ; do echo The number is $i done • The for loop assigns the value of each list item to the variable NUMBER. Introduction to Linux | Oliver Standford For loops and command substitution • For loops are meant to be used with Command Substitution. for FILE in `ls` ; do echo The file name is $FILE done • Display the contents of a directory. for i in `seq 1 10` ; do echo “i = $i”; done • Print out a list of numbers one to ten using seq command. Introduction to Linux | Oliver Standford While Loops • A while loop executes while a condition is true. • A simple while loop. #!/bin/bash i=0; while [ $i -lt 10 ]; do echo "i= $i"; let i=$i+1; done • While i is less than 10, loop through the body, adding 1 to i each time. Introduction to Linux | Oliver Standford Shell Arithmetic • Integer arithmetic can be performed using the built-in let, expr or bc commands. BIGNUM=1024 - Assigns number to variable. let RESULT=$BIGNUM/16 - divides 1024 by 16. RESULT=`expr $BIGNUM / 16` - divides 1024 by 16. RESULT=`echo “$BIGNUM /16” | bc` - divides 1024 by 16. let foo=$RANDOM%10; echo $foo - generates a random number between 1 and 10. Introduction to Linux | Oliver Standford Exercises • Write a simple shell script that takes a path of a directory as a command line argument and list all files and folders inside the given directory. • Write a script that asks for the user's age. If it is equal to or higher than 18, print a message saying that this user is allowed to drink alcohol. If the user's age is below 18, print a message telling the user how many years he or she has to wait before legally being allowed to drink. • Write a script that takes exactly one argument, a directory name. If the number of arguments is more or less than one, print a usage message. If the argument is not a directory, print another message. For the given directory, print the ten biggest files. (tip, you will need to use the following commands du -a, sort, head.) Introduction to Linux | Oliver Standford TOOLS & MISC Introduction to Linux | Oliver Standford grep • grep: Global Regular Expression Printer • Searches for a text pattern in files. • Prints the lines that match the pattern. grep [options] [reg expr] [filename] • grep Syntax grep oliver /etc/passwd • Searches for oliver in the system password file. Introduction to Linux | Oliver Standford Regular Expressions • A formula for matching strings that follow a pattern. • Supported by many utilities including grep, vim, sed, awk. Expression Description . Matches any single character. $ Matches the end of the line. ^ Matches the beginning of the line. * Zero or more occurrences of previous character. \ Treat the following character as an ordinary character. [] Any of the characters between the brackets. [C1 - C2] Range of the characters between C1 and C2. \<\> Matches the beginning (\<) or end (\>) of a word. \(\) Saves the characters matched by the expression into a holding area. Introduction to Linux | Oliver Standford Examples • Find 3 letter pattern with first letter r and last letter n. [oliver@ubuntu ~]$ grep r.n myfile.txt • Match lines that end with the word ‘finish’. [oliver@ubuntu ~]$ grep “finish$” myfile.txt • Match lines that begin with \begin (Latex). [oliver@ubuntu ~]$ grep “^\\begin” filename.tex Introduction to Linux | Oliver Standford Sed - Stream Editor • A pattern matching engine that can perform manipulations on lines of text. • Works on a line by line basis. • sed is a batch editor. • Syntax: • sed [options] [sed-function] [filename] [oliver@ubuntu ~]$ sed ‘s/a/b/g’ myfile.txt > newfile.txt • Change a to b in myfile.txt and direct results to newfile.txt. • s - means substitute. • g- make a global change. Introduction to Linux | Oliver Standford sed - Text Deletion • To delete use the d function. [oliver@ubuntu ~]$ sed ‘3,7d’ myfile.txt • Delete lines 3 to 7. [oliver@ubuntu ~]$ sed ‘/dogs/d’ myfile.txt • This will delete any lines that have the word ‘dogs’ in. Introduction to Linux | Oliver Standford Exercises • Use the following command to download grepdata.txt to your home directory. • wget www.lancs.ac.uk/staff/standfor/Linux/grepdata.txt • Use grep to print all lines that contain a phone number with an extension (the letter x or X followed by four digits). • Print all lines that begin with three digits followed by a blank. • Use sed to replace all ‘High School’ with ‘Secondary school’. • Delete any email addresses that are from @example.com Introduction to Linux | Oliver Standford Ssh • Linux allows remote users interactive facilities. • ssh runs an interactive terminal on a remote machine. [oli@ubuntu ~]$ ssh [email protected] • You are presented with a password prompt.