INTRODUCTION TO PT2

Oliver Standford STOR-I, Lancaster University Introduction to Linux | Oliver Standford

SHELL SCRIPTING Introduction to Linux | Oliver Standford

Intro

• SHELL • A program that interprets commands. • An intermediary between the user and system. • A .

• SCRIPTS • A Script is a shell program. • Easy to use tool for building applications by ‘gluing together’ different programs. • Well suited for repetitive tasks. Introduction to Linux | Oliver Standford

Hello World

#!/bin/ echo "Hello World!"

• #! /bin/bash - comment explaining which program to use. • echo “Hello World!” - making use of the echo program.

• #!/bin/bash • tar -zcvf my-backup.tgz /home/guest00

• A very simple backup script • Create a tar backup of a user’s home directory. • NOT intended to be used. • Can use any text editor - nano, emacs, vim, gedit Introduction to Linux | Oliver Standford

Executing a script

• Execute your script. • bash your-script-name.sh • sh your-script-name.sh • ./your-script-name.sh – needs to be executable.

• Set Executable permissions. • chmod +x your-script-name • chmod 755 your script name (owner - R,W,X, Group/Other - R,X)

• Example. • chmod +x hello.sh • ./hello.sh Introduction to Linux | Oliver Standford

Command Substitution

• Command substation is to run a shell command store its output to a variable or display it back using the echo command. #!/bin/bash OF=my-backup-`date +%Y_%m_%d`.tgz tar -zcvf $OF /home/guest00

• A better backup script. • What is the function of: `date +%Y_%m_%d` • Why is it useful in this context? • For command substitution we use ` (backticks) not “” (quote). • Create a similar script. Introduction to Linux | Oliver Standford

Variables

• Scripts can use variables as in any programming language. • There are no data types. • Can contain a number, character or a string of characters.

#!/bin/bash STR=“Hello!”; here=`pwd`; echo “$STR you are at $here”

• What will the output of this script be? Introduction to Linux | Oliver Standford Passing Arguments bash myscript.sh arg1 arg2 arg3 echo There are $# arguments to $0: $* echo first argument: $1 echo second argument: $2 echo third argument: $3

• Arguments typed after the script name on a shell command line are passed to the script as variables. • $* - returns a single string containing all arguments. • $1,2 etc - referrers to a numbered argument. • $0 - referrers to the script name itself. • $# - referrers to the number of arguments in the command line. • What is the output of the above script? Introduction to Linux | Oliver Standford

Setting variables interactively

• Using the Read command. • Reads a line from standard input, and • Assigns each word to the corresponding shell variable.

#!/bin/bash echo "What is your name?"; read MYNAME; echo "Hello $MYNAME - hope you’re well." Introduction to Linux | Oliver Standford

Backup Script Revisited #!/bin/bash echo "Extension of files to back-up"; read extension; OF=my-backup-`date +%Y_%m_%d`.tgz tar -zcvf $OF `find ~/ -name "*.${extension}“ -print`

• A more specialised backup script. • READ - asks the user what extension to backup. • FIND - searches for files matching that extension. Introduction to Linux | Oliver Standford

Exercise

• Create a backup script that has the following features. • When the script is run, the user will be prompted to specify the location of the files to be backed up. • The script will then ask the user where they want to save their backup. • The script will prompt the user to give the backup a name. • The script name will be appended with the date and time of the backup. • Modify the script so that the folder to be backed up is passed by an argument instead of prompting the user for the location. Introduction to Linux | Oliver Standford

Conditionals

• Decide whether or not to perform an action. • Decision taken by evaluating an expression. #!/bin/bash T1=“foo” T2="bar” if [ "$T1" = "$T2" ]; then echo expression evaluated as true else echo expression evaluated as false fi Introduction to Linux | Oliver Standford

File test operators

• -e file exists • -f file is a regular file (not a directory etc) • -s file is not zero size • -d file is a directory • -L file is a symbolic link • -r file has read permission • -w file has write permission • -x file has execute permission • f1 -nt f2 f1is newer than f2 • f1 -ot f2 f1 is older than f2 Introduction to Linux | Oliver Standford

Other comparison operators

Integer Comparison String Comparison -eq is equal to = is equal to -ne is not equal to != is not equal to -gt is greater than -z is null, zero length -ge is greater than or -n is not null equal to. -lt is less than -le is less than or equal to. Introduction to Linux | Oliver Standford

For do Loops

• Loops are used to perform actions over and over until a condition is met or until all data is has been processed. • The For Do loop iterates through a list, executing the body of the loop for each value in the list. for i in 0 1 2 3 4 5 ; do echo The number is $i done

• The for loop assigns the value of each list item to the variable NUMBER. Introduction to Linux | Oliver Standford

For loops and command substitution

• For loops are meant to be used with Command Substitution. for FILE in `ls` ; do echo The file name is $FILE done • Display the contents of a directory. for i in `seq 1 10` ; do echo “i = $i”; done

• Print out a list of numbers one to ten using seq command. Introduction to Linux | Oliver Standford

While Loops

• A while loop executes while a condition is true. • A simple while loop.

#!/bin/bash i=0; while [ $i -lt 10 ]; do echo "i= $i"; let i=$i+1; done

• While i is less than 10, loop through the body, adding 1 to i each time. Introduction to Linux | Oliver Standford

Shell Arithmetic

• Integer arithmetic can be performed using the built-in let, expr or bc commands. BIGNUM=1024 - Assigns number to variable. let RESULT=$BIGNUM/16 - divides 1024 by 16.

RESULT=`expr $BIGNUM / 16` - divides 1024 by 16.

RESULT=`echo “$BIGNUM /16” | bc` - divides 1024 by 16. let foo=$RANDOM%10; echo $foo - generates a random number between 1 and 10. Introduction to Linux | Oliver Standford Exercises • Write a simple shell script that takes a path of a directory as a command line argument and list all files and folders inside the given directory.

• Write a script that asks for the user's age. If it is equal to or higher than 18, print a message saying that this user is allowed to drink alcohol. If the user's age is below 18, print a message telling the user how many years he or she has to wait before legally being allowed to drink.

• Write a script that takes exactly one argument, a directory name. If the number of arguments is more or less than one, print a usage message. If the argument is not a directory, print another message. For the given directory, print the ten biggest files. (tip, you will need to use the following commands du -a, sort, head.) Introduction to Linux | Oliver Standford

TOOLS & MISC Introduction to Linux | Oliver Standford • grep: Global Regular Expression Printer • Searches for a text pattern in files. • Prints the lines that match the pattern. grep [options] [reg expr] [filename]

• grep Syntax grep oliver /etc/passwd

• Searches for oliver in the system password file. Introduction to Linux | Oliver Standford

Regular Expressions

• A formula for matching strings that follow a pattern. • Supported by many utilities including grep, vim, sed, awk. Expression Description . Matches any single character. $ Matches the end of the line. ^ Matches the beginning of the line. * Zero or more occurrences of previous character. \ Treat the following character as an ordinary character. [] Any of the characters between the brackets. [C1 - C2] Range of the characters between C1 and C2. \<\> Matches the beginning (\<) or end (\>) of a word. \(\) Saves the characters matched by the expression into a holding area. Introduction to Linux | Oliver Standford

Examples

• Find 3 letter pattern with first letter r and last letter n. [oliver@ubuntu ~]$ grep r.n myfile.txt

• Match lines that end with the word ‘finish’. [oliver@ubuntu ~]$ grep “finish$” myfile.txt

• Match lines that begin with \begin (Latex). [oliver@ubuntu ~]$ grep “^\\begin” filename.tex Introduction to Linux | Oliver Standford

Sed - Stream Editor

• A pattern matching engine that can perform manipulations on lines of text. • Works on a line by line basis. • sed is a batch editor. • Syntax: • sed [options] [sed-function] [filename]

[oliver@ubuntu ~]$ sed ‘s/a/b/g’ myfile.txt > newfile.txt

• Change a to b in myfile.txt and direct results to newfile.txt. • s - means substitute. • g- a global change. Introduction to Linux | Oliver Standford

sed - Text Deletion

• To delete use the d function.

[oliver@ubuntu ~]$ sed ‘3,7d’ myfile.txt

• Delete lines 3 to 7.

[oliver@ubuntu ~]$ sed ‘/dogs/d’ myfile.txt

• This will delete any lines that have the word ‘dogs’ in. Introduction to Linux | Oliver Standford Exercises • Use the following command to download grepdata.txt to your home directory. • wget www.lancs.ac.uk/staff/standfor/Linux/grepdata.txt • Use grep to print all lines that contain a phone number with an extension (the letter x or X followed by four digits). • Print all lines that begin with three digits followed by a blank. • Use sed to replace all ‘High School’ with ‘Secondary school’. • Delete any email addresses that are from @example.com Introduction to Linux | Oliver Standford

Ssh

• Linux allows remote users interactive facilities. • ssh runs an interactive terminal on a remote machine.

[oli@ubuntu ~]$ ssh user@.lancs.ac.uk

• You are presented with a password prompt. • To connect through ssh to a remote machine, the ssh server must be installed and running on the remote machine. • You can also use an ssh client program on other operating systems to connect to a remote machine (PUTTY). Introduction to Linux | Oliver Standford Scp - secure copy

• Copy files and directories to a remote machine over ssh.

• Copy a single file to a remote host. • scp /home/user/a.txt [email protected]:filename • Copy all .tex files to a remote host. • scp textFiles/*.tex [email protected]:dir/ • Copy ~/data/ directory to remote host. • scp -r -p ~/data/ [email protected]:dir/ • Copy a single file from a remote host to your computer. • scp [email protected]:a.txt /home/user/

• -r - Recursively • -p - Preserve modification, access times, permissions. Introduction to Linux | Oliver Standford

rsync

• Synchronizes files and directories from one location to another while minimizing data transfer. • Advantages: • Speeds up file transfer by copying the only differences between two files. • Perform synchronization over network. • Preserves permissions, ownership, symbolic links etc.

• Rsync syntax: rsync [options] source destination Introduction to Linux | Oliver Standford

rsync backup

[oli@ubuntu ~]$ rsync -avh /home/user/hdir/ /media/disk/udir/

• Copies the content of hdir to udir. • -a - Archive mode. Includes Recursive, permissions, links. • -v - Verbose - show information about what rsync is doing. • -h - Output numbers in a human readable format. • -e - Use SSH to connect to remote destination [oli@ubuntu ~]$ rsync -av --exclude=‘.bak’ /home/user/hdir/ /media/disk/udir/ • --exclude - use this to exclude files from. • --delete - (Be careful)use to remove deleted files from backup.