CHAPTER 9 –SIMPLE FILTERS
Updated by Eng. Rana AlQurem What is a Filter?
Filters are commands which accept data from the standard input, manipulate it, and produce standard output.
Filters can work with pipes and redirections to modify where they get their data and where they send it. They are all written to use generic character streams. more - Paging Output
In unix, the output of a command is simply displayed to the screen no matter how long it is or if it scrolls.
The more command allows you to receve the output one screen at a time.
More may be used via a pipe or it can take a file name as an argument.
more file1.txt more < file1.txt cat file1.txt | more Pr-Paginating Files
pr [options] [file1…fileN]
pr prepares a file for printing. It puts 5 lines of margin at the top and bottom and places the last modified date in the header along with the file name and page number.
Options
-k produce output in k columns -l n set length of page to n lines -t remove header and footer +k start printing from page k
Pr can be piped into the lp command or redirected to a file. wc - Counting Words
wc [options][file1…fileN]
By default wc prints out a line for each file giving the number of lines, number of words, number of characters, and file name in order.
The wc command is a filter and can have input redirected or piped to it.
Options -l count lines -w count words -c count characters cmp-Comparing Files
cmp [options] file1 file2
cmp compares two files one byte at a time. Note that a character is one byte in size.
By default cmp simple prints out the character and line where the first difference occurs.
Using the -l option, the cmp command prints out all the differences but only gives the character number where the difference occurs and the octal representation of the 2 characters that differ. diff - Comparing Files
diff [options] file1 file2
The diff command use special symbols and instructions to indicate the changes that are required to make two files identical.
Tell you how to change the first file to look like the second.
Comm -Print What is in Common
comm [options] file1 file2
Takes two sorted files in order and prints out differences using three columns.
It prints out in the first column if the word only appears in file1 It prints out in the second column if the word only appears in file 2 It prints out in the third column if the word appears in both head and tail
head [options] filename – displays the top of the file Options -n k, print the first k number of lines
tail [options] filename - displays the bottom of the file Options -n k, print the last k number of lines cut-Slitting a File Vertically
The head and tail commands let you split a file horizontally, to do it vertically you need cut.
Cut uses the following syntax: cut [-f | -c] filename.
The c option cuts the file using character columns. cut –c 1-4 fname.txt will cut out the 1st four characters cut -c3,7-9,12-15 fname.txt will cut out characters 3,7,8,9,12,13,14,15
The f option looks for delimited fields instead of columns. The default delimiter is tab. To change the delimiter use -d option. cut –d”:” -f1,3 /etc/passwd gets the 1st and 3rd fields of the passwd paste - Pasting Files Vertically
The paste [-d ] fname1 fname2 … command will vertically join the files.
Like the cut command, the paste command will take -d option to specify the field delimiter it should place in between fields. It will use tab if not delimiter is given.
The –s option joins lines as fields in one line.
Example: paste –s –d “::\n” filename Sorting Files
sort[options][file-list] sorted lines in ASCII order in file-list Options: -tchar uses delimiter char to identify fields -k n sorts on the nth field -k m,n starts on mth field and ends on nth field -k m.n starts on nth column in mth field -u removes repeated lines -n sort numerically -r sort in reverse order -f case-insensitive sort -m list merge sorted files in list -c check if the file is sorted -o filename place output in filename Sorting Files (cont…)
$ sort students file will be sorted by all chars from left to right.
$sort –k 2 students file will be sorted starting with the second field of the file
$sort –k 4 -r students file will be sorted starting with the fourth field, in reverse order Locate Repeated Lines
uniq [options] [input-file] [output-file]
$ cat sample This is a test file for the uniq command. It contains some repeated and some nonrepeated lines. Some of the repeated lines are consecutive, like this. Some of the repeated lines are consecutive, like this. Some of the repeated lines are consecutive, like this. And, some are not consecutive, like the following. Some of the repeated lines are consecutive, like this. The above line, therefore, will not be considered a repeated line by the uniq command, but this will be considered repeated! line by the uniq command, but this will be considered repeated! $ uniq sample This is a test file for the uniq command. It contains some repeated and some nonrepeated lines. Some of the repeated lines are consecutive, like this. And, some are not consecutive, like the following. Some of the repeated lines are consecutive, like this. The above line, therefore, will not be considered a repeated line by the uniq command, but this will be considered repeated!
Locate Repeated Lines
$ uniq -c sample 1 This is a test file for the uniq command. 1 It contains some repeated and some nonrepeated lines. 3 Some of the repeated lines are consecutive, like this. 1 And, some are not consecutive, like the following. 1 Some of the repeated lines are consecutive, like this. 1 The above line, therefore, will not be considered a repeated 2 line by the uniq command, but this will be considered repeated! $ uniq -d sample Some of the repeated lines are consecutive, like this. line by the uniq command, but this will be considered repeated! $ uniq -d sample out $ cat out Some of the repeated lines are consecutive, like this. line by the uniq command, but this will be considered repeated! tr - Translating Characters
The tr command takes the unusual format tr options expr1 expr2 standard input
tr command replaces the characters in expression 1 with the corresponding ones in expression 2 for the input provided. Note, number of characters in expr1 and expr2 must match.
A range of characters can be specified using - .For example, [a-z]. Options d, s, c are user with tr. Examples head -5 file.txt | tr ‘a’ ‘f’ replaces a by f in first 5 lines in file.txt tr –d ‘:\’ < file.txt delete : and \ from file.tx