Pipes & Filter – Unix Shell Programming

Home , Filter (software), Pipeline (software), Pipeline (Unix), Standard streams

Dr. T.Logeswari Associate professor, Department of Computer Science New Horizon College Kasturai Nagar Banglore

Abstract

In this research paper, the concept of Pipes and Filter is discussed and various aspects of filter commands also studied. Many Commands in Unix produce output on the terminal screen or can take input from the keyboard. Actually these commands have been written to accept input from standard input file and produce output on a standard output file. Generally the standard input files are set to keyboard and the output file to the terminal screen. These special files are streams of character seen as input and the output by the commands Keywords: cat, grep, sort, paste.

I. Introduction

A shell script is a computer program to design to run by the II.Standard Files unix shell, a command line interpreter. The various dialects of shell scripts are considered to be scripting languages[1][5]. There are three file representing three streams each associated The most influential Unix shells have been the Bourne shell with a default device. i.e standard Input, standard output, and the C shell. These shells have both been used as the Standard Error[3]. coding base and model for many derivative and work-alike shells with extended feature sets. The Bourne shell, sh, was written by Stephen Bourne at AT&T as the original Unix command line interpreter; it introduced the basic features common to all the Unix shells, including piping, command substitution, variables, control structures for condition-testing and looping and filename wildcarding. The language, including the use of a reversed keyword to mark the end of a block, was influenced by ALGOL 68.The C shell, csh, was written by Bill Joy while a graduate student at University of California, Berkeley. The language, including the control structures and the expression grammar, was modeled on C. Figure 1. standard streams for Input, Output & Error The most generic sense of the term shell means any program that users employ to type commands[2]. A shell hides the a. I/O Redirection information of the primary operating system and manages the procedural details of the operating system kernel line, which is Redirection is normally implemented by placing certain the lowest-level, or "inner-most" component of most operating character between commands. The input source is redirected systems. In Unix-like operating systems, users typically have using < (less than) operator. many choices of command-line interpreters for interactive For example $catpipeline

Program can run together such that one program reads the output from another without using an intermediate fie[4]

IJCRT1812312 International Journal of Creative Research Thoughts (IJCRT) www.ijcrt.org 653

c. Filters -c list : The list after -c specifies the character positions -f list : The list after -f is a list of fields separated by a Filters are UNIX commands that take their inputs from the delimiter character standard input file, process it and send it to the standard -d list : The character following -d is the field delimiter output. A filter is actually a program that can read directly from files whose names are provided as argument, process it Example: or filter it and send the results to the standard output, unless a $ cut -d ‘:’ -f 1,3 stdlist redirection or a pipeline symbol is used. It won’t process or 001 : 90 filter & send the results to the standard output unless a 002 : 68 pipeline (|) is used. Some commonly used filter commands are 011 : 80 sort, cut, paste, grep, tr, wc,uniq, etc[7][8]. 006 : 72 003 : 88 i)sort iii) paste The sort command is used to sort the information in a text file The paste command is used to join contects of two files in ascending or descending order. It is also used to merge vertically.The contexts of two files can be viewed side by sorted files. side by pasting them Sort options -m : merges two or more given sorted files into one sorted Example : output file $ cat empno $cat empname $cat phone -n : sorts according to numeric order 0100 Gautam 23236549 -r : sorts in reverse order 0102 Rithish 27496536 -o outfile : Places the sorted output in the file 0103 Sonam 28407862 -d : Sorts according to dictionary & ignores the punctuation -f : Ignores while sorting $ paste empnoempname phone >empdet -c : Can be used to check whether a file is sorted or not. $cat empdet Example: 0100 Gautam 23236549 Steps to create a file using cat command : 0102 Rithish 27496536 $cat > filename 0103 Sonam 28407862 $cat >stdlist Steps to display a file using cat command : iv) uniq $cat stdlist When duplicate entries are found in a file, they can be 001 :Deepali : 90 removed from the file by using the uniq command.This file 002 : Vidya: 68 takes only one sorted file as its argument 011 : Mohena : 80 006 : Robert :72 uniq options: 003 : Krithika : 88 -c : Precedes each output line with a count of the number of times the line occurred in the input $ sort stdlist -d : Selects only one copy of the duplicate entries 001 :Deepali :98 -u : Selects only the lines which are not repeated 002 : Vidya : 68 003 : Krithika : 88 Example 006 : Robert :72 $cat names 011 : Mohena : 80 Venky  Merging of two files can be performed by the command : Neelima $ sort -m -o namelist list1 list2 , where the files list1 & Rupali list 2 are merged into one namelist. Rupali ii)cut Vidya The cut command splits files vertically. This command can Diya be used to extract the required fields or columns from the Vidya file. Each field is separated by a ‘delimiter’ . $uniq names Example: Venky $ cut -c1 -3 stdlist Neelima 001 Rupali 002 Vidya 011 Diya 006 . $uniq -c names 003 1 Venky cut options : 1 Neelima

IJCRT1812312 International Journal of Creative Research Thoughts (IJCRT) www.ijcrt.org 654

2 Rupali It is used to display contents of a file , create a new file , 2 Vidya append data to an existing file & concatenate two or more 1 Diya files. . $uniq -d names cut options : Rupali -n : Displays each line output with tis line number at the Vidya beginning . $uniq -u names -b : Numbers the lines , like -n but omits the line numbers of Venky blank lines Neelima -u : The output is not buffered. The default is buffered output. Meenu Example : $cat empdet v) wc 0100 Sowmya 23246549 wc command is a filter used to count the number of lines, 0102 Nikhil 24496523 words & characters of one or more files. It takes one or more 0103 Deepthi 2840786665 filenames as its arguments. It gives the output in 4 columns, the first colum indicates the number of lines, second viii) grep indicates the number of words, third indicates the number of The grep command searches its input for a pattern and characters & last column shows filename. displays lines containing the pattern, the line numbers or the wcoptions : file names where the pattern occurs. -l : prints the line count Syntax is : -L : prints the length of the longest line $grp [ options ] [ pattern ] [ filename ]…. -m : prints the number of characters Example : -w : counts words delimited by the white space characters or $cat stdlist new line characters. 001 : Deepak : 92 : BCA Example: 002 : Vishal : 78 : BCA $ wcemplist 003 : Payal : 80 : BSc O/P : 5 19 90 emplist 004 : Peter : 60 : BSc $ wc -l emplist 005 : Gopal : 45 : BCA O/P : 5 emplist Grep options : $ wc -L -c : It is the count option O/P : 17 emplist $grep -c ‘BSc’ stdlist $ wc -m emplist 2 O/P : 90 emplist -n : This option is used to display the line numbers of $ wc -w emplist matching records along with the record. O/P : 19 emplist $grep -n ‘y’ stdlist vi) tr stdlist : 3 : 003 : Payal : 80 : BSc tr command is used to translate characters. It is a filter that -v : This option is known as the inverse option manipulates individual characters in a line $grep -v BCA stdlist The general syntax is : 003 : Aman : 80 : BSc tr[ options] [ expression1 ] [expressions 2] < input file 004 : Akhila : 60 : BSc

Where tr takes the input file & replaces the expression1 with REFERENCES expression2. [1] ttp://linuxshellscripting.blogspot.in/2011/02/shell- Example: Scripting-advantages-and.html $ cat list 1 [2] http://www.thegeekstuff.com/2010/07/execute- A friend in need is a friend in deed. $ tr ‘frie’ ‘FRI’ < list 1 Shell-script/ A FRIIndInnIId Is a FRIIndIndIId [3] http://en.wikipedia.org/wiki/Shell_script Here, all f are replaced by F [4] http://en.wikipedia.org/wiki/Filters all r are replaced by R [5] http://www.calpoly.edu/~rasplund/script.html all I are replaced with I troptions : [6]http://supportweb.cs.bham.ac.uk/docs/tutorials/doc -c : complements the set of characters specified by the first system/build/tutorials/unixscripting/unixscripting.htm[7] string Jayalakshmi ‘ Unix Shell Programming” ISBN: 978-93-5267- -d : Deletes all occurences of input characters that are 892-1, 2017. specified by the first string [8] Shivangi Shandilya, Surekha Sangwan, Ritu -s : Replaces repeated characters with a single character. Yadav” SHELL SCRIPTING AND SHELL PROGRAMMING IN UNIX” IJIRT | Volume 1 vii) cat Issue 11 | ISSN: 2349-6002,2014

IJCRT1812312 International Journal of Creative Research Thoughts (IJCRT) www.ijcrt.org 655