Part Workbook 7. Standard I/O and Pipes Table of Contents 1. Standard In and Standard Out .......................................................................................... 3 Discussion .............................................................................................................. 3 Three types of programs .................................................................................... 3 Standard in (stdin) and Standard out (stdout) ......................................................... 4 Redirecting stdout ............................................................................................ 4 Writing Output to a File ............................................................................ 4 Appending Output to a File ........................................................................ 4 Redirecting stdin .............................................................................................. 5 Under the Hood: Open Files and File Descriptors ................................................... 5 Open Files and File Descriptors .................................................................. 5 Redirection .............................................................................................. 7 Examples ................................................................................................................ 8 Example 1. Getting Out of sort's .......................................................................... 8 Example 2. Automated FTP Transfers ................................................................... 9 Example 3. Automating Graph Generation with gnuplot .......................................... 11 Online Exercises .................................................................................................... 12 Online Exercise 1. Using Standard In and Standard Out .......................................... 12 Specification .......................................................................................... 12 Deliverables ........................................................................................... 13 Suggested Strategy for Automating gnuplot ................................................. 13 Questions .............................................................................................................. 13 2. Standard Error ............................................................................................................. 17 Discussion ............................................................................................................. 17 Standard Error (stderr) ..................................................................................... 17 Redirecting stderr ........................................................................................... 18 Combining stdout and stderr: Old School ............................................................ 19 Combining stdout and stderr: New School ........................................................... 20 Summary ...................................................................................................... 20 Examples .............................................................................................................. 20 Example 1. Using /dev/null to filter out stderr ....................................................... 20 Online Exercises .................................................................................................... 21 Specification .................................................................................................. 21 Deliverables ................................................................................................... 22 Questions .............................................................................................................. 22 3. Pipes .......................................................................................................................... 26 Discussion ............................................................................................................. 26 Pipes ............................................................................................................ 26 Filtering output using grep ............................................................................... 27 Pipes and stderr .............................................................................................. 27 Commands as filters ........................................................................................ 28 Examples .............................................................................................................. 28 Example 1. Listing Processes by Name ................................................................ 28 Example 2. Searching Command History Efficiently .............................................. 29 Example 3. Unix philosophy: Simple Tools that Work Well Together ........................ 29 Online Exercises .................................................................................................... 31 Specification .................................................................................................. 32 Deliverables ................................................................................................... 32 Clean Up ...................................................................................................... 32 Questions .............................................................................................................. 32 2 Chapter 1. Standard In and Standard Out Key Concepts • Terminal based programs tend to read information from one source, and write information to one destination. • The source programs read from is referred to as Standard In (stdin), and is usually connected to a terminal's keyboard. • The destination programs write to is referred to as Standard Out (stdout), and is usually connected to a terminal's display. • When using the bash shell, stdout can be redirected using > or >>, and stdin can be redirected using <. Discussion Many Linux commands read input from the keyboard and display output to the terminal. In this Workbook, you'll learn how you can redirect where input is read from and where output goes. The output of one command can be used as the input for another command, allowing simple commands to be used together to perform more complicated tasks. Three types of programs In Linux (and Unix), programs can generally be grouped into the following three designs. Graphical Programs Graphical programs are designed to run in the X graphical environment. They expect the user to be using a mouse, and use common graphical components, such as popup menus and buttons, for user input. The mozilla web browser is an example of a graphical program. Screen Programs Screen based programs expect to use a text console. They make use of the entire display, and handle text placement and screen redraws in sophisticated ways. They do not require a mouse, and are appropriate for terminals and virtual consoles. The vi and nano text editors, and links web browser, are examples of screen based programs. Terminal Programs Terminal programs collect input and display output in a stream, seldom if ever redrawing the screen, as if writing directly to a printer that does not allow the cursor to move back up the page. Because of their simplicity, terminal based programs are often called simply commands. ls, grep, and useradd are examples of terminal based programs. This chapter focuses on the latter type of program. Do not let the simplicity of the way these commands receive input and output fool you. You will find that many of these commands are very sophisticated, and allow you to use the command line interface in powerful ways. 3 Standard In and Standard Out Standard in (stdin) and Standard out (stdout) Terminal based programs generally read information as as stream from a single source, such as a terminal's keyboard. Likewise, they generally write information as a steam to a single destination, such as a display. In Linux (and Unix), the input stream is referred to as Standard In (usually abbreviated stdin), and the output stream is referred to as Standard Out (usually abbreviated stdout). Usually, stdin and stdout are connected to the terminal that runs the command. Sometimes, in order to automate commonly repeated commands, or in order to record the output of a command for later inclusion in a report or email, people find it convenient to redirect stdin from or stdout into files. Redirecting stdout Writing Output to a File When a terminal based program generates output, it generally writes that output to its stdout stream, without knowing what is connected to the receiving end of that stream. Usually, the stdout stream is connected to the terminal that started the process, so the output is written to the terminal's display. The bash shell uses > to redirect a process's stdout stream to a file. For example, suppose the machine elvis is using becomes very sluggish and non-responsive. In order to diagnose the problem, elvis would like to examine the currently running processes. Because the machine is so sluggish, however, he wants to collect the information now, but analyze it later. He can redirect the output of the ps aux command into the file sluggish.txt, and come back to examine the file when the machine is more responsive. [elvis@station elvis]$ ps aux > sluggish.txt [elvis@station elvis]$ Notice that no output is displayed to the terminal. The