System-Level I/O I Have Modified Them and Added New Slides
Total Page:16
File Type:pdf, Size:1020Kb
JDEP 284H Giving credit where credit is due Foundations of Computer Systems Most of slides for this lecture are based on slides created by Drs. Bryant and O’Hallaron, Carnegie Mellon University. System-Level I/O I have modified them and added new slides. Dr. Steve Goddard [email protected] http://cse.unl.edu/~goddard/Courses/JDEP284 2 Topics A Typical Hardware System CPU chip register file Unix I/O ALU Robust reading and writing system bus memory bus Reading file metadata I/O main bus interface Sharing files bridge memory I/O redirection Standard I/O I/O bus Expansion slots for other devices such USB graphics disk as network adapters. controller adapter controller mouse keyboard monitor disk 3 4 Reading a Disk Sector: Step 1 Reading a Disk Sector: Step 2 CPU chip CPU chip CPU initiates a disk read by writing a Disk controller reads the sector and register file command, logical block number, and register file performs a direct memory access (DMA) destination memory address to a port transfer into main memory. ALU ALU (address) associated with disk controller. main main bus interface bus interface memory memory I/O bus I/O bus USB graphics disk USB graphics disk controller adapter controller controller adapter controller mouse keyboard monitor mouse keyboard monitor disk disk 5 6 Page 1 Reading a Disk Sector: Step 3 Unix Files CPU chip When the DMA transfer completes, the register file disk controller notifies the CPU with an A Unix file is a sequence of m bytes: interrupt (i.e., asserts a special “interrupt” B , B , .... , B , .... , B ALU 0 1 k m-1 pin on the CPU) All I/O devices are represented as files: /dev/sda2 (/usr disk partition) main bus interface /dev/tty2 (terminal) memory Even the kernel is represented as a file: I/O bus /dev/kmem (kernel memory image) /proc (kernel data structures) USB graphics disk controller adapter controller mouse keyboard monitor disk 7 8 Unix File Types Unix I/O Regular file The elegant mapping of files to devices allows kernel to Binary or text file. export simple interface called Unix I/O. Unix does not know the difference! Key Unix idea: All input and output is handled in a Directory file consistent and uniform way. A file that contains the names and locations of other files. Basic Unix I/O operations (system calls): Character special and block special files Opening and closing files Terminals (character special) and disks ( block special) z open()and close() Changing the current file position (seek) FIFO (named pipe) z lseek (not discussed) A file type used for interprocess comunication Reading and writing a file Socket z read() and write() A file type used for network communication between processes 9 10 Opening Files Closing Files Opening a file informs the kernel that you are getting Closing a file informs the kernel that you are finished ready to access that file. accessing that file. int fd; /* file descriptor */ int fd; /* file descriptor */ if ((fd = open(“/etc/hosts”, O_RDONLY)) < 0) { int retval; /* return value */ perror(“open”); exit(1); if ((retval = close(fd)) < 0) { } perror(“close”); exit(1); } Returns a small identifying integer file descriptor fd == -1 indicates that an error occurred Closing an already closed file is a recipe for disaster in threaded programs (more on this later) Each process created by a Unix shell begins life with three open files associated with a terminal: Moral: Always check return codes, even for seemingly 0: standard input benign functions such as close() 1: standard output 2: standard error 11 12 Page 2 Reading Files Writing Files Reading a file copies bytes from the current file Writing a file copies bytes from memory to the current file position to memory, and then updates file position. position, and then updates current file position. char buf[512]; char buf[512]; int fd; /* file descriptor */ int fd; /* file descriptor */ int nbytes; /* number of bytes read */ int nbytes; /* number of bytes read */ /* Open the file fd ... */ /* Open file fd ... */ /* Then write up to 512 bytes from buf to file fd */ /* Then read up to 512 bytes from file fd */ if ((nbytes = write(fd, buf, sizeof(buf)) < 0) { if ((nbytes = read(fd, buf, sizeof(buf))) < 0) { perror(“write”); perror(“read”); exit(1); exit(1); } } Returns number of bytes written from buf to file fd. Returns number of bytes read from file fd into buf nbytes < 0 indicates that an error occurred. nbytes < 0 indicates that an error occurred. As with reads, short counts are possible and are not errors! short counts (nbytes < sizeof(buf) ) are possible and are not errors! Transfers up to 512 bytes from address buf to file fd 13 14 Unix I/O Example Dealing with Short Counts Copying standard input to standard output one byte at a Short counts can occur in these situations: time. Encountering (end-of-file) EOF on reads. Reading text lines from a terminal. #include "csapp.h" Reading and writing network sockets or Unix pipes. int main(void) { Short counts never occur in these situations: char c; Reading from disk files (except for EOF) while(Read(STDIN_FILENO, &c, 1) != 0) Write(STDOUT_FILENO, &c, 1); Writing to disk files. exit(0); } How should you deal with short counts in your code? Use the RIO (Robust I/O) package from your textbook’s Note the use of error handling wrappers for read and csapp.c file (Appendix B). write (Appendix B). 15 16 The RIO Package Unbuffered RIO Input and Output RIO is a set of wrappers that provide efficient and robust I/O in Same interface as Unix read and write applications such as network programs that are subject to short counts. Especially useful for transferring data on network RIO provides two different kinds of functions sockets Unbuffered input and output of binary data z rio_readn and rio_writen #include “csapp.h” Buffered input of binary data and text lines ssize_t rio_readn(int fd, void *usrbuf, size_t n); z rio_readlineb and rio_readnb ssize_t rio_writen(nt fd, void *usrbuf, size_t n); z Cleans up some problems with Stevens’s readline and readn Return: num. bytes transferred if OK, 0 on EOF (rio_readn only), -1 on error functions. z Unlike the Stevens routines, the buffered RIO routines are thread- safe and can be interleaved arbitrarily on the same descriptor. rio_readn returns short count only it encounters EOF. Download from rio_writen never returns a short count. http://csapp.cs.cmu.edu/public/code.html http://csapp.cs.cmu.edu/public/ics/code/include/csapp.h Calls to rio_readn and rio_writen can be interleaved http://csapp.cs.cmu.edu/public/ics/code/src/csapp.c arbitrarily on the same descriptor. 17 18 Page 3 Implementation of rio_readn Buffered RIO Input Functions /* * rio_readn - robustly read n bytes (unbuffered) */ Efficiently read text lines and binary data from a file ssize_t rio_readn(int fd, void *usrbuf, size_t n) partially cached in an internal memory buffer { size_t nleft = n; #include “csapp.h” ssize_t nread; char *bufp = usrbuf; void rio_readinitb(rio_t *rp, int fd); while (nleft > 0) { ssize_t rio_readlineb(rio_t *rp, void *usrbuf, size_t maxlen); if ((nread = read(fd, bufp, nleft)) < 0) { ssize_t rio_readnb(rio_t *rp, void *usrbuf, size_t n); if (errno == EINTR) /* interrupted by sig handler return */ nread = 0; /* and call read() again */ Return: num. bytes read if OK, 0 on EOF, -1 on error else return -1; /* errno set by read() */ rio_readlineb reads a text line of up to maxlen bytes from } file fd and stores the line in usrbuf. else if (nread == 0) break; /* EOF */ z Especially useful for reading text lines from network sockets. nleft -= nread; rio_readnb reads up to n bytes from file fd. bufp += nread; } Calls to rio_readlineb and rio_readnb can be interleaved return (n - nleft); /* return >= 0 */ arbitrarily on the same descriptor. } z Warning: Don’t interleave with calls to rio_readn z Note: your text has a typo on this topic, pg 784. 19 20 RIO Example File Metadata Copying the lines of a text file from standard input to Metadata is data about data, in this case file data. standard output. Maintained by kernel, accessed by users with the stat and fstat functions. #include "csapp.h" /* Metadata returned by the stat and fstat functions */ int main(int argc, char **argv) struct stat { { dev_t st_dev; /* device */ int n; ino_t st_ino; /* inode */ rio_t rio; char buf[MAXLINE]; mode_t st_mode; /* protection and file type */ nlink_t st_nlink; /* number of hard links */ rio_readinitb(&rio, STDIN_FILENO); uid_t st_uid; /* user ID of owner */ while((n = rio_readlineb(&rio, buf, MAXLINE)) != 0) gid_t st_gid; /* group ID of owner */ rio_writen(STDOUT_FILENO, buf, n); dev_t st_rdev; /* device type (if inode device) */ exit(0); off_t st_size; /* total size, in bytes */ } unsigned long st_blksize; /* blocksize for filesystem I/O */ unsigned long st_blocks; /* number of blocks allocated */ time_t st_atime; /* time of last access */ time_t st_mtime; /* time of last modification */ time_t st_ctime; /* time of last change */ }; 21 22 How the Unix Kernel Represents Example of Accessing File Metadata Open Files /* statcheck.c - Querying and manipulating a file’s meta data */ #include "csapp.h" Two descriptors referencing two distinct open disk int main (int argc, char **argv) bass> ./statcheck statcheck.c files. Descriptor 1 (stdout) points to terminal, and { type: regular, read: yes struct stat stat; bass> chmod 000 statcheck.c descriptor 4 points to open disk file. char *type, *readok; bass> ./statcheck statcheck.c type: regular, read: no Descriptor table Open file table v-node table stat(argv[1], &stat); [one table per process] [shared by all processes] [shared by all processes] if (S_ISREG(stat.st_mode)) /* file type*/ File A (terminal) type = "regular"; else if (S_ISDIR(stat.st_mode)) stdin fd 0 File access stdout type = "directory"; fd 1 File pos File size Info in else stderr fd 2 stat fd 3 refcnt=1 File type struct type = "other"; ..