Topic 7: File and IPC Tongping Liu University of Massachusetts Amherst 1 Midterm: 3/17 8pm-10pm • Zoom Link (lectures link): – https://umass-amherst.zoom.us/j/91463665907 • Close book and close notes, but with one cheat sheet • Open the video but mute yourself • Ask questions verbally in a different room or over the chat • Clarify some questions over the chat Let me know if you can’t make it for a valid reason!! University of Massachusetts Amherst 2 Outline • File and IO Operations • File Representations: FDT, SFT, inode table • Fork and Inheritance, Filters and Redirection • Hard vs. Symbolic Links • IPC: Pipe and Named Pipe • IPC: Shared Memory • IPC: Memory-Mapped Files University of Massachusetts Amherst 3 Information Storing • Applications may store information in the process’s address space. But it is a bad idea. – Size is limited to virtual address space • May not be sufficient for airline reservations, banking, etc. – The data is lost when an application terminates • Even when a computer doesn’t crash! – Multiple processes cannot share the same data University of Massachusetts Amherst 4 Information Storing • Criteria for long-term information storing: – Can store large amount of information – Must survive the process of creating it – ShoulD proviDe concurrent accesses to multiple processes • Solution: – Store information on Disks in a unit calleD “file” – Files are persistent, and only owner can explicitly Delete it – Files are manageD by the OS • File System: controls how data is stored and retrieved. University of Massachusetts Amherst 5 Unix File System Layout • Files are stored on disks – Disk is divided into multiple partitions – Sector 0 of disk called Master Boot Record – End of MBR has partition table (start & end of partitions) University of Massachusetts Amherst 6 Tracking Free Disk Space • Two approaches to keep track of free disk blocks – Linked list or bitmap University of Massachusetts Amherst 7 File Naming • Motivation: abstract information stored on disk – You do not need to remember block, partition, … – Should have human readable names • A process creates a file, and gives it a name – Other processes can access the file by that name • Naming conventions are OS dependent – Usually less than 255 characters – Digits and special characters are sometimes allowed – MS-DOS and Windows are not case sensitive, UNIX family is University of Massachusetts Amherst 8 File Attributes • File-specific info maintained by the OS – File size, modification date, creation time, etc. – OS dependent • Other attributes: – Name – only information kept in human-readable form – Identifier – a unique tag (number) identifies a file within file system – Type – normal or special types – Location – pointer to file location on device – Protection – controls who can do reading, writing, executing – Time, date, and user identification – data for protection, security, and usage monitoring University of Massachusetts Amherst 9 File Protection • File owner/creator should be able to control: – What can be done by whom • Types of access – Read – Write – Execute – Append – Delete University of Massachusetts Amherst 10 Unix Example • Mode of access: read, write, execute • Three classes of users: – Owner: – Group: – Public: RWX 7: 111 owner group public 6: 110 1: 001 chmod 761 game chmod can change file permissions University of Massachusetts Amherst 11 I/O Related System Calls • Five I/O system calls: – open, close, read, write , ioctl – Return -1 on error and set errno • Use file descriptor (integer) to identify each file • 3 shared/opened file descriptors: – STDIN_FILENO (0) , STDOUT_FILENO (1), STDERR_FILENO (2) University of Massachusetts Amherst 12 open() #include <fcntl.h> #include <sys/stat.h> int open(const char *path, int oflag); Creating a file int open(const char *path, int oflag, mode_t mode); • Return a file descriptor (an integer) • oflag: O_RDONLY, O_WRONLY, O_RDWR, O_APPEND, O_CREAT, O_EXCL, O_NOCTTY, O_NONBLOCK, O_TRUNC; University of Massachusetts Amherst 13 open() • O_CREAT flag: use the 3-parameter form to specify permissions University of Massachusetts Amherst 14 Implementation of Copying A File University of Massachusetts Amherst 15 close() #include <unistd.h> int close(int fildes); • You could close a file explicitly • Open files are closed when a program exits normally. University of Massachusetts Amherst 16 read() #include <unistd.h> ssize_t read(int fildes, void *buf, size_t nbyte); • Need to allocate a buffer to hold the bytes of read • size_t is an unsigned long type • Return value of read is a signed value – Can return fewer bytes than requested – Return value of -1 with errno set to the error – EINTR is not an error University of Massachusetts Amherst 17 An Exmaple of read() Read one char at a time! University of Massachusetts Amherst 18 write() #include <unistd.h> ssize_t write(int fildes, void *buf, size_t nbyte); • Not an error if return value is less than nbyte • Return value of -1 with errno set to EINTR is not usually an error University of Massachusetts Amherst 19 An Example for write() reads from one file and writes out to another University of Massachusetts Amherst 20 Quiz on read() • Suppose infile has "abcdefghijklmnop". Assuming no errors occur, what are the possible outputs of the following program? abcd aXbZ abcZ etc. What should we do to ensure “abcd” in the buffer? University of Massachusetts Amherst 21 Outline • File and IO Operations • File Representations: FDT, SFT, inode table • Fork and Inheritance, Filters and Redirection • Hard vs. Symbolic Links • IPC: Pipe and Named Pipe • IPC: Shared Memory • IPC: Memory-Mapped Files University of Massachusetts Amherst 22 File Representation • File Descriptor Table: – An array of pointers indexed by file descriptors – Have pointers pointing to entries in System File Table • System/Open File Table: – Contains one entry for each open file – Each entry with a pointer to inode table – Other information: current file offset; count of file descriptors that are using this entry – When a file is closeD, the count is DecrementeD. The entry is freeD when the count becomes 0. • In-Memory inode Table: in-use inodes University of Massachusetts Amherst 23 inode: Store File Information University of Massachusetts Amherst 24 File Size and Block Size • Block size is 8K and pointers are 4 bytes – How large a file can be represented using only one single indirect pointer? – What about one double indirect pointer? – What about one triple indirect pointer? University of Massachusetts Amherst 25 File Size and Block Size • Block size is 8K and pointers are 4 bytes • One single indirect pointer – A block contains 2K pointers to identify file’s 2K blocks – File size = 2K * 8K = 16MB • One double indirect pointer – 2K pointers à 2K blocks, where each block contains 2K pointers to identify blocks for the file – Number of blocks of the file = 2K*2K = 4M – File size = 4M * 8K = 32 GB • One triple indirect pointer – Number of blocks of the file = 2K*2K*2K = 8 *K*K*K – File size à 64 TB ( = 2^33) ; 4B pointer à 32 TB University of Massachusetts Amherst 26 Relationship between File Representations Per Process Per System University of Massachusetts Amherst 27 What Happened for File Read/Write 1. The process passes the file descriptor to the kernel through a system call (read/write) 2. The kernel will access the entry in per-process descriptor, and get the pointer to system file table 3. For read/write, after obtaining the file offset in system file table, then it can compute the block/sector information from in-memory inode table. 4. OS will issue read. IO scheduler may combine multiple requests with the close location together. After getting, OS will get an interrupt and return from the system call. 5. For write, the OS will return when the data is putted into the buffer University of Massachusetts Amherst 28 File Representations Descriptor table System file table inode table [one table per process] [shared by all processes] [shared by all processes] File A (terminal) stdin fd 0 File access stdout fd 1 File pos File size Info in stderr fd 2 stat fd 3 refcnt=1 File type ... ... struct fd 4 File B (disk) File access File size File pos File type refcnt=1 ... ... System file table: dynamic information, like pos, refcnt inode table: more static, no change of inode if a file is not changed University of Massachusetts Amherst 29 Quiz on File Representations • If a file is opened twice in one process, how many entries will be created in file descriptor table? How many will be created in open file table? How many in inode table? • Each open() will return a file descriptor, which will has one entry in the open file table. But there is only one entry in inode table – Open file table contains Dynamic information about each file, such as file offset – Inode contains static information about a file, such as blocks…. University of Massachusetts Amherst 30 Quiz on File Representations • What about one file is opened separately by two processes? – Then we will have two entries in open file table, but only one inode entry University of Massachusetts Amherst 31 An Example of File Representation • Multiple entries in open file table may point to the same entry in inode table • Multiple processes may point to the same entry in open file table What happened here? University of Massachusetts Amherst 32 Outline • File and IO Operations • File Representations: FDT, SFT, inode table • Fork and Inheritance, Filters and Redirection • Hard vs. Symbolic Links • IPC: Pipe and Named Pipe • IPC: Shared Memory • IPC: Memory-Mapped Files University of Massachusetts Amherst 33 Inheritance File Descriptors • When fork creates a child – Child inherits a copy of the parent's address space, and its file descriptor table – This makes that multiple processes shares the same entry in open file table What will happen if the child closes a file descriptor it inherits? Closing the file descriptor in the child will close it for the child, but not for the parent, and vice versa.