Outline ❚ Basics of File Systems Unix File and I/O ❚ Directory and Unix File System: inode ❚ UNIX I/O System Calls: open, close, read, write, ioctl (USP Chapters 4 and 5) ❚ File Representations: FDT, SFT, inode table ❚ fork and inheritance, Filters and redirection ❚ File pointers and buffering Instructor: Dr. Tongping Liu ❚ Directory operations ❚ Links of Files: Hard vs. Symbolic 1 2 Storing Information File Systems ❚ Applications may store information in the process ❚ For long-term information storage: address space. But it is a bad idea. Ø Should be able to store very large amount of information Ø Size is limited to size of virtual address space Ø Information must survive the processes of using it ü May not be sufficient, e.g. airline reservations, banking Ø Should provide concurrent accesses to multiple processes Ø The data is lost when the application terminates ❚ Solution: ü Even when a computer doesn’t crash! Ø Store information to disks in units called as “files” Ø Multiple processes might want to access the same data Ø Files are persistent until users delete it explicitly ü Imagine a telephone directory of one company Ø Files are managed by the OS ❚ File Systems: How the OS manages files! Refer the slides of Profs. Sirer and George at Cornell 1 File Naming File Attributes ❚ Motivation: ❚ File-specific info maintained by the OS Ø People cannot remember block, sector, … Ø File size, modification date, creation time, etc. Ø Varies a lot across different OSs Ø They should have human-readable names ❚ Some examples: ❚ Basic idea: Ø Name – human-readable form Ø Process creates a file, and gives it a name Ø Identifier – a unique tag (number) identifies a file within the file system ü Other processes can access the file by the name Ø Type – required for systems supporting different types (e.g. ro, exe) Ø Naming conventions are OS dependent Ø Location – pointer to the file location on device Ø Size ü Usually names are allowed to be less than 255 characters Ø Protection – controls who can read, write, execute the file ü Digits and special characters are sometimes allowed Ø Time, date, and user identification – data for protection, security, and ü MS-DOS and Windows are not case sensitive, but UNIX-like OSs are usage monitoring File Protection Access Example in Linux ❚ File owner/creator should be able to control: ❚ Mode of access: read, write, execute Ø what can be done ❚ Three classes of users: Ø by whom Ø Owner: Ø Group: ❚ Types of access Ø Public: RWX Ø Read 7: 1 1 1 Ø Write owner!group! public! 6: 1 1 0 Ø Execute chmod! game! 1: 0 0 1 Ø Append 761! Ø Delete Ø List 2 Outline Directory: An Example ❚ Basics of File Systems ❚ Directory and Unix File System: inode (SGG 12) ❚ UNIX I/O System Calls: open, close, read, write, ioctl ❚ File Representations: FDT, SFT, inode table ❚ fork and inheritance, Filters and redirection ❚ File pointers and buffering ❚ Directory operations ❚ Links of Files: Hard vs. Symbolic An example tree structure of the file system. 9 10 Structure of a Unix File System Definition of inode ❚ The inode is a data structure in a Unix-style file system that describes a filesystem object, such as a file or directory. ❚ Each inode stores the attributes and disk block location(s) of the object's data. ❚ Inode attributes may include metadata (times of last change, access, modification), as well as owner and permission data. A directory entry contains only a name and an index into a table with the information of a file (inode) 11 12 3 More about “inode” Traditional inode Structure ❚ Each file system (such as ext2 or ext3) maintains an array of inodes-- the inode table, which contains list of all files. Each individual inode has a unique number (unique to that filesystem): the inode number. ❚ An inode stores: Pointers to location of file Ø File type: regular file, directory, pipe etc. Ø Permissions to that file: read, write, execute Ø Link count: The number of hard link relative to an inode Ø User ID: owner of file Ø Group ID: group owner Ø Size of file: or major/minor number in case of some special files Ø Time stamp: access time, modification time and (inode) change time Ø Attributes: immutable' for example Ø Access control list: permissions for special users/groups Ø Link to location of file Ø Other metadata about the file 13 14 File Size and Block Size File Size and Block Size ❚ Block size is 8K and pointers are 4 bytes ❚ Block size is 8K and pointers are 8 bytes ❚ One single indirect pointer Ø How large a file can be represented using only one single Ø A block contains 2K pointers to identify file’s 2K blocks indirect pointer? Ø File size = 2K * 8K = 16MB Ø What about one double indirect pointer? ❚ One double indirect pointer Ø What about one triple indirect pointer? Ø 2K pointers à 2K blocks, where each block contains 2K pointers to identify blocks for the file Ø Number of blocks of the file = 2K*2K = 4M Ø File size = 4M * 8K = 32 GB ❚ One triple indirect pointer Ø Number of blocks of the file = 2K*2K*2K = 8 *K*K*K Ø File size à 64 TB ( = 2^33) ; 15 16 4 Outline Unix I/O Related System Calls ❚ Basics of File Systems ❚ Device independence: uniform device interface ❚ Directory and Unix File System: inode ❚ I/O through device drivers with standard interface ❚ UNIX I/O System Calls: open, close, read, write, ioctl ❚ Use file descriptors (FDs) ❚ File Representations: FDT, SFT, inode table Ø FD is an abstract indicator (handle) used to access a file or other input/output resource ❚ fork and inheritance, Filters and redirection ❚ 3 file descriptors open when a program starts ❚ File pointers and buffering Ø STDIN_FILENO (0), STDOUT_FILENO (1), ❚ Directory operations STDERR_FILENO (2) ❚ Links of Files: Hard vs. Symbolic ❚ 6 main system calls for I/O Ø open, close, read, write, ioctl, lseek Ø Return -1 on error and set errno 17 18 File position open ❚ An open file has its file position that keeps track of #include <fcntl.h> where the next character is to be read or written. #include <sys/stat.h> Ø On GNU systems, and all POSIX.1 systems, the file int open(const char *path, int oflag); position is simply an integer representing the number of bytes from the beginning of the file. int open(const char *path, int oflag, Ø The file position is normally set to the beginning of the file mode_t mode); when it is opened, and each time a character is read or written, the file position is incremented. In other words, accessing to a file is normally sequential. ❚ Oflag: O_RDONLY, O_WRONLY, O_RDWR, O_APPEND, O_CREAT, O_EXCL, O_NOCTTY, Ø File position can be changed by lseek O_NONBLOCK, O_TRUNC; 19 20 5 open (cont.) ❚ O_CREAT flag: must use the 3-parameter form of open and specify permissions POSIX Symbolic Names for Permissions (mode) defined in sys/stat.h Historical layout of the permissions mask. 21 22 Program 4.9 (p106): copyfilemain.ccopy a file. close and its usage #include <unistd.h> int close(int fildes); ❚ Open files are closed when program exits normally. 23 24 6 System Call: read An example to read in a line #include <unistd.h> ssize_t read(int fildes, void *buf, Read one char at a time! size_t nbyte); ❚ Need to allocate a buffer *buf to hold the bytes read; ❚ size_t is an unsigned long type; ❚ ssize_t is a signed long type; ❚ Can return fewer bytes than requested; ❚ Return value of -1 with errno set to EINTR is not usually an error. 25 26 System Call: write Example for write ?? Function copyfile reads from one file and writes out to another #include <unistd.h> ssize_t write(int fildes, const void *buf, size_t nbyte); ❚ Not error if return value > 0 but less than nbyte Ø Must restart write if it returns fewer bytes than requested ❚ Return value of -1 with errno set to EINTR is not an error usually 27 28 7 Outline File Representation ❚ Basics of File Systems ❚ File Descriptor Table: ❚ Directory and Unix File System: inodes Ø An array of pointers indexed by the file descriptors ❚ UNIX I/O System Calls: open, close, read, write, ioctl Ø The pointers point to entries in System File Table ❚ File Representations: FDT, SFT, inode table ❚ System File Table: Ø Contains entries for each opened file ❚ fork and inheritance, Filters and redirection Ø Entries contain pointers to a table of inodes kept in ❚ File pointers and buffering memory ❚ Directory operations Ø Other information in an entry: current file offset; file ❚ Links of Files: Hard vs. Symbolic mode; count of file descriptors using this entry; Ø When a file is closed, the count is decremented. The entry is freed when the count becomes 0. ❚ In-Memory Inode Table: copies of in-use inodes 29 30 File Representation (cont.) What Happened for File Read/Write 1. The process passes the file descriptor to the kernel through a system call (read/write) Per Process 2. The kernel will access the entry in file table on behalf of the process, by getting the pointer to the inode All Processes 3. For read/write, after obtaining the file pointer (offset) in the file table, then it can compute the All Processes block/sector information. 4. In the end, it will issue IO request. IO scheduler may combine multiple requests with the close Figure 4.2 (page 120): Relationship between the file descriptor table, the system file table and the in-memory inode table.