File System Implementation CISC3595, Spring 2015 1 Objectives for a File Management System ! Meet the data management needs of the user! ! Provide I/O support for a variety of storage device types! ! Provide a standardized set of I/O interface routines to user processes! ! Provide I/O support for multiple users (if needed)! ! Guarantee that the data in the file are valid! ! Minimize lost or destroyed data! ! Optimize performance Requirements for a general purpose system 1. user should be able to create, delete, read, write and modify files! 2. user may have controlled access to other users’ files! 3. user may control what type of accesses are allowed to his/her files! 4. user should be able to restructure his/her files! 5. user should be able to move data between files! 6. user should be able to back up and recover files in case of damage! 7. user should be able to access files using symbolic names Virtual File Systems ! Virtual File Systems (VFS):! ! same system call interface (API) used for different types of concrete file systems! ! Support numerous file system types! ! ext2, ufs, fat, vfat, hpfs, minix, isofs, sysv, hfs, affs, NTFS! ! /proc file system! ! NFS, CoDA, AFS ncpfs! ! umsdos, userfs 8 Virtual File Systems (1) Figure 4-18. Position of the virtual file system. Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639 Mount ! Various file systems are mounted at different directories (mounting points) in the files system name space! $ mount! $ /dev/sda3 on / type ext3 (rw,relatime,errors=remount-ro)! tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755) ! proc on /proc type proc (rw,noexec,nosuid,nodev) ! sysfs on /sys type sysfs (rw,noexec,nosuid,nodev) ! varrun on /var/run type tmpfs (rw,nosuid,mode=0755) ! varlock on /var/lock type tmpfs (rw,noexec,nosuid,nodev,mode=1777) udev on /dev type tmpfs (rw,mode=0755) ! tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev) ! devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620) fusectl on /sys/ fs/fuse/connections type fusectl (rw) lrm on /lib/modules/2.6.28-11-generic/volatile type tmpfs (rw,mode=755) securityfs on /sys/kernel/security type securityfs (rw) binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev) gvfs-fuse-daemon on /home/zhang/.gvfs type fuse.gvfs- fuse-daemon (rw,nosuid,nodev,user=zhang) 9 Hard Disk Accessing Hard disk ! ! Seek time: in ms! (moving read head to track)! Rotation time: in ms! (wait until sector rotate to read! head)! Transmission time: in hundreds of! MB/s! # of bytes/ (transfer speed) ! ! ! vs accessing RAM: ! 10 ns every 32 bits! Block • File system block: the allocation unit of disk storage space ! • Also transfer unit when read/write disk! • similar concept in paging memory management: page! • Always two’s power: 512, 1024, 2048, 4096, …! • Choosing block size: ! • Small block size => ? ! • Large block size => ? ! • In this class (chapter of book), assume an abstract view of disk: ! • a disk is nothing but an “array” of blocks, ! • We can read block k, write block k! • We want to support file system service we learnt last week! • hierarchy structure! • file/directory operations (system calls)… ! Disk Space Management Block Size (1) Figure 4-20. Percentage of files smaller than a given size (in bytes). Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639 Disk Space Management Block Size (2) For more info,! See book P263! ! Data rate = ! ! Figure 4-21. The solid curve (left-hand scale) gives the data rate of a disk. The dashed curve (right-hand scale) gives the disk space efficiency. All files are 4 KB. Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639 File System Layout Boot block: used to boot operating system (i.e., load OS code into RAM)! Superblock: keep file system parameters ! Free space mgmt: what blocks in this partition is free ! i-nodes: metadata and address of blocks allocated to each file/directory! Figure 4-9. A possible file system layout. Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639 Outline ! Abstract view of hard disk! ! Disk space allocation and management! ! Efficiency and Performance! ! Recovery! ! NFS 20 Allocation Methods ! Allocate disk space to files! ! ! Contiguous allocation! ! ! Linked allocation! ! ! Indexed allocation 21 Contiguous Allocation ! Each file occupies a set of contiguous blocks on the disk! ! Pros:! ! Simple – only starting location (block #) and length (number of blocks) are required! ! Random access! ! Cons:! ! Wasteful of space: dynamic storage- allocation problem: how to satisfy request from list of non-contiguous free holes! ! External fragmentation! ! Files cannot grow 22 Linked Allocation ! Each file is a linked list of disk blocks: blocks may be scattered anywhere on the disk.! ! Directory contains pointer to the first and last blocks of the file.! ! pointer to next block is stored in block! ! data stored in each block is no more two’s power 23 Linked List Allocation Figure 4-11. Storing a file as a linked list of disk blocks. Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639 File-Allocation Table ! File-allocation table (FAT)! ! collectively store “next block” for entire file system in one table! ! used in MS-DOS and OS/2 (FAT12, FAT16,FAT32)! ! one FAT per partition! ! one entry for each disk block! ! store pointer to next block in file/directory! ! For each file, only needs to block # for the first block! 24 Linked List Allocation Using a Table in Memory Figure 4-12. Linked list allocation using a file allocation table in main memory. Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639 MS-DOS File System # of bits per entry Maximum partition size for different block sizes. The empty boxes represent forbidden combinations. Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639 Indexed Allocation (i-node, index-node) ! Brings all pointers (block #s) belonging to a file/ directory together into index block.! ! Each file has its own index block, or i-node (used in Unix file systems) 25 Indexed Allocation entry for a file 26 Combined Scheme: UNIX inode (4K bytes per block) 28 Free-Space Management (Cont.) ! Linked list (free list)! ! Cannot get contiguous space easily! ! No waste of space! ! Grouping ! ! First free block contains address of n free blocks! ! The n-th block therein contains address of another n free blocks, … ! ! Counting! ! Free blocks might be contiguous! ! Keep starting block # and length 30 Implementing Directories (1) A UNIX V7 directory entry. Figure 4-14. (a) A simple directory containing fixed-size entries with the disk addresses and attributes in the directory entry. (b) A directory in which each entry just refers to an i-node. Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639 Shared Files (1) Figure 4-16. File system containing a shared file. Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639 Shared Files (2) Figure 4-17. (a) Situation prior to linking. (b) After the link is created. (c) After the original owner removes the file. Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639 Outline ! Abstract view of hard disk! ! Disk space allocation and management! ! Efficiency and Performance! ! Recovery! ! NFS 20 Caching (1) Figure 4-28. The buffer cache data structures. Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639 Caching (2) • Some blocks, such as i-node blocks, are rarely referenced two times within a short interval. ! • Consider a modified LRU scheme, taking two factors into account: •Is the block likely to be needed again soon? •Is the block essential to the consistency of the file system? Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639 Outline ! Abstract view of hard disk! ! Disk space allocation and management! ! Efficiency and Performance! ! Recovery! ! NFS 20 The MS-DOS File System (1) Figure 4-31. The MS-DOS directory entry. Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639 The UNIX V7 File System (1) Figure 4-33. A UNIX V7 directory entry. Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639 The UNIX V7 File System (2) Figure 4-34. A UNIX i-node. Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639 The UNIX V7 File System (3) Figure 4-35. The steps in looking up /usr/ast/mbox. Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639 File system structures in memory ! Mount table: contains info. about each mounted volume! ! In-memory directory-structure cache: contains recent accessed directory info.! ! For a directory that is a mounting point, contains flag indicating it’s a mount point, and a pointer to an entry in mount table! ! System-wide open-file table: a copy of i-node for each open file! ! Per-process open-file table: contains pointer to appropriate entry in system-wide open-file table! ! Buffer: hold file system blocks being read from disk or written to disk 12 Supporting file system interface: