Lecture 23: Filesystems

Fall 2018 Jason Tang

Slides based upon Operating System Concept slides, http://codex.cs.yale.edu/avi/os-book/OS9/slide-dir/index.html Copyright Silberschatz, Galvin, and Gagne, 2013 1 Topics

• File Organization

• Filesystem Structure

• Filesystem Implementation

2 Files

• File: logical storage unit, collection of related information

• File consists of actual stored contents plus file attributes (so-called metadata)

• Metadata kept in directory structure (usually not with file data)

Attribute Usage Name Human readable symbolic name Identifier Unique numeric identifier Size Current file size Time, data, user Data for protection, security and usage identification monitoring

3 Higher-Order Structures

• Directory: collection of nodes containing information about all files

• Both directory structure and files reside on disk

• Partitions: subdivision of disks; also known as a slice

• Volume: any entity containing a file system

• May be subset of a device, or set of devices (as per a RAID)

• Tracks file system information in directories

4 Typical File System Organization

5 Directory Operations

• Search for a file: find a particular file, or find files whose names match a pattern

• Create a file and delete files

• List a directory: obtain list of all files in a directory, including file metadata

• Rename file: not supported in all OSes

6 Directory Organization

• Directory organized logically to obtain:

• Efficiency: locating a file quickly

• Naming: convenient to users

• Multiple users can have same name for different files

• Same file can have multiple names

• Grouping: logical grouping of files by properties (e.g., all Java programs)

7 Single-Level Directory

• Single directory, shared by all users

• Suffers from naming problem

• Suffers from grouping problem

8 Tree-Structured Directories

9 Tree-Structured Directories

• Efficient searching

• Grouping capability

• Absolute or relative path name

• Creating a new file is done in current working directory

• When attempting to delete a non-empty directory, either forbid operation or recursively delete all contents within

10 Acyclic-Graph Directories

• Multiple names for same file (aliasing)

• When deleting a file, either remove all pointers to file, or just remove link and actually delete file when its reference count reaches 0

11 Directory Implementation

• Linear list of file names with pointer to data blocks

• Simple to program, requires least amount of space

• Linear search time

• Could keep file names ordered alphabetically via linked list or B+ tree

• Hash table to decrease directory search time

• Collision when two file names hash to same location

12 Filesystem Structure

• Filesystem: provides user space interface to secondary storage (HDDs and SDDs)

• Maps logical files to physical locations on disk

• Efficient and convenient access to disk by allowing data to be stored, located, and retrieved easily

• Filesystems are optimized for specific use cases; there is no such thing as a “universal filesystem”

13 Modern Filesystems

• Windows: FAT32, NTFS

• Linux: Btrfs, Ext4, ReiserFS, XFS, ZFS

• Mac OS: HFS+, UFS, APFS

• Optical disks: ISO 9660, UDF

• Optimized for SSDs (without integrated wear controllers): JFFS, UBIFS

• Network-based: CEPH, Lustre, NFS

14 Filesystem Constraints

• HDDs and SSDs require I/O transfers performed in blocks of sectors (often in increments of 512 bytes)

• For HDDs, disk provides in-place rewrite and random access

• For SSDs, filesystem must take into account wear leveling and sector erase sizes

• Filesystems are often fault-tolerant

• Example: a small scratch on a DVD should not prevent access to data

• Filesystem driver usually organized into layers

15 Layered Filesystem

• I/O control layer manage devices

• Example: given command “read drive 1, cylinder 72, track 2, sector 10, into memory location 1060”, outputs low-level hardware specific commands to device controller

• Base filesystem layer tracks blocks and their physical locations; manages buffers and caches

• Example: given command “retrieve block 123”, outputs “drive 1, cylinder 72, track 2, sector 10”

16 Layered Filesystem

• File organization layer uses file allocation algorithm(s) to reserve logical blocks for files

• Translates logical block numbers (starting with 0) to physical location

• Logical filesystem manages metadata information

• Directory management

• Translates file name into file control block (FCB)

• Contains permissions, last modified time, etc

17 Filesystem Implementation

• Boot Control Block: contains data needed by system to boot OS stored on that volume

• Volume Control Block (also known as superblock or master file table): contains details, such as number of blocks, number of free blocks, block size, and mapping of blocks in use

• Directory Structure: contains names and inode numbers of items within (files and subdirectories)

18 Inodes

• For many filesystems, each file has an inode (index node), identified as a unique unsigned integer number

• Inodes stored on disk and contains file metadata

• Files can have multiple names, via hard links

• When file is moved to another directory within same device, its inode number stays the same

19 Opening Files

• Kernel has a system-wide open file table

• When a process opens a file, kernel searches system-wide table

• If file already opened, a per-process open file entry created pointing to existing entry (referenced by user space via a file handle or file descriptor)

• Otherwise, kernel copies contents of inode from disk into a new entry

20 File Operations

• User space reads/writes using its file handle

• Kernel tracks number of references to each entry within system-wide table

• Count decremented when process calls close() or exits

• When count reaches 0, kernel removes entry from table

21 Mounting Filesystems

• When kernel first accesses a filesystem, it must mount it to check for consistency

• Checks that all metadata are correct

• Verifies all checksums

• Mount point: location within filesystem that user space may access new filesystem

• Example: When placing a DVD into a DVD drive, Windows will automatically mount the disc’s UDF’s filesystem at the D: mount point

22 Linux Virtual Filesystem

• In Linux, all files are accessed through the virtual filesystem (VFS), an in- kernel abstraction layer

• Not to be confused with virtual files (like those in /proc or /sys)

• Allows same system call interface to be used for different filesystem types

• Separates filesystem generic operations from implementation details

• Implementation can be one of many filesystem types, or even reach across a network for network-based filesystems

23 Linux Virtual Filesystem

24 Linux Virtual Filesystem

• Every Linux filesystem registers itself as a series of function pointer table

• struct super_operations: functions to allocate blocks, perform consistency checks, and other operations involving superblocks

• struct inode_operations: functions to manipulate inodes, lookup inodes within directories, update inode access times

• struct file_operations: handles actual reading and writing of files to secondary storage

• When a file is opened, its in-kernel handle’s function pointer tables are set to underlying filesystem’s function pointer tables

25