12/16/2009
File Systems
Project: implement simple UNIX-like file system Abstractions: Directories and files instead of disks Naming and creation File and directory naming Open, close, mkdir, rmdir COS318: File System Precept File access Read, write, seek Stat Princeton University, Fall 2009 Permission (not part of project) User management (not part of project)
Inodes Inode Structure Direct and Indirect Blocks Which disk blocks go with which file Inode: data structure for bookeeping List of blocks Our project: •Single level File or directory Link count Size Other information (owner, mode, times…)
Disk Layout Superblock
Contains the layout of the disk Boot Block (Our OS == entire image) Size of Disk
Super Block Number of Inodes Number of Data Blocks Inode Blocks Block Allocation Bitmap Where inodes start, where data blocks start, etc…. Data Blocks
1 12/16/2009
Project Formatting: fs_mkfs()
System call to initialize file system module Make a file system: init Write superblock System calls to access file system Mark inodes and data blocks to be free mkfs: Formatting Create root directory open: file creation Initialize file descriptor table link, unlink: close, read, write, lseek: file access mkdir, chdir, rmdir: directory stuff stat: information about a file or directory
Fs_init() vs. fs_mkfs() File Creation and Deletion
What need to be done where: fs_open(), fs_link(), fs_unlink() FS_init(): related to running system open: create a new file if it does not exists Takes care of system wise FS related initialization. FS_mkfs(): related to the FS on disk. link: hard link to a file Initialization of the FS itself. create a link to an existing file How to tell? hard vs. soft link? Think what will happen if you have two disks in your system unlink: delete a file if link count == 0 When to initialize file descriptor table? delete directory entry When to set bitmaps and inodes to be free? When to initialize current Inode (correspond to “/”)?
File Access Fs_lseek() semantic
open: open existing file (allocate file descriptor) Our fs_lseek() only takes two arguments read: read bytes from open file fd, offset write: write bytes to open file Unix lseek() takes three lseek: change position in file fd, offset, whence close: free file descriptor Whence: SEEK_SET, SEEK_CUR, SEEK_END Our fs_lseek() will assume SEEK_SET What if lseek() want to seek after end of file?
2 12/16/2009
Directories Directories (Implementation)
Like a file: List of files and directories mkdir: make a directory Name to inode number mapping create an entry in parent directory Can read it like a file create two directories: “.”, “..” Use existing file I/O functions to do directory manipulation. rmdir: remove directory if empty Avoid duplicate efforts internally chdir: change the current directory Always has at least 2 entries: For relative path names “.” current directory “..” parent directory
Example: mkdir() Absolute pathnames Do we need to support absolute pathname? int fs_mkdir(char *file_name) { When I am in “/foo”, do I need to support: chdir /bar/tmp/? if (file_name exists) return ERROR; No. /* allocate data block */ Otherwise, you would have to support absolute pathname everywhere: open, link, mkdir, etc /* allocate inode */ /* set directory entries for „.‟, „..‟ */ /* set inode entries appropriately * /* update parent */ return SUCCESS }
Removing a directory Extra credits
Do I need to support recursive directory removal? Implement file system caching (1 point) I.e.: Remove all the subdirectories and files contained in the 1. Define cache data structures parent directory? 2. Modify system calls such that cache is checked before disk No. data structures Just return error if directory to be removed is non-empty. 3. Implement the LRU replacement policy 4. Implement a flush operation (fs_flush) Implement a consistent cache (1 point) Data on disk should always be in a consistent state What if the system crashed during a flush? What order must meta-data and data blocks be written to disk?
3 12/16/2009
File system check Doing the Assignment
File system check function is useful for debugging (but not Mostly under Linux environment a requirement nor extra credit) Use a file to simulate a disk Verify integrity of: code is provided (*Fake files) 1. Superblock magic number make lnxsh 2. Block allocations Should be able to move right over to our OS 3. Inode allocations Do not use glibc functions 4. Bitmaps Shell supports 5. Directory content System calls for File System 6. … Commands like “ls”, “cat foo”, “create foo 200” ~1000 lines of code to be written
Testing Design Document
We provide a python script used to test the file system No design review operation You need to submit a document describing your design Multiple tests that each: TA should understand design by just reading the document 1. Execute the linux shell (./lnxsh) Maximum 8 pages, but 6 should be enough 2. Open an existing file system (or format a new) Describe: 3. Write commands to shell (cat foo) 1. File system disk layout 4. Read output from shell (ABCDEFGHIJKL) 2. Super block structure/content 5. Exit 3. Inode data structure on disk You should write your own test cases 4. Free block data structure and management and submit these with your code 5. How to implement link/unlink 6. File cache design (for extra credits)
Practicalities
This is an individual project No discussion of project design/implementation with other students No help from TA, but: Send e-mail for clarifications or other questions A lot of code to be written so hard to help anyway No design review, instead you will submit a design document No Q/A planned but may do one after Christmas if needed
4