CS162: Operating Systems and Systems Programming Lecture 24: Filesystems: FAT, FFS, NTFS !" July 201$ Charles &eiss %ttps://cs162.eecs.ber*eley(edu' +uilding a File System File System: Layer of OS that transforms block interface of disks (or other block devices) into Files, Directories, etc. File System Components ― 0isk Management: collecting disk )locks into files ― aming: Interface to find files )y name, not by blocks ― Protection: Layers to keep data secure ― &elia)ility'0urability: 4eeping o, files dura)le despite crashes, media failures, attacks, etc 2 Recall: Building a File System -2) 5ser vs. System 6ie7 of a File ― 5ser’s vie7: ● 0ura)le Data Structures ― System8s vie7 (system call interface/: ● Collection o, Bytes -5 39/ ● 0oesn’t matter to system w%at kind o, data structures you 7ant to store on dis*: ― System8s vie7 (inside OS/: ● Collection o, bloc*s (a )loc* is a logical trans,er unit, w%ile a sector is t%e p%ysical trans,er unit/ ● +loc* si;e >= sector si;e; in 5 39, bloc* si;e is 44+ ! FAT (File Allocation Table) FAT Disk Blocks 0: 0: Simple way to store blocksfile number of a file: linked list 31: File 31, Block 0 File 31, Block 1 File number is the first block FAT contains pointers to the next block for each block File 31, Block 2 ― One entry for each data )loc* N-1: N-1: 4 FAT (File Allocation Table) FAT Disk Blocks 0: 0: file number Assume (for no7/ we have a 7ay to translate a path to a 31: File 31, Block 0 File 31, Block 1 ?2le number@ ABample: 2le_read 31, < 2, x< ― IndeB into FAT 7ith file num)er ― Follo7 linked list to block 2 o, the file File 31, Block 2 ― &ead the block from disk into mem N-1: N-1: $ FAT Properties FAT Disk Blocks 0: 0: File is collection o, dis* file number bloc*s 31: File 31, Block 0 FAT is lin*ed list 1E1 7ith File 31, Block 1 bloc*s File um)er is indeB o, first bloc* list ,or the file File offset -o = +:x / free Follo7 list to get bloc* G File 31, Block 2 Unused bloc*s = FAT ,ree list N-1: N-1: mem 6 Storing the FAT FAT Disk Blocks 0: 0: 2le number On disk 7%en system is s%utdown File 31, Block 0 ― 31: FiBed, 7ellE*nown location File 31, Block 1 File 63, Block 1 !o"ied in memory when runnin% ― 1akes accesses, updates ,ast ― Ot%erwise lots of disk seeking File 31, Block 3 File 63, Block 0 to locate t%e )loc*s o, a 2le ,ree 63: File 31, Block 2 2le descriptor 2 number N-1: N-1: H FAT Setup FAT Disk Blocks 0: 0: Format a new FAT drive? File 31, Block 0 File 31, Block 1 Lin* up the free list ,ree File 63, Block 1 ( File 31, Block 3 ( File 63, Block 0 ( File 31, Block 2 N-1: N-1: I FAT Assessment FAT Disk Blocks 0: 0: 2le number all over the place 5sed File 31, Block 0 ― 31: 0OS File 31, Block 1 ― Windows -sometimes) File 63, Block 1 ― T%umb 0ri.es &eally sim"le File 31, Block 3 File 63, Block 0 ,ree 63: ot muc% else in its fa.or((( File 31, Block 2 ― &andom access slo7 ― File creation slo7 N-1: N-1: K Lhat about the DirectoryJ File containing DfileCname: 2le_num)er< mappings Free space for new entries 3n FAT: attri)utes kept in directory (:::/ Eac% directory a linked list of entries L%ere do you find root directory ( “'@ )J 10 Directory Structure (Con8t) Mo7 many disk accesses to resol.e “/my/)ook'count”J ― Read in file %eader for root (fixed spot on dis*/ ― Read in first data )loc* ,or root ● Table o, 2le name/indeB pairs. Search linearly – ok since directories typically very small ― Read in file %eader for “my@ ― Read in first data )loc* ,or ?my@> search ,or ?)oo*@ ― Read in file %eader for “boo*@ ― Read in first data )loc* ,or ?boo*@> searc% ,or ?count@ ― Read in file %eader for “count@ Current working directory: Per-address-space pointer to a directory (2le #) used for resol.ing file names ― Allows user to speci,y relati.e filename instead o, absolute pat% (say CW0=?'my')oo*@ can resolve ?count@/ 11 +ig FAT security holes FAT %as no access rights FAT %as no %eader in the file blocks Just gives and index into the FAT ― -file num)er = block num)er) 12 Designing Better Filesystems Question: What 7ill they )e used forJ 13 Ampirical Characteristics of Files 1ost 2les are small 1ost of the space is occupied )y the rare )ig ones 14 So what about a “real” 2le system Meet the inode file_number 15 So what about a “real” 2le system Meet the inode Array on disk at wellE*nown file_number locations Reserved 7%en disk is "formatted" (more later/ 16 +SD Fast File System -1) Original inode format appeared in BS0 4.1 ― Berkeley Standard 0istri)ution UniB File Number is indeB into inode 'rrays Multi-level index structure ― Qreat for little to large files ― Asymmetric tree with fiBed sized blocks 17 +SD Fast File System -2) Metadata associated with the file ― &ather than in the directory that points to it 5 39 FFS: BS0 4.2: Locality Heuristics ― Attempt to allocate files contiguously ― Block group placement ― &eserve space Scalable directory structure 18 An “almost real” file system Pintos: src/filesys'2le.c, inode.c file_number 19 Mow do you find an inode on diskJ Inode is, say, 128 bytes Inode G10"" is 128 ) 1000 bytes into inode array Inode array is fixed location on disk ― A.g. starts at )lock #2 +locks are 4KB? Inode G1000 is in the middle of block G!! (= 2 + 128 S 1000 / 4096) 20 FFS: File Attributes Inode metadata – stored $ithin inode User Qroup K basic access control bits - UGO x RL9 Setuid bit E eBecute at o7ner permissions E rat%er t%an user Qetgid bit 21 E eBecute at group’s permissions FFS: Data Storage Small files: 12 pointers direct to data blocks 0irect pointers 4*+ bloc*s: suTcient ,or files up to 4I4+ 22 FFS: Freespace Management +it vector wit% a bit per storage block Stored at a fixed location on disk 23 Lhere are inodes stored? -1) In early U 39 and 0OS/Windo7s’ FAT 2le system, %eaders/inodes stored in special array in outermost cylinders ― Outermost because ,astest to read ― FiBed size, set when disk is form'tted. 24 Lhere are inodes stored? -1) In early U 39 and 0OS/Windo7s’ FAT 2le system, %eaders/inodes stored in special array in outermost cylinders ― Outermost because ,astest to read ― FiBed size, set when disk is form'tted. Mow does OS *now t%e size/location w%en booting up? Written in fixed location on every filesystem FFS: Called the "superblockP (Sometimes backup copies in case of failure) 25 Lhere are inodes stored? -2) In early U 39 and 0OS/Windo7s’ FAT 2le system, %eaders/inodes stored in special array in outermost cylinders Problem: How do you read a small fileJ ― &ead its inode -outermost cylinders/ ― &ead its data – pro)a)ly ,ar away ― +ots of seek time 26 Logistics Project 3 Design Revie7s Tomorro7 ― You should have already su)mitted your design doc ML3 out 27 +reak 28 Locality: Block Groups File system .olume is divided into a set of block groups ― !lose set of tracks Idea: Low see* times between inode/directories for a file and )locks of a file 29 Locality: Block Groups File data blocks, metadata, and free space are interleaved within block group ― o huge seeks Put directory and its 2les in common bloc* group !" FFS First Fit Block Allocation • Fills in the small holes at the start of block group • Avoids fragmentation, leaves contiguous free space at end 31 Locality: Block Groups FirstE,ree allocation o, ne7 2le bloc* ― Few little holes at start, big seWuential runs at end o, group ― Sequential layout for big files &eserve space in t%e BQ ― 10X ― Makes sure there's seWuential holes for )ig files ― Lets P2rst 2t" be fast – likely to find somet%ing Wuickly 32 Attack of the Rotational Delay Problem: Missing blocks due to rotational delay ― &ead one block, do processing, and read neBt )lock. ― In meantime, disk has continued turning: missed neBt )loc*: ― eed 1 revolution')lock! !! Attack of the Rotational Delay S*ip Sector Trac* Buffer -Molds complete trac*/ Solution 1: S*ip sector positioning (?interlea.ing”/ ― Place the )loc*s from one 2le on e.ery ot%er )lock of a track: give time for processing to o.erlap rotation ― FFS did this Solution 2: Read a%ead: read neBt bloc* right a,ter first – s"ecul'te t%at it will )e needed ― By OS -make larger reWuest/ N reWuires &A1 to %old result of read ― By disk -track buffers/ N reWuires &A1 in the controller ― 1ost modern disk controllers do t%is 34 +SD Fast File System Pros ― ATcient storage ,or )ot% sm'll and l'r%e 2les ― +ocalit# ,or bot% small and large 2les ― +ocalit# ,or metadata and data Cons ― 3neTcient ,or tiny files -a 1 byte 2le requires )ot% an inode and a data bloc*/ ― 3neTcient encoding 7%en 2le is mostly contiguous on dis* -no 7ay to say Pbloc*s 1"26E4"I$P N need to 7rite out eac% bloc* number/ ― Need to reserve 1"E2"X of ,ree space to prevent r'%ment'tion !$ +SD Fast File System Pros ― ATcient storage ,or )ot% sm'll and l'r%e 2les ― +ocalit# ,or bot% small and large 2les ― +ocalit# ,or metadata and data Cons ― 3neTcient ,or tiny files -a 1 byte 2le requires )ot% an inode and a data bloc*/ ― 3neTcient encoding 7%en 2le is mostly contiguous on dis* -no 7ay to say Pbloc*s 1"26E4"I$P N need to 7rite out eac% bloc* number/ ― Need to reserve 1"E2"X of ,ree space to prevent r'%ment'tion 36 Linux Axample: Ext2/!'4 Disk Layout Disk di.ided into bloc* groups ― Li*e FFS, pro.ides locality ― Each group %as t7o )lockE si;ed bitmaps -,ree blocks/inodes/ Actual Inode structure similar to 4.2+SD FFS ― 7it% 12 direct pointers ABt!: EBt2 w'#ournaling ― Better durability – more on t%at later Axample: create a file1.dat /dir1/ under in Ext! !H A bit more on directories /usr 5niB: Directories are just files Contents: list o (file n'me, file number- .usr.lib .usr.lib/*3 "'irs FFS: Lin*ed list li*e FAT .usr.lib4.3/foo &ead'7ritten