File System Implementation - Overview Structure of Disk
Total Page:16
File Type:pdf, Size:1020Kb
File System Implementation - Overview Structure of disk Sectors Reading: Tanenbaum ch 5 Track How is the file system structured? Cylinder Platter How is disk space for files and directories allocated and managed? Surfaces Files on disk - overview Layered File System applications programs logical file system file-organisation module Disk Partitions Blocks basic file system I/O control devices Directories, file system Files, free list File System Structure Disk blocks File system is on secondary storage I/O efficiency Primarily disk Block ID: numeric disk address: Issues: Drive, cylinder, surface and sector numbers File storage and access Size varies 32 to 4096 bytes, 512 being Disk space allocation a typical size Free space recovery Advantages of hard disk Read, modify, write back in place Performance Random or sequential access Blocks into files: inode (T p503) File System Layout Attributes Data Zones Direct 0 Each Direct 1 data and Boot Super Inode Zone Direct 2 indirect block block bit map bit map Direct 3 block 1k Direct 4 (up to 4k) Inodes Data Direct 5 blocks Each Direct 6 indirect Single block indirect holds 256 MINIX code: read_super p950-952; Double (up to indirect 1024) alloc_bit pp947-948 (Triple entries indirect) File Create and Open File-System Mounting Logical File System Required before access to file system Create: Read directory, update, write Requires back. Device or partition name Open: Read directory entry Mount point: where to mount While open, directory information Ensure valid file system (check stored in open file table structure) MINIX Mounting MINIX Mounting / / /bin bin usr /usr len mike /bin bin usr /usr Root Unmounted file system file system /usr/len len mike /usr/mike MINIX Mounting Implementation Up and Down the Tree Read super block into table (sp ptr) MINIX source pp994-995 (last_dir) Get inode of mount point (rip) 26371: Initialise root or current dir Get root inode of new fs (root_ip) 26382-26407: Component at a time rip->i_mount = I_MOUNT; pp 996-997 (advance) 26476-26484: Normal directory lookup sp->s_imount = rip; Special case: .. At root sp->s_isup = root_ip; 26491-26509: Up across mount point sp->s_rd_only = rd_only; 26516-26528: Down across mount MINIX source pp 1000-1002 Allocation Methods Contiguous Allocation Allocating disk space to files Files occupy contiguous blocks on disk Three major allocation methods Minimises head movement Contiguous Start block b, length n: Linked File blocks b, b+1, b+2,..., b + n -1 Indexed Directory entry: Start block b Length n Contiguous Allocation Accessing a Contiguous File Sequential: increment disk address 0 1 2 3 Direct: Calculate from first block and FILE start length 4 5 6 7 count 0 2 relative block number 8 9 10 11 tr 14 3 12 13 14 15 mail 19 6 f 6 2 16 17 18 19 20 21 22 23 24 25 26 27 Problems with Contiguous Allocation Extents Locating contiguous space Modified contiguous allocation External fragmentation Extending a file Directory Entry Linked Allocation using Linked Allocation Solves storage allocation problem Blocks can be allocated from anywhere on Directory entry disk Pointer to the first block in the file Size For empty file, pointer can be nil Phys block 4 11 7 2 Advantages & Disadvantages of Linked Allocation Clusters (= MINIX Zones) + easy file extension (no extents) + no external fragmentation Reduce pointer space: - inefficient random access Allocate groups of blocks (cluster/zone) - pointer space Fewer pointers (1 per cluster) Fewer disk seeks (cluster is contiguous) Increases internal fragmentation Reliability File Allocation Table FAT: A variation of linked allocation 0 directory entry - Pointer chains are vulnerable FAT test … 217 OS bug 217 618 Disk corruption 339 end of file 618 339 File Allocation Table Indexed Allocation Partition contains a section for FAT FILE index block jeep 19 Directory 0 1 2 3 First block number 4 5 6 7 8 9 10 11 Size 9 12 13 14 15 16 Easy to add new blocks (extend list) 1 16 17 18 19 19 10 FAT loaded into memory for efficiency 20 21 22 23 25 -1 Consistency checks 24 25 26 27 -1 Index Blocks inodes Each file has own index block Work out how much storage can be New file: All index entries nil referenced by single, double and triple indirect Any block may be used to extend file Remember, in a 32 bit machine, the No external fragmentation maximum addressable is 4 gigabytes Index block required (even very small file) Are double or triple indirection actually used then? Multiple index levels I-Nodes in Unix Version 7 BSD 4.3 inodes are numbered in sequence Cylinder group – consecutive cylinders Kernel uses 16 cylinders (settable during newfs) (logical device number, inode number) Super block (redundant) to identify file Cylinder block Maps: free inodes, data inodes located in array immediately following a single superblock at inodes, data beginning of logical device Staggered header blocks so not all on same platter Then followed by data blocks Windows 2000 File System - BSD 4.3 Layout NTFS Inode and file in same cylinder group NTFS designed for Win NT/2000 Directory and files in same group Features New directory low usage cyl group Recoverability Security Large file not hog cylinder group Large disks and large files supported After 1MB force use another group Multiple data streams New block: if cylinder group is full General indexing facilty Take hash of current cylinder group microsoft.com ntfs.com linux-ntfs.org NTFS Volume and File Structure NTFS Volume Layout Sector Every element on a volume is a file smallest physical storage unit on disk Every file is a collection of attributes Usually 512 bytes Even the data content is an attribute Cluster This makes things simple One or more contiguous sectors (on same NTFS volume layout track) – power of 2 sectors Partition Volume System Boot Master File Table File Area Files Maximum size is 2^64 bytes Sector NTFS Volume Layout NTFS Volume Layout Partition boot sector Called a sector, but can be up to 16 Master File Table (MFT) physical sectors long Contains information about all the files and Contains information about volume layout folders on the volume and file system structures And the unallocated space And Boot startup information and code MFT is a list of all contents on the volume, organised as rows in a relational database Master File Table NTFS Volume Inode Filename OS Description Rows are variable length Master File Table - An index of every 0 $MFT file Each row describes a file or folder on A backup copy of the first 4 records of 1 $MFTMirr the volume the MFT 2 $LogFile Transactional logging file Small files are held in the row 3 $Volume Serial number, creation time, dirty flag Large files have some data in row and 4 $AttrDef Attribute definitions rest in additional clusters 5 . (dot) Root directory of the disk Contains volume's cluster map (in-use Pointers to these clusters in the MFT row 6 $Bitmap vs. free) 7 $Boot Boot record of the volume 8 $BadClus Lists bad clusters on the volume NTFS Volume NTFS Volume Inode Filename OS Description Inode Filename OS Description 9 $Quota NT Quota information Any $ObjId 2K Unique Ids given to every file Security descriptors used by the Any $Quota 2K Quota information 9 $Secure 2K volume Any $Reparse 2K Reparse point information Table of uppercase characters used for 10 $UpCase Any $UsnJrnl 2K Journalling of Encryption collating >24 A_File An ordinary file A directory: $ObjId, $Quota, $Reparse, 11 $Extend 2K $UsnJrnl >24 A_Dir An ordinary directory 12-15 Marked as in use but empty 16-23 Marked as unused Some NTFS Attributes Recoverability Attribute Type Description Return file system to consistent state Standard Information Includes information such as timestamp and link count. Recovers file system not file contents Attribute List Additional record locations. File Name Repeatable (long and short A log of disk writes is maintained names, hard links) Each operation that alters the file Security Descriptor Describes who owns the file and who can access it. system is treated as a transaction Data Contains file data. Each part of transaction is logged to disk Object ID A volume-unique file identifier. Reparse Point Used for volume mount points. before modifying file system Bitmap For folders and other indexes. After crash, use log to undo/redo incomplete transactions WinFS Free Space Management PDC05 presentation DAT209 Reuse deleted file space Start with “What is WinFS?” Free list Bit vector Efficient if kept in memory Linked list Grouping Clustering Directory Implementation Disk Caching Linear list of names (unsorted) Performance enhancement Sorted linear list of names Disk controller – cache track or blocks Hash table OS cache blocks Reserved memory space LRU replacement Sequential files better served not by LRU Secondary Storage Structure Disk Scheduling the secondary storage structure Disks are slow involves the Schedule requests for better service disk head scheduling algorithms Disk access consists of three parts disc formatting seek boot block management rotation latency damaged block management transfer time swap space Bandwidth disk reliability I/O Request FCFS Scheduling Each I/O request: First Come First Served Operation is input or output Longest waiting request processed first Disk address for the transfer Easy to implement Memory address for the transfer Fair Number of bytes to be transferred May not give the best performance Each drive/controller: queue of requests Scheduling: select next request to process FCFS Scheduling SSTF Scheduling Queue : 98, 183, 37, 122, 14, 124, 65, 67 Head starts at 53 Shortest Seek Time First 0 14 37 53 65 67 98 122 124 183 199 Closest request to current head position A form of shortest job first scheduling Improves average disk service rate Disadvantages Starvation SSTF Scheduling SCAN/Elevator Scheduling Queue : 98, 183, 37, 122, 14, 124, 65, 67 Head starts at 53 Heads move across disk, processing 0 14 37 53 65 67 98 122 124 183 199 requests.