Project 3 Help Document Hard disk drive structure Since the FAT32 file system is originally designed for hard disk drives, it is necessary to understand the structure of a hard drive because FAT32 organize data according to the hard drive structure. As an example, FAT32 always allocates space for files in terms of sectors. More information about the hard drive structure can found at this web page linkage http://en.wikipedia.org/wiki/Cylinder-head-sector. The above figure illustrates a simplified structure of a hard disk drive. A sector is a fixed division of a track on a disk. And A cluster is a group of contiguous sectors and basic unit for FAT32 files. The FAT tables in FAT32 provide information about used clusters, reserved clusters, and free clusters. All clusters allocated for a file is organized by FAT tables in a linked list manner . A USB flash drive does not resemble the structure of a hard drive, therefore, the sectors and clusters used in FAT32 on the USB flash drive are only logical concepts used by FAT32 instead of physical structures existing on the USB flash drive. FAT32 USB flash drive layout A USB flash drive formatted with FAT32 consists of four major sections: the boot sector, the FS information sector, FAT tables, and data area. FS Data Region (for Information More reserved File File files and Boot Sector sectors Allocation Allocation directories) ... Sector (FAT32 (optional) Table #1 Table #2 (To end of only) partition or disk) The Reserved sectors, located at the very beginning. The first reserved sector (sector 0) is the Boot Sector (aka Partition Boot Record). For FAT32 file systems, the reserved sectors include a File System Information Sector at sector 1 and a Backup Boot Sector at Sector 6. The FAT Region. This typically contains two copies (may vary) of the File Allocation Table for the sake of redundancy checking, although the extra copy is rarely used, even by disk repair utilities. These are maps of the Data Region, indicating which clusters are used by files and directories. The Data Region. This is where the actual file and directory data is stored and takes up most of the partition. The size of files and subdirectories can be increased arbitrarily (as long as there are free clusters) by simply adding more links to the file's chain in the FAT. Note however, that files are allocated in units of clusters, so if a 1 KiB file resides in a 32 KiB cluster, 31 KiB are wasted. FAT uses little endian format for entries in the header and the FAT(s). Organization of the FAT tables The FAT tables of FAT32 are organized as an array of FAT entries. Each entry of the FAT tables has a 4-byte value and only 28 (starting from the least significant bit) bits are used. The index of a FAT entry is associated with a cluster in the data region of the USB flash drive, for example, the 2nd entry of the FAT tables is associated with cluster 2 and the FAT table’s value of this 2nd entry indicates a possible cluster immediately following cluster 2 used by a file or directory. The first cluster in data area of the USB flash drive corresponds to cluster 2 for the first two FAT entries in the FAT tables are reserved. The FAT table can be regarded as a large array containing pointers into the data region of the USB flash drive. It is organized in a linked list manner. Whenever you try to access a file occupying more than one cluster, you need to reference the FAT table to obtain the next cluster of the file. A 32bit FAT entry could have the following values: FAT32 Description 0x00000000 Free Cluster 0x00000001 Reserved value; do not use 0x00000002–0x0FFFFFEF Used cluster; value points to next cluster 0x0FFFFFF0–0x0FFFFFF6 Reserved values; do not use. 0x0FFFFFF7 Bad sector in cluster or reserved cluster 0x0FFFFFF8–0x0FFFFFFF (EOC) Last cluster in file As we mentioned early, clusters of files or directories are organized in a linked list method in the FAT tables. The following figure illustrates this linked list method. File1.txt occupies a number of clusters and their numbers are 2, 3, and 4 respectively. When the File1.txt is accessed, cluster 2, 3, and 4 will be visited in order. After checking the 4th entry value in the FAT table, which is EOC (End-of-Cluster), we know that we have reached the end of a file. FAT table index (Cluster number) 0 1 2 3 4 5 N …… Reserved 3 4 EOC 0 0 FAT table value Cluster 2 Cluster 3 Cluster 4 Cluster N File1.txt File1.txt File1.txt …... (Part 1) (Part 2) (Part 3) Datat Region File1.txt … … … … Start cluster = 2 Directory table Directory structures A directory entry contains all of the information necessary to reference the contents of a file or directory on the volume, including additional information about the file/directory's attributes and other information including: time of creation/modification, date created, and size of the file (in bytes). All of this information is packed into a small 32-byte structure. The directory entry can be a file or a directory. If it is a directory, it is usually referred to as a directory table, which is a special type of file that represents a directory (also known as a folder). Each file or directory stored within it is represented by a 32-byte entry directory entry in the table. Directories with more than 16 entries require more than one sector (maximum entries per sector = 512 bytes per sector / 32 bytes per directory entry = 16 directory entries ). Here, we suppose the sector size is 512 bytes. For subdirectories that refer back to its parent directory, the startcluster value might be 0. Note that cluster 0 is reserved in the FAT table--attempting to look up that value should result in an error. You must re-direct a startcluster with value zero to the root directory. Delete files or directories Delete a file or directory does not immediately erase a file or directory when it is deleted. In fact, it does nothing to the clusters that contain the information. Delete operation simply remove the file or directory's cluster list from the FAT table and write a special character (0xe5) in the first byte of the directory entry to indicate that this entry has been deleted. You will need to follow the same procedure to delete a file or directory. Start with the cluster indicated in the directory entry, traverse the cluster list, and set each FAT entry to zero including the EOC entry. Convert a cluster number to offset Given the number of a cluster, you need to covert this number to a USB flash drive offset in order to read from/write to the location specified by the cluster number on the USB flash drive. You can use the following formula to calculate this offset: (numberOfReservedSectors + numberOfFat * sectorsPerFat + (numberOfCluster – 2) * sectorsPerCluster) * bytesPerSector The reserved Sectors consist of the boot Sector , the FS Information Sector, and the Optional Reserved Sectors. How to get a cluster number from a File Allocation Table 01 00 00 00 01 00 00 00 03 00 00 00 04 00 00 00 07 00 00 00 00 00 00 00 00 00 00 00 FF FF FF F0 What are the cluster numbers in the FAT32 file allocation table given in the above figure? Reserved, Reserved, 3, 4, 7, 0, 0, last cluster in a file (EOC). Each FAT entry is a 4-byte value with first 28 bits are valid and stored in little endian order. Virtual Filesystem (VFS) One of Linux's important features is to coexist comfortably wit different file systems. You can transparently mount disks or partitions that formatted by FAT32, NTFS, JFS, ext2, and many other file systems. Linux manages to support multiple file systems through the concept of the Virtual Filesystem(VFS). The key idea behind the VFS is to provide an abstraction layer between user space programs and different file systems. Whenever a program perform operations on files(regular files, directories, and symbolic links) in a specific file system, it will first invoke the common interfaces provided by the VFS, then the VFS will choose operations supported by the specific file system to do the real work. The following figure illustrates a simple file copy operation using VFS. (The picture is from Understanding Linux Kernel 3rd) For a device file, the VFS changes the default file operations of a device file and handle it differently compared with a regular file, a directory, or a symbolic link. And each system call on the device file is translated to an invocation of a device-related function instead of the corresponding function of the hosting filesystem. The device-related function (usually provided by the device driver) acts on the hardware device to perform the operation requested by a process. (Shown in the following figure) Read VFS f1=open(/dev/sdb1, O_RDONLY, 0); f2=open(/media/disk/readme.txt, O_RDONLY,0); Device Driver FAT32 file operations i = read(f1, buf1, 4096); i = read(f2, buf2, 4096); close(f1); close(f2); FAT32 FAT32 /dev/sdb1 /media/disk1/readme.txt Other documents 1. Official fat32 white paper 2. Wikipedia web page about File allocation table .