File System Fundamentals Addressing File System Structure Microsoft File Systems Journal Extents COW Defragmentation 6th Slide Set Operating Systems Prof. Dr. Christian Baun Frankfurt University of Applied Sciences (1971–2014: Fachhochschule Frankfurt am Main) Faculty of Computer Science and Engineering [email protected] Prof. Dr. Christian Baun – 6th Slide Set Operating Systems – Frankfurt University of Applied Sciences – SS2016 1/49 File System Fundamentals Addressing File System Structure Microsoft File Systems Journal Extents COW Defragmentation Learning Objectives of this Slide Set At the end of this slide set You know/understand. the functions and basic terminology of file systems an overview about Linux file systems and their characteristics what inodes and clusters are how block addressing works the structure of selected file systems an overview about Windows file systems and their characteristics what journaling is and why it is used by many file systems today how addressing via extents works and why it is implemented by several modern file systems what copy-on-write is how defragmentation works and when it makes sense to defragment Exercise sheet 6 repeats the contents of this slide set which are relevant for these learning objectives Prof. Dr. Christian Baun – 6th Slide Set Operating Systems – Frankfurt University of Applied Sciences – SS2016 2/49 File System Fundamentals Addressing File System Structure Microsoft File Systems Journal Extents COW Defragmentation File Systems. organize the storage of files on data storage Files are sequences of Bytes of any length which belongs together with regard to content manage file names and attributes (metadata) of files form a namespace Hierarchy of directories and files are a layer of the operating system (=⇒ system software) Processes and users access files via their abstract file names and not via their memory addresses should cause only little overhead for metadata Prof. Dr. Christian Baun – 6th Slide Set Operating Systems – Frankfurt University of Applied Sciences – SS2016 3/49 File System Fundamentals Addressing File System Structure Microsoft File Systems Journal Extents COW Defragmentation Linux File Systems Past Present Future classic file systems file systems with a journal Copy-on-write Minix Btrfs (1991) (unstable) ext2 ext3 ext4 (1993) (2001) (2008) ReiserFS Reiser4 (2001) (2009) JFS XFS (2002) (2002) Brackets contain the years of Linux kernel integration More file systems exist: Shared storage file systems: OCFS2, GPFS, GFS2, VMFS Distributed file systems: Lustre, AFS, Ceph, HDFS, PVFS2, GlusterFS File systems for flash storage: JFFS, JFFS2, YAFFS ... Prof. Dr. Christian Baun – 6th Slide Set Operating Systems – Frankfurt University of Applied Sciences – SS2016 4/49 File System Fundamentals Addressing File System Structure Microsoft File Systems Journal Extents COW Defragmentation Basic Terminology of Linux File Systems The creation of a file causes the creation of an inode too Inode (index node) Stores a file’s metadata, except the file name Metadata are among others the size, UID/GID, permissions and date Each inode has a unique inode number The inode contains references to the file’s clusters All Linux file systems base on the functional principle of inodes A directory is a file too Content: File name and inode number for each file in the directory File systems address clusters and not blocks of the storage device Each file occupies an integer number of clusters In literature, the clusters are often called zones or blocks This results in confusion with the sectors of the devices, which are in literature sometimes called blocks too Prof. Dr. Christian Baun – 6th Slide Set Operating Systems – Frankfurt University of Applied Sciences – SS2016 5/49 File System Fundamentals Addressing File System Structure Microsoft File Systems Journal Extents COW Defragmentation Cluster Size The size of the clusters is essential for the efficiency of the file system The smaller the clusters are. Rising overhead for large files Decreasing capacity loss due to internal fragmentation The bigger the clusters are. Decreasing overhead for large files Rising capacity loss due to internal fragmentation The bigger the clusters, the more memory is lost due to internal fragmentation File size: 1 kB. Cluster size: 2 kB =⇒ 1 kB gets lost File size: 1 kB. Cluster size: 64 kB =⇒ 63 kB get lost! The cluster size can be specified, while creating the file system In Linux: Cluster size ≤ size of memory pages (page size) The page size depends on the architecture x86 = 4 kB, Alpha and Sparc = 8 kB, IA-64 = 4/8/16/64 kB Prof. Dr. Christian Baun – 6th Slide Set Operating Systems – Frankfurt University of Applied Sciences – SS2016 6/49 File System Fundamentals Addressing File System Structure Microsoft File Systems Journal Extents COW Defragmentation Names of Files and Directories Important characteristic of file systems from the user perspective: Ways of naming files and directories File systems are often different from each other in this area File name length: All file systems accept character strings with 1-8 characters as file name Current file systems accept much longer file names and also numbers and special characters in file names Uppercase and lower case DOS/Windows file systems are not case-sensitive UNIX file systems are case-sensitive Prof. Dr. Christian Baun – 6th Slide Set Operating Systems – Frankfurt University of Applied Sciences – SS2016 7/49 File System Fundamentals Addressing File System Structure Microsoft File Systems Journal Extents COW Defragmentation Significance of File Names Current file systems support file names, which consist of 2 parts The second part, the extension is used to indicate the file content The extension is a short sequence of letters and numbers after the last dot in the file name Some files have 2 or more extensions. e.g. programm.c.Z The extension .C indicates that the file contains C source code The extension .Z stands for the Ziv-Lempel compression algorithm On UNIX, file extensions have originally no significance The file extension serves only to remind the owner what kind of data a file contains On Windows, file extensions always played an important role and they are allocated to applications Prof. Dr. Christian Baun – 6th Slide Set Operating Systems – Frankfurt University of Applied Sciences – SS2016 8/49 File System Fundamentals Addressing File System Structure Microsoft File Systems Journal Extents COW Defragmentation Accelerating Data Access with a Cache Modern operating systems accelerate the access to stored data with a cache (called Buffer Cache or Page Cache) in the main memory If a file is requested for reading, the kernel first tries to allocate the file in the cache If the file is not present in the cache, it is loaded into the cache The cache is never as big as the amount of data on the system That is why infrequently needed data must be replaced If data in the cache was modified, the modification must be passed down (written back) at some point in time Optimal use of the cache is impossible because data accesses are non-deterministic (unpredictable) Most operating systems do not pass down write accesses immediately (=⇒ write-back) DOS and Windows use the buffer Smartdrive Linux automatically uses the entire free main memory as buffer Benefit: Better system performance Drawback: System crashes may cause inconsistencies Prof. Dr. Christian Baun – 6th Slide Set Operating Systems – Frankfurt University of Applied Sciences – SS2016 9/49 File System Fundamentals Addressing File System Structure Microsoft File Systems Journal Extents COW Defragmentation Single-level Directory Structure File systems provide directories (folders) to organize the data Directories are just files, which contain the names and paths of files Most simple directory hierarchy: The root directory contains all files Situation on early computers: only a single user little storage capacity =⇒ only a few files Drawback: Causes issues in multi-user operating systems If a user wants to create a file, and the file of another user already has the same name, it will be overwritten Prof. Dr. Christian Baun – 6th Slide Set Operating Systems – Frankfurt University of Applied Sciences – SS2016 10/49 File System Fundamentals Addressing File System Structure Microsoft File Systems Journal Extents COW Defragmentation Two-level Directory Structure Challenge: Different users use the same file names Solution: Each user gets its own private directory This way, it is no problem, when multiple users create files with the same filename Prof. Dr. Christian Baun – 6th Slide Set Operating Systems – Frankfurt University of Applied Sciences – SS2016 11/49 File System Fundamentals Addressing File System Structure Microsoft File Systems Journal Extents COW Defragmentation Hierarchical Directory Structure Directory structures with 2 levels are not always sufficient if many files are stored, it is not sufficient to separate the files by users It is helpful to arrange the files according to their content and/or belonging of projects or applications In a hierarchical tree structure, the users can sort their files and create an unlimited number of directories The directory structures of nearly all modern operating systems operate according to the hierarchical principle Exceptions exist and they are tiny embedded systems Prof. Dr. Christian Baun – 6th Slide Set Operating Systems – Frankfurt University of Applied Sciences – SS2016 12/49 File System Fundamentals Addressing File System Structure Microsoft File Systems Journal Extents COW Defragmentation Linux/UNIX Directory Structure