Virtual file system The Linux Storage Stack Diagram A virtual file system (VFS) or virtual filesystem switch version 4.0, 2015-06-01 outlines the Linux storage stack as of Kernel version 4.0 is an abstraction layer on top of a more concrete file sys- ISCSI USB mmap Fibre Channel Fibre over Ethernet Fibre Channel Fibre Virtual Host Virtual FireWire (anonymous pages) Applications (processes) tem. The purpose of a VFS is to allow client applications LIO malloc vfs_writev, vfs_readv, ... ... stat(2) read(2) open(2) write(2) chmod(2) to access different types of concrete file systems in a uni- VFS tcm_fc sbp_target tcm_usb_gadget tcm_vhost tcm_qla2xxx iscsi_target_mod Block-based FS Network FS Pseudo FS Special ext2 ext3 ext4 xfs NFS coda proc purpose FS target_core_mod Direct I/O sysfs Page form way. A VFS can, for example, be used to access (O_DIRECT) btrfs ifs iso9660 smbfs ... tmpfs pipefs futexfs ramfs cache target_core_file gfs ocfs ... ceph usbfs ... devtmpfs local and network storage devices transparently without Stackable FS target_core_iblock ecryptfs overlayfs unionfs FUSE userspace (e.g. sshfs) the client application noticing the difference. It can be target_core_pscsi target_core_user network stackable (optional) struct bio - sector on disk used to bridge the differences in Windows, Mac OS and BIOs (block I/Os) Devices on top of “normal” BIOs (block I/Os) - sector cnt block devices - bio_vec cnt drbd LVM - bio_vec index device mapper mdraid - bio_vec list dm-crypt dm-mirror ... dm-cache dm-thin bcache dm-raid dm-delay Unix filesystems, so that applications can access files on userspace local file systems of those types without having to know BIOs BIOs Block Layer BIOs what type of file system they are accessing. I/O scheduler blkmq Maps BIOs to requests multi queue hooked in device drivers noop Software (they hook in like stacked ... queues cfq devices do) A VFS specifies an interface (or a “contract”) between deadline Hardware Hardware dispatch ... dispatch the kernel and a concrete file system. Therefore, it is easy queue queues Request Request BIO to add support for new file system types to the kernel based drivers based drivers based drivers simply by fulfilling the contract. The terms of the con- Request-based device mapper targets tract might change incompatibly from release to release, dm-multipath SCSI mid layer sysfs scsi-mq /dev/zram* /dev/rbd* /dev/mmcblk*p* /dev/nullb* /dev/vd* /dev/rssd* /dev/skd* (transport attributes) SCSI upper level drivers which would require that concrete file system support be /dev/nvme*n* /dev/sda /dev/sd* ... /dev/rsxx* Transport classes scsi_transport_fc /dev/sr* /dev/st* zram rbd mmc null_blk virtio_blk mtip32xx nvme skd rsxx recompiled, and possibly modified before recompilation, scsi_transport_sas scsi_transport_... network memory to allow it to work with a new release of the operating SCSI low level drivers libata megaraid_sas qla2xxx pm8001 iscsi_tcp virtio_scsi ufs ... system; or the supplier of the operating system might ahci ata_piix ... aacraid lpfc mpt3sas vmw_pvscsi make only backward-compatible changes to the contract, network HDD SSD DVD LSI Qlogic PMC-Sierra Micron nvme stec para-virtualized virtio_pci mobile device drive RAID HBA HBA SCSI flash memory PCIe card device device Adaptec Emulex LSI 12Gbs VMware's SD-/MMC-Card so that concrete file system support built for a given re- RAID HBA SAS HBA para-virtualized IBM flash Physical devices SCSI adapter lease of the operating system would work with future ver- The Linux Storage Stack Diagram http://www.thomas-krenn.com/en/wiki/Linux_Storage_Stack_Diagram sions of the operating system. Created by Werner Fischer and Georg Schönberger License: CC-BY-SA 3.0, see http://creativecommons.org/licenses/by-sa/3.0/ The position of the VFS layer within various parts of the Linux 1 Implementations kernel's storage stack.[1] One of the first virtual file system mechanisms on Unix- including Mac OS X. like systems was introduced by Sun Microsystems in Other Unix virtual file systems include the File System SunOS 2.0 in 1985. It allowed Unix system calls to ac- cess local UFS file systems and remote NFS file systems Switch in System V Release 3, the Generic File System in Ultrix, and the VFS in Linux. In OS/2 and Microsoft transparently. For this reason, Unix vendors who licensed the NFS code from Sun often copied the design of Sun’s Windows, the virtual file system mechanism is called the Installable File System. VFS. Other file systems could be plugged into it also: there was an implementation of the MS-DOS FAT file The Filesystem in Userspace (FUSE) mechanism allows system developed at Sun that plugged into the SunOS userland code to plug into the virtual file system mech- VFS, although it wasn't shipped as a product until SunOS anism in Linux, NetBSD, FreeBSD, OpenSolaris, and 4.1. The SunOS implementation was the basis of the VFS Mac OS X. mechanism in System V Release 4. In Microsoft Windows, virtual filesystems can also be im- John Heidemann developed a stacking VFS under SunOS plemented through userland Shell namespace extensions; 4.0 for the experimental Ficus file system. This de- however, they do not support the lowest-level file system sign provided for code reuse among file system types access application programming interfaces in Windows, with differing but similar semantics (e.g., an encrypting so not all applications will be able to access file systems file system could reuse all of the naming and storage- that are implemented as namespace extensions. KIO and management code of a non-encrypting file system). Hei- GVFS/GIO provide similar mechanisms in the KDE and demann adapted this work for use in 4.4BSD as a part of GNOME desktop environments (respectively), with sim- his thesis research; descendants of this code underpin the ilar limitations, although they can be made to use FUSE file system implementations in modern BSD derivatives techniques and therefore integrate smoothly into the sys- 1 2 5 REFERENCES tem. 3 See also • 9P (protocol) – a distributed file system protocol that 2 Single-file virtual file systems maps directly to the VFS layer of Plan 9, making all file system access network-transparent Sometimes Virtual File System refers to a file or a group of files (not necessarily inside a concrete file system) that acts as a manageable container which should provide the functionality of a concrete file system through the usage 4 Notes of software. Examples of such containers are SolFS or a single-file virtual file system in an emulator like PCTask 1. ^ Emulation on Amiga Comparison between PCX or so-called WinUAE, Oracle’s VirtualBox, Microsoft’s and PCTask, Amiga PC emulators. Virtual PC, VMware. The primary benefit for this type of file system is that it 2. ^ See also This article explaining how it works PC- is centralized and easy to remove. A single-file virtual Task. file system may include all the basic features expected of any file system (virtual or otherwise), but access to the 3. ^ Help About WinUAE (See Hardfile section). internal structure of these file systems is often limited to programs specifically written to make use of the single- 4. ^ Help About WinUAE (See Add Directory section) file virtual file system (instead of implementation through a driver allowing universal access). Another major draw- back is that performance is relatively low when compared to other virtual file systems. Low performance is mostly 5 References due to the cost of shuffling virtual files when data is writ- ten or deleted from the virtual file system. [1] Werner Fischer; Georg Schönberger (2015-06-01). “Linux Storage Stack Diagram”. Thomas-Krenn.AG. Re- trieved 2015-06-08. 2.1 Implementation of single-file virtual filesystems • Vnodes: An Architecture for Multiple File System Direct examples of single-file virtual file systems include Types in Sun UNIX emulators, such as PCTask and WinUAE, which encap- sulate not only the filesystem data but also emulated disk • Linux kernel’s Virtual File System layout. This makes it easy to treat an OS installation like any other piece of software—transferring it with remov- • Rodriguez, R.; M. Koehler; R. Hyde (June 1986). able media or over the network. “The Generic File System”. Proceedings of the USENIX Summer Technical Conference. Atlanta, Georgia: USENIX Association. pp. 260–269. 2.1.1 PCTask • Karels, M.; M. K. McKusick (September 1986). The Amiga emulator PCTask emulated an Intel PC “Towards a Compatible File System Interface”. 8088 based machine clocked at 4.77MHz (and later an Proceedings of the European UNIX Users Group 80486SX clocked at 25 MHz). Users of PCTask could Meeting. Manchester, England: EUUG. pp. 481– create a file of large size on the Amiga filesystem, and 496. this file would be virtually accessed from the emulator as if it were a real PC Hard Disk. The file could be format- • ted with the FAT16 filesystem to store normal MS-DOS Heidemann, John (1995). Stackable Design of File or Windows files.[1][2] Systems (Technical report). UCLA. CSD-950032. • The Linux VFS, Chapter 4 of Linux File Systems 2.1.2 WinUAE by Moshe Bar (McGraw-Hill, 2001). ISBN 0-07- 212955-7 The UAE for Windows, WinUAE, allows for large single files on Windows to be treated as Amiga file systems. In • Chapter 12 of Understanding the Linux Kernel by [3] WinUAE this file is called a hardfile. Daniel P. Bovet, Marco Cesati (O'Reilly Media, UAE could also treat a directory on the host filesystem 2005). ISBN 0-596-00565-2 -- (Windows, Linux, Mac OS, AmigaOS) -- as an Amiga filesystem.[4] • The Linux VFS Model: Naming structure 3 6 External links • AVFS - A Virtual File System for mounting com- pressed or remote files • fs-driver Ext2 Installable File System for Microsoft Windows • Anatomy of the Linux file system by M.