Fs-Utils: File Systems Access Tools for Userland
Total Page:16
File Type:pdf, Size:1020Kb
Fs-utils: File Systems Access Tools for Userland Arnaud Ysmal ([email protected]) and Antti Kantee ([email protected].fi) September 2009 Abstract consider the connection to the server to be the file sys- tem image. Currently, file system access is done either through File systems can be accessed by kernel drivers (what a kernel driver or a specialized userspace program. we do when using mount) or by userland tools. In this In the former case the file system is mounted with paper, we focus on the latter, or more precisely on user- the mount utility and then accessed with standard land tools using the file system code from the NetBSD userspace tools such as cat. An example of the latter kernel. This principle is implemented on NetBSD and case is the mtools utility suite which allows to access known as RUMP (Runnable Userspace Meta Program) msdos file systems. [1]. We can also use NetBSD kernel code in userspace The NetBSD Runnable Userspace Meta Program on non-NetBSD operating systems, as we show later (RUMP) framework enables the use of kernel file sys- in this paper. tem drivers as part of userspace programs. By building The rest of this paper is organized as follows. The upon RUMP and NetBSD base system utilities such as remainder of this Section will present fs-utils and ex- ls and cp, we have created the fs-utils application suite. plain what we wanted to achieve by creating this new It provides mtools-like file system access without re- set of tools. Section 2 focuses on the way it was de- quiring mount privileges or an in-kernel driver. In con- veloped, discover possible use cases and point out its trast to mtools, fs-utils reuses the kernel file system benefits. Section 3 analyses our approach and Section drivers instead of relying on a userspace reimplemen- 4 presents some common use cases. Section 5 presents tation, supports a total of 12 file systems from NetBSD some recent developments and finally Section 6 con- plus FUSE file systems, and offers the same usage as cludes. the well-known tools (e.g. all of the flags of ls are sup- ported). In addition to running on NetBSD, there has been initial success in running fs-utils on Linux. 1.1 What is fs-utils? This paper explains how these programs were writ- ten and recounts the evolution of the project. It also fs-utils is a set of utilities (including commands like cat, chflags, chmod, cp, du, find, ln, ls, mkdir, mkfifo, shows the benefits and use-cases of fs-utils. mv, rm, rmdir, touch) that give access to a file system image. It was developed on top of RUMP and UKFS libraries. With this, it is possible to use commands 1 Introduction like: $ fsu_ls ffs.img /etc/defaults A file system is a mechanism that allows a user to or- daily.conf pf.boot.conf security.conf ganize his or hers data with files and directories. It monthly.conf rc.conf weekly.conf $ fsu_cat ffs.img /etc/defaults/weekly.conf enables to read a file “foo” in the directory “bar” in- $ echo "foo" | fsu_write ffs.img bar stead of having to remember the block and sector of $ fsu_cat ffs.img bar foo the data. A file system image contains the file system’s in- ternal representation of the file system’s state. A file 1.2 Why fs-utils? system image can either be a regular file or it can be contained on a mass media partition. Additionally, in The aim of this project is to provide a set of tools to the case of NFS or another networked file system, we manipulate file systems, like mtools[2], e2fsprogs[3]. Mtools opens the possibility of accessing and/or and the mountpoint is accessible only within the pro- modifying FAT file systems by offering a set of com- cess which is did the mounting. mands. For instance, it is possible to use the command The following is an example of a program using mdir to list a directory or mren to rename a file. UKFS. It mounts an ffs image, creates a directory fs-utils wants to achieve the same main goal. How- called “dir” and reads 11 bytes from a file. The file ever, unlike mtools, the purpose of fs-utils is to pro- system is then unmounted. vide a generic set of commands which could be used on many different file systems instead of focusing on a fs = ukfs_mount("ffs", "/ffs.img", "/", 0, &args, sizeof(args)); particular one. ukfs_mkdir(fs, "/dir", 0777); ukfs_read(fs, "/a/file", 0, buf, 11); That is why fs-utils has been designed to handle ev- ukfs_release(fs, 0); ery file system supported by RUMP. With this set of tools, anyone can manipulate a file system image he has access to without having to mount 2 Development it. Mounting is, by default, an operation only allowed to root user. However with this type of access, there is 2.1 Origin of fs-utils no need to be root. Possible operations depend on the file system image fs-utils started as a NetBSD project for the Google permissions. The user will be able to read file contents Summer of Code 2008[6]. It was written as a proof and browse directories with read access and modify that file systems servers can work in userland. the files contained in the image if he or she has write permissions to the image. Permissions of files inside the image do not have any effect as the user running 2.2 Development a utility from fs-utils is considered root for operations The first step was to start writing some tools to get in on the file system image. touch with the UKFS library and see what should be added in it to ease its usage. 1.3 RUMP and UKFS Few functions were added, mostly about files status like changing the owner, setting permissions and get- fs-utils was developed on top of the UKFS library ting information. These commands were needed due (User-Kernel File System Library) [4] and RUMP to their use in a large number of commands that were (Runnable Userspace Meta Programs) [5]. planned. The attempt was to write commands completely Regular Standalone Userspace File System File System Using ukfs from scratch. This created problems in terms of main- kernel (runtime) tainability since a new set of commands had to be kernel fs maintained, although it was quite similar to the stan- vfs dards commands. kernel syscall entry While writing these commands, some functions ap- peared to be generic enough to be shared in libraries. user In these functions, we can find file handling and di- application application rectory opening and reading and virtually mounting libukfs and unmounting. It was clear that we needed at least kernel fs two libraries, the first one, called fsu mount, takes care librump of the mounting and unmounting and the second one, called fsu utils, replaces functions from the libc that UKFS is a library which uses RUMP. The UKFS access file systems. interface is based on pathnames and each call is self- After writing both libraries, and doing some exper- contained for ease of use. This means that it is not imentation, the decision to base future versions on ex- necessary to e.g. open a file before reading it. The file isting utility code was made. This means that fs-utils system image is mounted internally within the process mkdir would use the same code as the base system using UKFS. The kernel mount system call is not used, mkdir, and so forth. 2.2.1 fsu mount 2.2.3 Commands fsu mount takes care of everything needed to mount The remaining work was to modify classic programs a file system. It is able to test all supported file sys- to use these libraries. These modifications consist in: tems until it finds one that match the file system used for the current device. The algorithm used to do this • Adding a ukfs declaration is quite simple. It tries to mount the image with each (DECLARE_UKFS(ukfs)) supported file system, one by one, until it find a file and call to the virtual mount system which is able to parse the command line argu- (FSU_MOUNT(argc, argv, ukfs)), ments and mount returns successfully. With this, the provided by the fsu mount library. user does not have to give the file system type of the • Replacing the calls to libc functions to their coun- image. terparts in the fsu utils library or in ukfs: Code needed to mount a file system image with this library: #define stat(path, sb) ukfs_stat(ukfs, path, sb) #include <fsu_mount.h> #define lstat(path, sb) ukfs_lstat(ukfs, path, sb) DECLARE_UKFS(ukfs) • Updating the usage to contain the file system im- int main(int argc, char *argv[]) age path argument { ... FSU_MOUNT(argc, argv, ukfs); These changes were added to the code using ifdef so ... } as to be able to compile the programs with file sys- tem access through RUMP or through libc and system DECLARE_UKFS(ukfs) means that the program calls. will use a struct ukfs which is called ukfs. Then So far, updates of these programs are done manually. FSU_MOUNT(argc, argv, ukfs); will use the standard command line arguments to process the 2.2.4 Installing mount. For example, the arguments “-o ro /dev/wd1a” might be given. This would result in a read-only mount As of now, there are two ways of installing these pro- of /dev/wd1a with the mount handle stored in “ukfs”.