Study of File System Evolution Swaminathan Sundararaman, Sriram Subramanian Department of Computer Science University of Wisconsin {swami, srirams} @cs.wisc.edu Abstract File systems have traditionally been a major area of file systems are typically developed and maintained by research and development. This is evident from the several programmer across the globe. At any point in existence of over 50 file systems of varying popularity time, for a file system, there are three to six active in the current version of the Linux kernel. They developers, ten to fifteen patch contributors but a single represent a complex subsystem of the kernel, with each maintainer. These people communicate through file system employing different strategies for tackling individual file system mailing lists [14, 16, 18] various issues. Although there are many file systems in submitting proposals for new features, enhancements, Linux, there has been no prior work (to the best of our reporting bugs, submitting and reviewing patches for knowledge) on understanding how file systems evolve. known bugs. The problems with the open source We believe that such information would be useful to the development approach is that all communication is file system community allowing developers to learn buried in the mailing list archives and aren’t easily from previous experiences. accessible to others. As a result when new file systems are developed they do not leverage past experience and This paper looks at six file systems (Ext2, Ext3, Ext4, could end up re-inventing the wheel. To make things JFS, ReiserFS, and XFS) from a historical perspective worse, people could typically end up doing the same (between kernel versions 1.0 to 2.6) to get an insight on mistakes as done in other file systems. the file system development process. We study the source code of these file systems over years to observe We chose to look at the file systems in the Linux kernel changes in complexity of the system and relate these for the following three reasons: (1) Linux is open metrics to the features and stability of file systems. We source, implying that we can get access to the source identified that only a few components in file systems code of the file systems, without which it is almost are difficult to get right the first time. Also, file impossible to study their evolution. (2) As compared to system developers do not learn from each others other systems Linux support a large variety of file mistake. As a result they end up introducing the same systems. Most importantly, some of these have existed bugs in file systems. for a long time (10-15 years). Hence, it is possible to look at the file systems during different stages of its 1. Introduction lifetime and derive conclusions from them. (3) All communications (discussion, patches, bugs, etc.) In today’s world everything revolves around data. Data between file system developers can be obtained from is critical for day-to-day operation of any system. Data the mailing list and are publicly available in many management is taken care of by file systems which websites [16]. The bugzilla database, CVS, and git reliably store data on disks. Users typically have varied repositories are also publicly accessible. This allowed requirements ranging from scalability, availability, us to get insight into the development process of file fault-tolerance, performance guarantees in business systems in an open source environment. It would environment to small memory footprints, security, definitely be interesting to look at file systems in other reliability in desktop environments. This has driven the operating systems like Solaris, BSD. or Windows to file system community to develop a variety of systems compare and contrast it with file systems developed in that cater to different user requirements. Linux. An open source environment like Linux has around 50 In this paper, we look at file system code in the Linux different file systems that provide different guarantees, kernel (version numbers 1.0 to 2.6). This covers 14 targeting varied media. In an open source environment, years of kernel development (1994 to 2008). We also look at patches, bug reports and other communication that is available in each of the file system mailing list. frequently changed components. Section 7 discusses We have built a tool that automatically crawls the common bugs in file systems code and looks at turn mailing archives and extracts higher level information around time for bugs in XFS. We talk about the from them. We did not use Bugzilla database [9] as it is implications of this work and how open source restricted to Redhat releases and they are only a subset community should adapt itself for more efficient of the changes that are available in the mailing lists. We functioning in Section 8. We comment on related work did not use the CVS repositories for the file systems as in Section 9 and conclude in Section 10. they didn't have complete information. The repositories are solely used by the file system maintainer to 2. Problems in the Open Source Model consolidate the patches that developers submit over time and eventually the kernel maintainer picks up the To our knowledge, there has been no prior work on patches from each of the maintainers and integrates it to understanding the file system development process, the linux kernel. We already have this information by especially in the Linux world. Previous work on looking at the file system code between kernel releases software evolution looked at software repositories like and the mailing list. As far as we know, a study of CVS, etc[3,4]. We cannot apply the same kind of similar nature does not exist in the file system analysis because the development model in Linux is community. very different. There is to no easy way to get this information in the Linux world. We comment briefly The contributions of this paper are as follows: about the file system development process in Linux to understand the difficulties involved in it. • We have developed tools to systematically mine information available in mailing lists [14,15,16,18] and source code repositories [10, 11] so that developers can easily query to know file system properties or bugs. • As part of our preliminary analysis, we provide insights into the development process, common problem areas, and stability of file systems. • Identification of common bugs in file systems. This could serve as a reference document to file system developers to check if their file system are immune to the common bugs. From our study, we found that out of 60 or so file systems only a handful of them have lots of features and most widely developed and used. Most of the file systems are designed well and do not undergo design level changes during their lifetime. In a purely open source file system, many developers do contribute to the development of the file system and in contrast file systems that have been started. We also found that developers across file system do not communicate Figure 1: Linux kernel development process effectively and as a result end up repeating the same mistakes in their file systems. Figure 1 shows the development process in linux. From Rest of the paper is organized as follows. Sections 2 the figure, one can see that most of the communication explains the difficulties in extracting higher level between the developers happens through the mailing information in file system development in Linux. lists. All the discussion, patches, and issues get buried Section 3 gives a broad overview of file system in the in the mails and finally end up getting archived at some latest Linux kernel and justifies our choice of file public website (e.g., [16]). Unlike the software systems included in this study. Section 4 analyzes the development process employed in other open source trends in code change and complexity across file projects, the Linux model is very different in the sense systems. Section 5 and Section 6 looks at the code that there is no proper CVS repository where we can contributions of developers in file systems and track every change in the file system. Everything has to Figure 2. Lines of Code for all file systems in Linux 2.6.27 be inferred from the mails in the mailing lists. Although study. The Ext2 file system was chosen because it is a there are search engines that help extract information mature file system that was derived from the Ext file quickly from these archives, there are not tools that can system and has been around for more than 15 years. infer trends and features from these mails. To our Also, Ext2 was the default file system for Linux for knowledge, existing tools to understand the open source over 10 years which indicates its stability. We chose the project [7] etc. do not work well for extracting Ext3 and Ext4 because they were derived from Ext2. information from the file systems in Linux. To Ext3 is a mature file system (it is now the default file summarize, file system developers typically system in Linux) and the development in Ext4 started communicate within their community (file system recently (in 1995). The Ext2/3/4 file system were mailing list) about the issues and fixes and there is no started by the open source community. ReiserFS was easy way to get this information to other file system chosen as it was initially conceived by a single person developers. As a result, we believe that file system (Hans Reiser) and was slowly adopted by the open developers do not learn from each other’s mistakes and source community.