Attack Surface Metrics and Automated Compile-Time OS Kernel Tailoring Anil Kurmus1, Reinhard Tartler2, Daniela Dorneanu1, Bernhard Heinloth2, Valentin Rothberg2, Andreas Ruprecht2, Wolfgang Schroder-Preikschat¨ 2, Daniel Lohmann2, and Rudiger¨ Kapitza3 1IBM Research - Zurich 2Friedrich-Alexander University Erlangen-Nurnberg¨ 3TU Braunschweig Abstract number is expected to grow every year. This is a se- rious problem for system administrators who rely on a The economy of mechanism security principle states distribution-maintained kernel for the daily operation of that program design should be kept as small and simple as their systems. On the Linux distributor side, kernel main- possible. In practice, this principle is often disregarded to tainers can make only very few assumptions on the kernel maximize user satisfaction, resulting in systems supporting configuration for their users: Without a specific use case, a vast number of features by default, which in turn offers the only option is to enable every available configuration attackers a large code base to exploit. The Linux kernel option to maximize the functionality. The ever-growing exemplifies this problem: distributors include a large num- kernel code size, caused by the addition of new features, ber of features, such as support for exotic filesystems and such as drivers and file systems, at an increasing pace, socket types, and attackers often take advantage of those. indicates that the Linux kernel will be subject to ever more A simple approach to produce a smaller kernel is to vulnerabilities. manually configure a tailored Linux kernel. However, the In addition, as a consequence of the development, more than 11;000 configuration options available in recent testing, and patching process of large software projects, Linux versions make this a time-consuming and non-trivial the less a functionality is used, the more likely it is to task. We design and implement an automated approach contain defects. Indeed, developers mostly focus on fixing to produce a kernel configuration that is adapted to a issues that are reported by their user base. As rarely used particular workload and hardware, and present an attack functionalities only account for reliability issues in a small surface evaluation framework for evaluating security im- portion of the user base, this process greatly improves provements for the different kernels obtained. Our results the overall reliability of the software. However, malicious show that, for real-world server use cases, the attack attackers can, and do, still target vulnerabilities in those surface reduction obtained by tailoring the kernel ranges less-often-used functionalities. A recent example from the from about 50% to 85%. Therefore, kernel tailoring is an Linux kernel is an arbitrary kernel memory read and write attractive approach to improve the security of the Linux vulnerability in the reliable datagram sockets (RDS) (CVE- kernel in practice. 2010-3904), a rarely used socket type. If the intended use of a system is known at kernel compilation time, an effective approach to reduce the I. Introduction kernel’s attack surface is to configure the kernel to not include unneeded functionality. However, finding a suitable The Linux kernel is a commonly attacked target. In configuration requires extensive technical expertise about 2011 alone, 148 Common Vulnerabilities and Exposures currently more than 11;000 Linux configuration options, (CVE)1 entries for Linux have been reported, and this and needs to be repeated at each kernel update. There- fore, maintaining such a custom-configured kernel entails 1http://cve.mitre.org/ considerable maintenance and engineering costs. Moreover, while it is widely accepted that making pro- metric based on static call graphs and security models; grams “smaller” improves security, quantitatively measur- examples of metrics satisfying this definition, and a ing security improvements remains a difficult and impor- comparison of the effects of these choices on our tant problem [49]. Existing work on system security often measurements. measures improvements in terms of Trusted Computing • A tool that, given the kernel sources and run-time Base (TCB) reduction, which in practice often translates traces characterizing a use case, produces a small ker- into a measurement of the total number of source lines of nel configuration, taking into account LKMs, which code (SLOC) (e.g., [19, 34]). Although these metrics are includes all kernel functionalities necessary for the sensible (as every line of code can have a vulnerability) workload. and easy to obtain, they can be imprecise. For instance, • An evaluation of the attack surface reduction as well on a given kernel configuration, a large part of the kernel as performance results in the case of a LAMP-based sources will not be compiled, many parts will only be server and a workstation providing access to files via compiled as kernel modules which might never be loaded, NFS. and some functions might simply not be within reach of The remainder of this paper is structured as follows: an attacker. Section II defines the notions of attack surface and attack This paper presents metrics for quantifying the security surface measurement, as well as a set of attack surface of an OS kernel and a tool-assisted approach to automat- metrics that can be used in practice for our evaluation. Sec- ically determine a kernel configuration that enables only tion III presents an overview of the tailoring process, and kernel functionalities that are actually necessary in a given the implementation of the underlying automated kernel- scenario. Although it is easy to quantify the size of the configuration-tailoring tool. Section IV evaluates the attack resulting kernel binaries, this is not convincing evidence surface reduction and performance of such an approach in that the resulting kernel indeed presents less of an attack two use cases and with several attack surface reduction surface to potential attackers. Hence, after defining what metrics. These results are then discussed in Section V. attack surface means, we quantify the security gains in Section VI presents related work. The paper concludes in two distinct security models in terms of attack surface Section VII. reduction. The first security model considers that the entire kernel can be subject to attacks and is therefore a good reference for comparison to previous work, whereas the II. Security Metrics second considers the scenario of a restricted attacker, and is a good reference for evaluating the security improvements In this section, we present two distinct security models, of configuration tailoring in the context of unprivileged and, for each of them, security metrics (attack surface local attackers. Our measurements take into account the measurements) which we use in Section IV to evaluate static call graph of the kernel and the possible entry points and quantify the security of a running Linux kernel. The of the attacker to provide a more accurate comparison. dependence between notions defined or used in this section Our automated kernel-tailoring approach builds on our are summarized in Figure 1. previous work [51], and extends it with multiple improve- ments, including loadable kernel module (LKM) support. A. Preliminary definitions When compared to other hardening solutions, a notable advantage of the kernel-configuration tailoring approach is Definition 1 (Call graph). A call graph is a directed graph that it makes no changes to the source code of the kernel: (F;C), where F ⊆ N is the set of nodes and represents the therefore, it is impossible to introduce new defects into the set of functions as declared in the source of a program, kernel source. This approach uses run-time traces as input and C ⊆ F × F the set of arcs, which represent all direct for deducing a suitable kernel configuration, and we show and indirect function calls. We denote the set of all call it to work equally well in different use cases. We detail graphs by G . the use our tool to tailor a “Linux, Apache, MySQL and PHP (LAMP) stack” kernel on server hardware, as well as In practice, static source code analysis at compile time a network file system (NFS) running on a workstation. We (that takes all compile-time configuration options into obtain comparative measurements of the tailored kernels account) is used to obtain such a call graph. that show that configuration-tailoring incurs no overhead Definition 2 (Entry and barrier functions). A security and no stability issues, while greatly reducing the attack model defines a set of entry functions E ⊆ F, which surface in both security models. corresponds to the set of functions directly callable by The major contributions of this paper are: an attacker, and a set of barrier functions X ⊆ F, which • A definition of an attack surface and an attack-surface corresponds to the set of functions that, even if reachable, Application Application Application Application Security Entry and barrier Attack surface model functions (privileged) (unprivileged) (privileged) (unprivileged) Attack surface measurement LKM LKM System call interface System call interface Program source Call graph: Attack surface (on-demand (on-demand and configuration functions and calls metric loadable) loadable) Core Kernel LKM LKM Core Kernel LKM LKM (driver) (driver) LKM Hardware interface LKM Hardware interface Figure 1. Dependencies between notions de- (other) (other) attacker fined in this section. Hardware Hardware attack surface partial a.s. running kernel would prevent an attacker from progressing further into the call graph. Figure 2. On the left, the GENSEC model. On the right, the ISOLSEC model. E would typically be the interface of the program that is exposed to the attacker, whereas X would typically be the set of functions that perform authorization for privileges that the attacker is not assumed to have in the security specific code, because this code cannot be exercised by model (e.g., administrator privileges). an attacker, is already an improvement in precision. We now consider the case of the Linux kernel.