Universidad de Los Andes

Tesis de Maestr´ıa

Hardening Linux Processes Extending Grsecurity to Integrate System Call Filters and Namespaces

David Derby Cardona

Facultad de Ingenier´ıa Departamento de Ingenier´ıade Sistemas y Computaci´on June 2016 Universidad de Los Andes

Tesis de Maestr´ıa

Hardening Linux Processes Extending Grsecurity to Integrate System Call Filters and Namespaces

David Derby Cardona

Asesor: Sandra Rueda Rodr´ıguez Jurados: Rafael G´omezD´ıaz Fabian Molina Molina

Facultad de Ingenier´ıa Departamento de Ingenier´ıade Sistemas y Computaci´on June 2016 Abstract

The area of Linux sandboxing has seen various developments in recent years with the intro- duction of operating system containers and the ever present need to harden the security of applications. Two of the more prominent technologies that have been used when creating sandboxes are namespaces and system call filters. Whilst these technologies have been ef- fective for creating sandboxes, they are limited in that they require a developer to integrate them into their software. This work proposes to use these two technologies to enforce the Principle of Least Privilege on every process on a system. The solution extends a grsecurity hardened Linux kernel and allows the user to define security policies for each process which permit them to behave as intended. The presented results demonstrate the effectiveness of the extended Linux kernel and its impact on performance. The results provide a basis that may be built upon to deliver a comprehensive solution that would be appealing for use in real world environments.

1 Contents

Abstract 1

Index of Figures 4

Index of Tables 5

1 Introduction 1

2 Context and Problem Description 3 2.1 Linux ...... 3 2.1.1 Processes ...... 3 2.1.2 Permissions ...... 4 2.1.3 Authorisation ...... 5 2.2 Least privilege ...... 5 2.3 Limitations of existing solutions ...... 6 2.3.1 Type enforcement solutions ...... 6 2.3.2 Other solutions ...... 6 2.3.2.1 Android ...... 7 2.4 The problem ...... 8 2.4.1 General objective ...... 8 2.4.2 Specific objectives ...... 8

3 Related Work 9 3.1 Integrated kernel tools ...... 9 3.2 User space tools ...... 9 3.3 Other isolation work ...... 10 3.4 Access control technologies ...... 10 3.5 Summary ...... 10

4 Proposal 16 4.1 Design ...... 16 4.1.1 Constrained access ...... 17 4.1.2 Enforcing constrained execution ...... 18 4.1.3 Default policies ...... 18 4.1.3.1 IPC namespaces ...... 18 4.1.3.2 Network namespaces ...... 19

2 CONTENTS 3

4.1.3.3 Mount namespaces ...... 19 4.1.3.4 PID namespaces ...... 19 4.1.3.5 User namespaces ...... 19 4.1.3.6 UTS namespaces ...... 20 4.1.3.7 Secure computing (seccomp) ...... 20 4.1.4 Policy creation ...... 20 4.2 Limitations ...... 21

5 Implementation 22 5.1 Components ...... 22 5.1.1 Grsecurity ...... 22 5.1.1.1 Gradm ...... 23 5.1.1.2 Policy file structure ...... 23 5.1.1.3 Gradm special roles ...... 23 5.2 Extending grsecurity ...... 24 5.2.1 Extension to gradm ...... 24 5.2.1.1 Implementation of new policy types ...... 26 5.2.2 Extension to the grsecurity hardened Linux kernel ...... 27 5.2.2.1 Implementation of namespaces ...... 27 5.2.2.2 Implementation of system call filters ...... 28 5.3 Evaluation ...... 28 5.3.1 System call filtering ...... 28 5.3.2 Namespaces ...... 29 5.3.3 Performance ...... 29 5.4 Summary ...... 30

6 Conclusions and Future Work 38 6.1 Conclusions ...... 38 6.2 Future work ...... 38 6.2.1 Feature completeness ...... 39 6.2.2 Blacklists ...... 39 6.2.3 Namespace bind mounts ...... 39 6.2.4 Further namespace customisation ...... 39 6.2.5 Further seccomp customisation ...... 40 6.2.6 Policy file separation ...... 40 6.2.7 Learning mode ...... 41 6.2.8 Code refactoring ...... 41

A Gradm patch 42

B Kernel patch 53

Bibliography 59 List of Figures

2.1 Execution of a command in Linux ...... 4 2.2 Access control representation ...... 4 2.3 The Android execution environment ...... 7

4.1 Creation of constrained processes ...... 17

5.1 Structure of the grsecurity policy file ...... 24 5.2 Flex code to match namespace and system call filter rules ...... 25 5.3 Bison code to handle tokens ...... 25 5.4 Example of a policy in the policy file ...... 26 5.5 Sequence of interaction of grsecurity’s RBAC system ...... 31 5.6 The sequence of interaction between a process, execve() and stored policies . . 32 5.7 The C source code of the “rootshell” program ...... 32 5.8 Demonstration of “rootshell” malware succeeding ...... 33 5.9 Finding setuid programs with the find command ...... 33 5.10 Demonstration of “rootshell” malware failing ...... 33 5.11 Demonstration of a network namespace ...... 34 5.12 Demonstration of an IPC namespace ...... 35 5.13 Demonstration of a UTS namespace ...... 35 5.14 Tests run on a standard grsecurity enhanced Linux kernel ...... 36 5.15 Tests run on a grsecurity Linux kernel including the extension presented in this work...... 37

6.1 Policy rule for an IPC namespace with a bind mount ...... 39 6.2 Policy rule for a network namespace with a virtual network link ...... 40 6.3 Policy rule for a system call filter with a thread synchronisation flag ...... 40

4 List of Tables

3.1 Isolation through virtualisation of operating systems ...... 11 3.2 User space tools for sandboxing ...... 12 3.3 Linux Security Modules ...... 13 3.4 Other security facilities provided by mainline Linux ...... 14 3.5 Out-of-tree kernel patches ...... 15

5 Chapter 1

Introduction

The use of Linux based operating systems has grown in server, desktop/laptop and mobile deployments [1]. Linux usage of the Internet’s web servers is significant as it is the most popular operating system with hosting companies. Eight out of ten of the most reliable com- pany websites were hosted on Linux machines as reported by Netcraft in April 2016 [2]. The desktop and laptop market has seen growth with the Linux based ChromeOS, the operating system which runs on Chromebooks, recently outselling Apple’s range of Macs for the first time in the United States of America [3] [4]. In the mobile sector, Linux remains the dominant player with the Linux based Android operating system holding 80% market share [5]. The significant presence that Linux holds in the industry, highlights the need to research new ways to keep these systems secure. Malicious programs have frequently been developed for Windows systems, however the increased popularity of other operating systems including Linux, has led to an increase in those being targeted too. This rise shows that attackers are not only interested in targeting single clients but also the infrastructure that they use [6]. Although the volume of attacks targeting Linux is lower than those targeting Windows, the number of attacks against Linux has considerably increased, meaning that Linux system administrators have an increased responsibility to consider additional defence mechanisms to protect their systems. Unix based systems traditionally use discretionary access control (DAC) as a means of restricting access to objects based on the identity of subjects and/or groups to which they belong [7]. When a program is executed, it uses default permission inheritance which means that the created process will run with the same permissions as the user that started it. The process has the same visibility and access rights to the system as the user would have, thus each process exposes a surface that would be attractive to an attacker who may take advantage by exploiting a vulnerability in a program or by means of malware executed by that user. A user does not need to have the same system access rights as a privileged user such as root for an attacker to cause damage or to obtain confidential information. Existing solutions which improve upon the traditional DAC restriction on computer pro- grams include the use of: role based access control (RBAC), security modules or enhancements which provide mandatory access control (MAC), control of operating system privileges, op- erating system capabilities, memory page protections, integrity protection and a variety of methods to create a sandbox. These solutions often present their own set of challenges which impact their widespread adoption.

1 CHAPTER 1. INTRODUCTION 2

Recent additions to the Linux kernel provide new facilities that allow a software developer to isolate processes and restrict their behaviour. Given that these isolation and restriction techniques are relatively new, much of the large back catalogue of software already available for Linux does not yet take advantage of them. In order to do so, each software package would have to be analysed and then modified. A software developer may not necessarily take the time and effort to secure the software that they have already written or that they may write in future. This work explores the security features provided by the Linux kernel and proposes an original solution that complements other methods of locking down the runtime environment of all Linux processes. It aims to help Linux distribution maintainers and system administrators to implement policies that restrict the environment of existing unmodified software. The proposed solution not only helps to prevent attacks that originate from vulnerabilities in authorised software, including zero-day threats, but can also prevent attacks from purpose- built malware. The rest of the document is organised as follows: Chapter 2 presents the problem in depth; Chapter 3 reviews related work; Chapters 4 and 5 propose a solution that addresses the limitations of previous work; finally Chapter 6 analyses the proposed solution and its results. Chapter 2

Context and Problem Description

A typical computer program that runs on Linux: is granted visibility to the global file system (relying on DAC to restrict access), is able to obtain a list of every process running on the system, has access to network devices, has access to a variety of interprocess communication (IPC) systems, and is capable of making any system call. This defies the Principle of Least Privilege, also called the Principle of Least Authority, which implies that every program must only be able to access the resources that are necessary for its legitimate purpose [8].

2.1 Linux

2.1.1 Processes Like other Unix based operating systems, Linux assigns a unique user identifier (UID) to each user. It also handles groups of users, assigning a group identifier (GID) to each group. Each user belongs to a single primary group and may also be a member of secondary groups, thus each user has a unique UID, one primary GID, and zero or more secondary GIDs. Linux assigns a unique process identifier (PID) and a set of credentials to each running process. These credentials include a real UID and a real GID which determine who owns the process. Other credentials include an effective user identifier (EUID) and an effective group identifier (EGID). The EUID and EGID are used by the kernel to determine the permissions that the process will have when accessing shared resources. Unlike other operating systems based on Unix, Linux also has credentials for file system user and group identifiers (FSUID and FSGID) which determine permissions for accessing files [9]. If a user runs a program with the setuid file mode bit set, the EUID of the process will change to be the UID of the owner of that file. Linux uses the setuid bit to handle temporal domain transitions; a process temporarily runs within a protection domain different from the owner’s domain, with a different set of permissions. Files, processes and other resources are assigned the UID and GID of the process that created them [10]. Figure 2.1 illustrates the sequence of events that takes place when a user runs a command in Linux. The user first runs a command (cmd) from the shell and a fork() system call is made to create a duplicate of the calling process, in this case the user’s shell. This duplicate process has the same UID and EUID of the initial shell but it has a new PID. The new process then makes an execve() system call to replace the program code with the code associated with the command. It is assumed that the program does not have the setuid file mode bit set meaning

3 CHAPTER 2. CONTEXT AND PROBLEM DESCRIPTION 4 that the EUID is that of the user which started the command. Linux may also use the clone() system call to create a new process in a similar manner to fork(). Unlike fork(), it allows the child process to share parts of its execution context with the calling process.

Figure 2.1: Execution of a command in standard Linux. The user shell forks a new process, with the same UID but a new PID, then the new process replaces the old code for the code corresponding to the invoked command.

2.1.2 Permissions Linux implements a simplified version of an access control list (ACL) to represent permissions assigned to resources. In this model, every resource has an associated list of the subjects that have access to it and the permissions of those subjects. Figure 2.2 illustrates this model.

Figure 2.2: Access control representation. The system keeps a list of subjects for every object (resource) with permissions to operate on that object.

The classic model of permissions in Unix and Linux does not allow for an ACL of individual users to be applied to a resource. Permissions are limited to three categories:

• The resource owner, represented by a UID.

• The resource group, represented by a GID. These permissions apply to all members of this group.

• The other category. These permission apply to everyone else. That is to say everyone who is not the resource owner or a member of the resource group. CHAPTER 2. CONTEXT AND PROBLEM DESCRIPTION 5

Possible permissions that can be applied to each category are read, write and execute, represented as r, w and x respectively. For example, a file with the permissions r-xr-xr--, authorises the owner to read and execute the file (the first group of three: r-x), authorises members of the file’s group to read and execute the file (the second group of three: r-x) and authorises the rest of users in the system to only read the file (the last group of three: r--). When a process creates a new file it defines the file’s permissions, thus establishing who can access that file and how [10].

2.1.3 Authorisation Algorithm 1 shows pseudocode to evaluate whether a given user is authorised to act on a file.

Algorithm 1 Permission Evaluation 1: if UIDuser == UIDfile then 2: permissions ← permissionsP erCategory(owner) 3: else if anyGIDuser == GIDfile then 4: permissions ← permissionsP erCategory(group) 5: else 6: permissions ← permissionsP erCategory(others) 7: end if 8: 9: if requestedAction in permissions then 10: authorise 11: end if

Linux makes authorisation decisions based on the UIDs and GIDs of the entities requesting access (users or processes) and the targets of those requests (resources). This means that a Linux process runs within the same protection domain (i.e. with the same permissions) associated with the user that owns the process or with the user that owns the executable file, thus a new process will often run with more permissions than those that are actually needed.

2.2 Least privilege

The Principle of Least Privilege, a design principle for information protection mechanisms [11], states that every program and every privileged user of the system should operate using the least set of privileges necessary to complete the job [8]. Many operating systems, including Linux, do not adhere to this principle. Processes on Unix and Linux inherit the UID from the user that starts a process or from the file owner if the setuid file mode bit is set. All processes have the same visibility and set of access rights to the entire system as its owner or the EUID in effect. In addition to this, traditionally there is no restriction on the system calls that a process is allowed to execute. Whilst there have been attempts to restrict which system calls a process may execute, none of these solutions enforce system call restrictions on the entire system. These defects create an attack surface that is bigger than what is actually necessary. This escalates the number of attack opportunities. An increase of exposure of a system’s surface will present a higher number of attack opportunities, increasing the likelihood that it will be a target of an attack. System security may be improved by reducing its attack surface [12]. CHAPTER 2. CONTEXT AND PROBLEM DESCRIPTION 6

A system’s attack surface may be described by three abstract dimensions: targets and enablers, channels and protocols, and access rights. Targets are the resources that an attacker aims to take control of and enablers are other resources that an attacker may use to exe- cute the attack. Channels and protocols are the means of communication that an attacker uses to control resources. Access rights are what constrains the attacker’s ability to control resources [12]. The principle of least privilege addresses the access rights abstract dimension of a system’s attack surface. By imposing access rights on resources that do not already implement them, a system’s attack surface be can be reduced.

2.3 Limitations of existing solutions

There are a variety of solutions available for Linux which aim to apply the principle of least privilege by the use of type enforcement. Type enforcement gives priority to mandatory access control over discretionary access control [13]. Existing solutions however, are limited in certain aspects.

2.3.1 Type enforcement solutions Existing type enforcement solutions have no notion of namespaces. Namespaces are a rel- atively new concept to Linux which have been recently added to the mainline kernel. A namespace wraps a global system resource in an abstraction that makes it appear to the processes within the namespace that they have their own isolated instance of the global re- source [14]. Changes to the global resource are visible to other processes that are members of the namespace but are invisible to all other processes [14]. Another recent addition to the mainline Linux kernel that is not used by existing type enforcement solutions is SECure COMPuting with filters, commonly known as seccomp filters or seccomp-bpf. Seccomp filtering provides a way of constricting a process to a subset of system calls that it may use [15]. There is no type enforcement solution at present which takes advantage of the benefits provided by seccomp filters.

2.3.2 Other solutions Existing software solutions that do utilise namespaces or seccomp filters tend to be either focused on providing isolation for operating system level (OS level) virtualisation technolo- gies [16] or on providing a sandbox for individual programs and applications [17] [18] [19] [20]. Process sandboxing is often implemented independently for each program [17] [18]. The pro- cedure for using namespaces or seccomp filters to sandbox a process entails that a software developer implements these mechanisms into each of the programs that they write. As this practice is completely optional, there is no guarantee that a software developer will imple- ment these security features, thus leaving many programs with broad access rights to system resources. Solutions such as Firejail [19] and Mbox [20] allow a user to run any program they desire inside a sandbox by prefixing the command to run the program with another command that calls the sandbox program. Although these solutions are effective at ensuring the correct behaviour of any given program, they are user space programs which must be manually invoked CHAPTER 2. CONTEXT AND PROBLEM DESCRIPTION 7 by the user, therefore they do not enforce the principle of least privilege. The implications of this are that these solutions do not function automatically on a system-wide level and they will not mitigate attacks that originate from malware.

2.3.2.1 Android Modern mobile devices frequently use an operating system capable of pre-emptive multitasking meaning that they can handle multiple processes simultaneously. Android is one of these operating systems. It uses a Linux kernel [21] and is the most widely used operating system for mobile devices [5]. Unlike traditional Unix or Linux systems, mobile devices are typically used by only one user, the device’s owner. This would mean that each of the processes run by the device owner would all share the same UID. Android however, runs each application with a different UID meaning that each process has different access rights. Figure 2.3 illustrates the Android environment, where each application process runs with a different UID.

Figure 2.3: The Android execution environment. Each Android application process runs with a unique UID as if the processes were executed by different users.

Android also has other mechanisms to control access to resources but this particular feature is of interest because it enables the system to assign different access rights to different applications. This is important because although all applications are run on behalf of the same user, each application may have been developed by different software developers, has a different job, and should have access to different resources. Although Android’s mechanism of running each application with a different UID is a novel approach to solve the problem, implementing this in traditional Linux systems may not be so straightforward. The software development model for Android is quite different from that of traditional Linux. Android has its own model where developers create applications which are submitted to an application store. This allows Google, the company that develops Android, to enforce certain requirements that software must meet before it can be released. This differs from traditional Linux software development which has a much more open model. In this model, a developer may choose to develop software which could be a program or an application, there are no constraints set by a third party of what the software must or must not do, and the developer may choose to distribute it themselves. Whilst it is also possible to do this with Android, the user must enable an option to allow installation of applications from unknown sources and accept a disclaimer which states that the device will be more vulnerable to attacks [22] [23]. CHAPTER 2. CONTEXT AND PROBLEM DESCRIPTION 8

2.4 The problem

Many operating systems run many applications which may have been developed by different software developers or companies. These applications should run with only the access rights required for their legitimate purpose, however this is not always the case. Although traditional Linux systems will not automatically launch applications with a unique UID as Android does, some applications will be configured with their own UID when they are installed. Furthermore, some system administrators will manually configure other applications to run with their own unique user identifiers. This behaviour however, is not enforced and may leave a system vulnerable. Recently developed technologies enable software developers to create sandboxes to enhance the security of their programs and applications. These technologies allow a software developer to control namespaces and system call filters. Tools that use these technologies, aimed at users, have also been created. These tools allow users to run any given program inside a sandbox, however they run in user space meaning that they do not automatically enforce the principle of least privilege system-wide. Automatic enforcement of the principle of least privilege can only occur when implemented in the kernel. Existing security enhancements which have been implemented in the kernel, do not make use of recently developed sandboxing technologies.

2.4.1 General objective This work proposes the design and implementation of a security enhancement for Linux to coordinate several technologies and automatically start processes in policy defined constrained environments.

2.4.2 Specific objectives • Evaluate the scope and limitations of existing security technologies for Linux and develop a solution that further enhances system security by creating a constrained environment for process execution.

• Extend the Linux kernel to enforce a policy that will start processes in constrained environments using namespaces and system call filters.

• Provide a solution that helps Linux system administrators to secure the systems that they maintain by enabling them to define policies for the kernel extension. Chapter 3

Related Work

This chapter discusses the technologies considered to be the most pertinent to this proposal. An analysis of other related work is catalogued at the end of the chapter.

3.1 Integrated kernel tools

The Linux kernel provides a variety of security tools via Linux Security Module (LSM) hooks [24]. SELinux [25] is a commonly used module which provides a mechanism for sup- porting MAC security policies. SELinux allows fine-grained type enforcement policies which give the user a high level of control, however these policies are difficult to maintain which often leads to users granting broad rights [26]. Grsecurity [27] is a large patch for the Linux kernel which provides security enhancements that defend against a wide range of security threats through intelligent access control, memory corruption based exploit prevention and a host of other system hardening techniques that generally require no configuration. It is notable in that it is not implemented as an LSM [28]. Capsicum [29] is a cross platform operating system capabilities and sandbox framework that extends the standard POSIX API to provide sandboxed capability mode and capabilities kernel primitives. It also provides a user space sandbox API [26]. The Capsicum framework takes an approach that focuses on helping software developers to harden their programs and applications.

3.2 User space tools

Linux tools such as Firejail [19] and Mbox [20] also utilise kernel technologies to isolate or restrict applications inside a sandbox according to a given policy. These tools are good solutions to help users to provide a sandbox for a given application, however they require the user to prefix the command they wish to run inside the sandbox with a command to invoke the sandbox technology. This places the onus on the user to take the initiative to sandbox each and every application they wish to run and does not provide any operating system integration leaving it vulnerable to malware attacks. Systrace is a cross platform solution which limits an application’s access to the system by enforcing access policies for system calls [30]. Its implementation consists of a kernel patch and a user space tool. The Linux implementation of systrace predates the introduction of

9 CHAPTER 3. RELATED WORK 10 seccomp to the Linux kernel as a way of enabling system call policies. The user space tool is required to execute programs inside a sandbox. Systrace also provides a graphical user interface for generating policies interactively. As with other user space tools for sandboxing applications, it is limited in that it cannot automatically enforce system-wide MAC policies for all processes. Oz is a sandboxing system for Linux that isolates applications [31]. It achieves isolation through the use of namespaces, seccomp filters, capabilities and Xpra. Xpra is a persistent remote display server and client for forwarding applications and desktop screens and its use in Oz is to provide isolation for X11 applications. Oz is designed to sandbox applications auto- matically. It is limited in that a user must first declare which applications will be sandboxed automatically. In other words, it only supports blacklisting and cannot be used to enforce the principle of least privilege.

3.3 Other isolation work

Qubes OS [32] is a complete operating system that uses the Xen hypervisor to run individual or groups of applications in separate virtual machines. This approach provides strong isolation, however it is heavy on system resource usage due to its employment of full virtual machines per application or application group. OS level virtualisation solutions such as LXC [33] provide a type of virtual machine called a container, which shares its kernel with that of the host operating system. The LXC project makes use of the Linux kernel’s namespace isolation features to ensure that its containers can function as isolated, unprivileged operating system instances [16]. OpenBSD offers a new mitigation mechanism called Pledge. Pledge allows software de- velopers to filter system calls and create security policies for their programs with just a few lines of additional code. To facilitate the job of creating system call policies, Pledge has cat- egorised all systems calls into several groups which may be declared by a software developer for filtering [34] [35].

3.4 Access control technologies

This section presents a tabular summary of the advantages and disadvantages of the tech- nologies mentioned in the previous section and also other related technologies.

3.5 Summary

The range of security enhancing technologies available for Linux is both broad and diverse. There is no universal solution that has been established and alternatives are constantly being researched and developed. The following chapter proposes to build upon some of these tech- nologies to offer a solution which adds value to the range of technologies already available. CHAPTER 3. RELATED WORK 11

Technology Description Advantages Disadvantages Qubes OS [32] A complete operating system · Full virtual machines managed by Xen · Heavy on system resource usage due to that uses the Xen type 1 hy- provide strong isolation [36]. its use of full virtual machines. pervisor to run individual or · Its implementation as a complete, inde- groups of applications in sep- pendent operating system makes it diffi- arate virtual machines. cult to adapt for integration into existing operating systems. LXC [33] OS level virtualisation technol- · Relatively lightweight as it shares the ·Application isolation is possible, how- ogy for Linux. Achieves isola- kernel with the host OS. ever LXC is a solution for isolating com- tion by utilising namespaces. · Does not require a custom kernel, re- plete operating systems. cent mainline kernels provide full sup- · Poor documentation for configuring port [37]. unprivileged containers. · Containers can be run as an unprivi- leged user [16]. Linux- OS level virtualisation technol- · Offers an experimental kernel that in- · Not provided by mainline Linux [38]. VServer [38] ogy for Linux. cludes grsecurity patches [38]. · Root privilege isolation is not guaran- teed [39]. OpenVZ [40] OS level virtualisation technol- · Offers checkpointing and live migration · Not provided by mainline Linux [42]. ogy for Linux. [41]. Virtuozzo [43] OS level and full virtualisation · Offers enterprise features like inte- · Not provided by mainline Linux [42]. technology for Linux based grated backup and P2V migration [44]. · Commercial/proprietary software li- on OpenVZ. Full virtualisation cence [45]. provided via a hypervisor. systemd- OS level virtualisation technol- · Integrated OS level virtualisation solu- · Requires systemd which may not be nspawn [46] ogy for systemd. Uses systemd tion for Linux installations that use sys- installed or available for a given Linux to spawn namespace contain- temd. instance [47]. ers for debugging, testing and building.

Table 3.1: Isolation through virtualisation of operating systems CHAPTER 3. RELATED WORK 12

Technology Description Advantages Disadvantages Docker [48] Provides application contain- · Integration with various infrastructure · Portability means that containers are ers to facilitate the deployment tools [49] [50] [51]. large and must include all of the appli- of applications. · Container images can be run without cation’s dependencies [53] which may re- modification on multiple Linux distribu- sult in a large memory footprint. tions [52]. · Applications must be repackaged as containers images [53]. · Difficult to build containers images for large applications [54]. Firejail [19] An SUID security sandbox pro- · A general purpose sandbox that can · User space tool which must be man- gram that reduces the risk of easily be applied to any program. ually invoked for each command in or- security breaches by restrict- ·Can replace a user’s default command der to provide the sandbox. This means ing the running environment interpreter. that there is no automatic enforcement of untrusted applications using · It does not have the overhead of in- of least privilege by the operating sys- Linux namespaces, seccomp- stalling or executing an application in- tem. bpf and POSIX capabilities. side an OS container. Mbox [20] Prevents programs from modi- · A general purpose sandbox that can · Requires user interaction at the end of fying the host file system while easily be applied to any program. program execution to examine changes giving them the impression · It does not have the overhead of in- in the sandbox file system and then se- that they are in fact making stalling or executing an application in- lectively commit them back to the host those modifications by provid- side an OS container. file system. ing a layered sandbox file sys- · Allows an unprivileged user to install a · User space tool which must be man- tem and by interposing on sys- software package [20]. ually invoked for each command in or- tem calls with ptrace() and der to provide the sandbox. This means seccomp-bpf. that there is no automatic enforcement of least privilege by the operating sys- tem. Oz [31] A sandboxing system targeting · A general purpose sandbox that can · User space tool which only supports everyday workstation applica- easily be applied to any program. blacklisting. This means that there is no tions. It uses namespaces, sec- · Provides automatic sandboxing for ap- automatic enforcement of least privilege comp filters, capabilities and plications that the user has identified to by the operating system. Xpra to achieve isolation. be untrusted. PIP [55] [56] An information flow system · Fine-grained policies [56]. · User space tool which must be man- focused on usability to pro- · Decomposed sandbox architecture [55]. ually invoked for each command in or- vide integrity protection. The · Techniques for inferring policies from der to provide the sandbox. This means system combines userland ap- runtime and profile data for untrusted that there is no automatic enforcement proach with DAC to enforce se- processes [55]. of least privilege by the operating sys- curity policies. It does not rely · Spares users from making security- tem. on the kernel. critical policy decisions [55].

Table 3.2: User space tools that provide sandboxes for programs and applications. CHAPTER 3. RELATED WORK 13

Technology Description Advantages Disadvantages SELinux [57] LSM that provides a mecha- · Allows fine-grained Type Enforcement · Broad rights are often granted as fine- nism for supporting MAC poli- policies [24] [26]. grained Type Enforcement policies are cies. · Supports RBAC [24]. difficult to write and maintain [26]. · Optionally supports multi-level secu- · Requires that the file system supports rity [24]. “security labels” [59]. · Supports the concept of a “remote pol- · LSMs are not stackable [28]. icy server” [58]. AppArmor [60] LSM that supplements the tra- · Easier to learn and use compared to · File system objects are identified by ditional Unix DAC model by SELinux [61]. path name instead of inode meaning providing MAC. · File system agnostic [62]. that policies can easily be bypassed by creating a new hard link [63]. · Limited set of operations on files [64]. · LSMs are not stackable [28]. Smack [65] LSM that protects data and · Simplicity is its main design goal [66]. · LSMs are not stackable [28]. process interaction from mali- cious manipulation using a set of custom MAC rules. TOMOYO LSM that focuses on the be- · Can be used as a system analysis tool · Version 2.x has not yet been completely Linux [67] haviour of a system. Every [67]. mainlined [68]. process is created to achieve a · Easy to use [67]. · LSMs are not stackable [28]. purpose and is allowed to de- clare behaviours and resources needed to achieve their pur- pose. AKARI [69] LSM forked from TOMOYO · Fewer features than TOMOYO Linux · A few advanced networking operations Linux 1.x. [68] which may be desirable for certain are limited or unavailable for restriction use cases. [68]. · Restricting use of capabilities is not possible [68]. · LSMs are not stackable [28]. Yama [70] LSM that collects a number of · Aims to prevent an attacker from at- · Only ptrace() is restricted for now [70]. system-wide DAC security pro- taching other running processes to a · LSMs are not stackable [28]. tections that are not handled compromised application [70]. by the core kernel itself. Linux Intrusion LSM that implements MAC. · Offers a port scan detector [72]. · Not provided by mainline Linux. Detection Sys- · Offers file protection, even from root · Appears to be discontinued. tem [71] [72]. · LSMs are not stackable [28]. · Offers process protection [72].

Table 3.3: Linux Security Modules CHAPTER 3. RELATED WORK 14

Technology Description Advantages Disadvantages Linux names- A global system resource ab- · Strong process isolation alternative. · A Linux-only kernel feature. paces [14] straction that makes it ap- · Linux provides namespaces for Con- pear to the processes within trol Groups (cgroups), IPC, Network, the namespace that they have Mount, PID, User and UTS [14]. their own isolated instance of the global resource. Changes to the global resource are vis- ible to other processes that are members of the namespace, but are invisible to other pro- cesses. seccomp [15] A Linux kernel feature that · Strong process confinement. · It may be difficult to establish which restricts which system calls a · The filtering extension allows a devel- system calls are required by a process. thread may make. oper to filter system calls using a config- · The programming interface may only urable policy implemented using Berke- be used from user space. ley Packet Filter (BPF) rules [73]. · Time Of Check to Time Of Use (TOC- TOU) attacks are impossible [15]. POSIX capa- An abstraction of a set of op- · A set of privileges was defined to be · The POSIX.1e draft standard was bilities [74] erating system operations as portable across POSIX compliant oper- withdrawn [74] [75]. process privileges that can be ating systems. · Extended privileges are not portable granted or denied by a software · An extended set of OS specific priv- [74]. developer. ileges is available for Linux and other · No default enforcement policy is avail- operating systems [74]. able. · Must be implemented by the software developer of each program.

Table 3.4: Other security facilities provided by mainline Linux CHAPTER 3. RELATED WORK 15

Technology Description Advantages Disadvantages RSBAC [76] Access control framework that · Several well-known and new security · Not provided by mainline Linux. gives detailed access control models, like MAC, ACL and RC [77]. information and can imple- · Detailed control over individual user ment almost any access control and program network accesses [77]. model, e.g. as a runtime regis- · Virtual User Management, in kernel tered kernel module. and fully access controlled [77]. · On-access virus scanning with the Dazuko interface [77]. · Any combination of security models is possible [77]. · Easily extensible: write your own model for runtime registration [77]. · Powerful logging system which makes intrusion attempts easily detectable [77]. PaX [78] A patch for the Linux kernel · Offers executable space protection [79]. · Not provided by mainline Linux. that implements least privilege · Offers ASLR [80]. · Cannot block fundamental design flaws protections for memory pages. · Blocks attacks which introduce and ex- in either executable programs or in the ecute arbitrary code [81]. kernel that allow an exploit to abuse · Blocks attacks which attempt to ex- supplied services. ecute existing program code in the in- tended order with arbitrary data [81]. grsecurity [27] An extensive security enhance- · Provides protection against zero-day · Not provided by mainline Linux. ment to the Linux kernel that and other advanced threats [27]. · Stable patches are restricted to paying defends against a wide range of · Mitigates shared-host/controller weak- customers only [83]. security threats through intel- nesses [27]. ligent access control, memory · Not using LSM may provide various corruption based exploit pre- advantages [28] [82]. vention, and a host of other · Includes PaX patches [27] [81]. system hardening techniques that generally require no con- figuration. Openwall Small security-enhanced Linux · Proactive source code review for several · Not provided by mainline Linux. GNU/*/Linux distribution for servers, appli- classes of software vulnerabilities [84]. [84] ances, and virtual appliances. · Compatible with other Linux distribu- tions as a patchset. Systrace [85] A utility which limits an ap- · Confines untrusted binary applications · Not provided by mainline Linux. plication’s access to the system [30]. · User space tool which must be man- by enforcing access policies for · Interactive policy generation with ually invoked for each command in or- system calls. graphical user interface [30]. der to provide the sandbox. This means · Non-interactive policy enforcement that there is no automatic enforcement [30]. of least privilege by the operating sys- · Remote monitoring and intrusion de- tem. tection [30]. · Privilege elevation mode makes it pos- sible to eliminate setuid binaries [30]. Capsicum [86] OS capabilities and sandbox · Fine-grained sandboxing available to · Must be implemented by the software framework that extends the the software developer provides strong developer of each program [29]. POSIX API to provide new process isolation [29]. · When adapting existing software to run kernel primitives and a user in capability mode, identifying capabil- space sandbox API. ity requirements can be tricky [29]. · Only partially provided by mainline Linux. NUSE [87] Network stack in USerspacE. A · Similar to micro-kernel design, the net- · Not a complete solution for isolating new project to move the Linux work stack is isolated from the Linux processes or applications. network stack into user space. kernel. · Not a very mature technology.

Table 3.5: Out-of-tree kernel patches Chapter 4

Proposal

This chapter examines how namespace and system call filtering technologies may be used to improve the overall security of a Linux system. The proposed solution overcomes the limitations of user space tools by initiating these technologies from kernel space in order to enforce MAC policies. This is achieved by extending the Linux kernel so that it enforces all processes to auto- matically spawn inside a sandbox that uses namespaces and system call filters. This observes the principle of least privilege in ways that have not been explored by other security enhance- ments. This sandbox uses namespaces to isolate the process which constrains the global view of the system that it sees and uses system call filters to restrict what it is able do. Rather than offering a replacement for existing kernel security enhancements, the proposed solution is intended to complement them by adding extra layers of security. The end-user of this solution is intended to be a Linux distribution maintainer or a system administrator. The level of access required for a process to be fully functional may be granted by applying policies that have been previously defined by the end-user. This proposal provides a way of limiting the scope of an attack from vulnerabilities that may exist in authorised software, it helps to prevent the use of unauthorised software that a user may have installed and it can also mitigate attacks that originate from malware.

4.1 Design

An operating system kernel provides system calls to allow user space programs to make kernel service requests. When a system call is executed, the kernel takes over execution to run code in kernel space which has a higher privilege level than user space. In Linux operating systems, a new process is created when either of the execve() or execveat() system calls are initiated from user space. The execve() system call is used for executing new programs. The execveat() system call is similar to execve() but it allows a program to be executed relative to a directory file descriptor. Both system calls execute common code. The proposed kernel enhancement is achieved by modifying the common code that called by execve() and execveat() system calls. The kernel plays a key role in running an operating system; it is therefore critically impor- tant that the design of this kernel extension does not introduce any vulnerabilities which may compromise the security of the system, thus negating the whole objective of providing a secu-

16 CHAPTER 4. PROPOSAL 17 rity enhancement. New code introduced should be lean and kept to a minimum. Figure 4.1 shows a high level view of constrained process creation.

Figure 4.1: Creation of constrained processes. (1) A process makes a request to the kernel to create a new process. (2) The modified kernel function uses a previously defined policy to create a process that will run within a constrained environment.

4.1.1 Constrained access To build constrained processes, two resources are controlled: namespaces and system calls.

• The Linux kernel provides namespaces which wrap global system resources in an ab- straction that makes it appear to the processes within the namespace that they have their own isolated instance of the global resource [14]. Changes to the global resource are visible to other processes that are members of the namespace but are invisible to other processes that have been defined in their own namespace [14]. The Linux kernel currently provides seven different types of namespaces: Cgroups, IPC, Network, Mount, PID, User and UTS. Linux 4.6 introduced cgroup namespaces which was not available at the time of development of this solution, therefore they are not covered by this thesis.

• A process must execute a system call to request particular services from the kernel. The kernel receives the call, changes execution mode, from user mode to kernel mode and runs the routine that handles the request. When the routine finishes the task, the kernel changes execution mode back to user mode and returns control to the user program. Filtering system calls can significantly impede a process by limiting the actions that it is capable of performing, thus minimising the exposed kernel surface [15]. A process may filter incoming system calls using the seccomp with filters kernel facility.

Namespace and seccomp technologies have typically been used in containers and user space driven sandbox programs but their potential to be incorporated into MAC policy has yet to be explored. By tackling the problem inside the kernel, the principle of least privilege is extended to use these relatively new mechanisms to confine a process automatically inside a sandbox. CHAPTER 4. PROPOSAL 18

4.1.2 Enforcing constrained execution This thesis proposes to modify the execve() system call to automatically execute new processes inside a sandbox with isolated namespaces and a limited set of system calls that may be used. By modifying execve(), the principle of least privilege may be enforced at the earliest opportunity, providing an effective way of hardening a Linux system that is not currently used by existing MAC security enhancements in the kernel. Processes will need to be granted the access that they require in order to be fully functional. To achieve this, a policy file may be defined by the end-user. This policy file defines namespace and system call policies for each authorised program. The file will be parsed by a user space utility which implements an interface in the kernel that can then be used by the modified execve() system call to apply MAC policy prior to invoking a program’s main function. The policy for a given program will be identified by the filename argument of execve() and matched against an entry in the policy file. If a policy for the program pointed to by the filename is not found or if the policy file is not accessible, then the modified execve() will enforce a default restricted policy. This will thwart the capabilities of any unauthorised software and mitigate a substantial amount of attacks that originate from malware. This solution should not be the only means that a user has to protect their system. There are already many well established methods for securing a Linux system which provide strong protection. It is not the aim of this solution to replace these methods, but to supplement them. For greatest flexibility, the proposed kernel modification should not be implemented as an LSM as this kernel feature does not support stacking of multiple modules.

4.1.3 Default policies In order to maximise the protection of a system and enforce the principle of least privilege, it is desired that every process will spawn inside a sandbox which will severely inhibit its capabilities unless an overriding policy is explicitly defined by the policy file. It is expected that enforcing such a policy will result in the breakage of many processes and will prevent their normal operation. This is a frequent challenge when enforcing mandatory access control. A user that encounters a program that has broken due to MAC policy may become frustrated and decide to completely disable MAC in order to make their program work. This is an undesired effect when enforcing the principle of least privilege. Whilst it is possible to set a default policy that completely neuters a process, such a strict policy may be undesirable as it may serve to frustrate the user. There is no perfect solution. What follows is a brief description of each of the six types of namespaces offered by Linux at the time of development, and an evaluation of their usage as part of a default policy for isolating processes. Likewise, the suitability of seccomp for use in default policy is also discussed.

4.1.3.1 IPC namespaces IPC namespaces provide isolation for System V IPC objects and POSIX message queues [14]. System V IPC provides message queue, semaphore, and shared memory facilities. POSIX message queues provide similar functionality to System V message queues but have a distinct API. These are all mechanisms that allow processes to share data with one another. They operate on a system-wide level using object ownership, octal permissions and an identifier to CHAPTER 4. PROPOSAL 19 control access. Any user may obtain information on all IPC objects on a system regardless of whether they have permission to access the object. This information may be obtained using the ipcs command. A default policy that isolates processes inside separate IPC namespaces would eliminate this flaw and provide an extra layer of security.

4.1.3.2 Network namespaces Network namespaces provide isolation of the system resources associated with networking. These resources include network devices, IPv4 and IPv6 protocol stacks, IP routing tables, firewalls, the /proc/net and /sys/class/net directories and port numbers associated with sockets [14]. Physical network devices can be allocated to exactly one network namespace or can be shared across network namespaces through the configuration of virtual network devices. Spawning a process automatically inside an unconfigured namespace will prevent it from having any network access. This is a sensible default policy as it means that network access must be declared for any program that requires it.

4.1.3.3 Mount namespaces The mount namespace is relatively old compared to the other namespaces. It was first imple- mented in Linux 2.4.19 in 2002 and no other namespaces were planned at the time. Mount namespaces isolate the set of file system mount points meaning that processes in different mount namespaces can have different views of the file system hierarchy [14]. A process run- ning in its own mount namespace inherits the mount points of its parent process but may add or remove mount points that are private to the namespace. The privacy that this pro- vides makes mount namespaces a reasonable choice to include in a default policy that aims to enforce the principle of least privilege.

4.1.3.4 PID namespaces PID namespaces isolate the process ID number space, meaning that processes in different PID namespaces can have the same PID [88]. A typical process does not need to know about other processes running on the system yet they are able to read the /proc file system and obtain information on all running processes. This information may be very useful to an attacker that wants to explore a system. This can be mitigated by using a default policy that enforces every process to start in a new PID namespace.

4.1.3.5 User namespaces User namespaces isolate security related identifiers and attributes, in particular, UIDs and GIDs, the root directory, keys and capabilities [89]. This allows a process inside a user namespace to run as UID 0 whilst having a different UID outside of its namespace. This gives it super user privileges inside the namespace yet it will be isolated from performing privileged operations outside the namespace. Some exploits have been known to break out of user namespaces. It is hoped that this will improve as the technology matures and bugs are ironed out [90]. Nevertheless, with an up to date kernel, user namespaces still provide an extra layer of security that must be broken. If the process is started by an unprivileged user, the default policy will run the process inside a new user namespace as the nobody user with CHAPTER 4. PROPOSAL 20 a primary group of nogroup. If the process is started by the root user, the process will be started inside a new namespace with a UID of 0.

4.1.3.6 UTS namespaces UNIX Time-Sharing System namespaces, commonly referred to as UTS namespaces, provide isolation for the hostname and NIS domainname system identifiers [14]. These are typically used in containers to allow each container to have unique system identifiers. A default policy that automatically starts a process inside a new UTS namespace would prevent an attacker from being able to change these identifiers in the global namespace.

4.1.3.7 Secure computing (seccomp) The original implementation of seccomp did not support custom system call policies but allowed a process to use a fixed set of only four system calls. This operational mode, now known as strict mode, is still available in the seccomp kernel API. The only system calls that a calling thread in strict mode is permitted to make are read(), write(), exit() and sigreturn(). Other system calls result in the delivery of a SIGKILL signal which will immediately kill the process. The current implementation of seccomp includes an alternate mode, known as filter mode, which supports configurable system call policies using BPF rules. BPF originated as a means to provide a raw interface to the data link layer of the OSI model of computer networking. Whenever a packet is received by an interface, all file descriptors listening on that interface apply a user defined filter. In Linux, BPF has been extended and now allows a user to define filters to control system calls using seccomp. The proposed extension to grsecurity uses the filter operational mode to allow the end-user to use granular system call policies by defining a whitelist containing each of the system calls that a process is permitted to use. As with the strict mode, any attempt to use a system call that has not been defined in the whitelist will use a SIGKILL signal to immediately kill the process. It is possible to specify alternate behaviour for filter mode which drops the system call but allows the process to continue instead of sending a SIGKILL signal. This may be specified in the definition of the socket filter of the BPF program, however this behaviour falls outside of the scope of the presented solution. The structure of a seccomp BPF program and its filter is described in the seccomp documentation [15] [91]. Setting a default policy for filtering system calls is not an easy task due to the sheer volume of available system calls and the unpredictable nature of which ones a process will use. For this reason, system call filter policies will be left to be established by the end-user.

4.1.4 Policy creation An end-user who has chosen to use this solution will want to define their own policies. This process should be relatively straightforward for namespaces as their behaviour is clearly de- fined. It should be fairly easy figure out whether a process requires access to the network, requires access to the list of processes running on a system, requires access to IPC facilities and so forth. On the other hand, identifying which system calls a process may need to use can be quite unpredictable. Fortunately seccomp provides a feature to catch a failed system CHAPTER 4. PROPOSAL 21 call and report it. This significantly simplifies the task of creating new system call policies and will therefore be incorporated into the solution.

4.2 Limitations

The solution can only enforce policy rules at the moment that a process is spawned. Although this solution provides better protection than traditional process execution, a process may not need the complete access to a particular resource that has been granted by a policy throughout its lifetime. It is therefore not recommended as a replacement for making sure that software has been properly sandboxed through the modification of source code. The life cycle of a process may be generalised into two phases: the initialisation phase followed by the main loop phase. The resources that a process requires for each of these phases normally differ substantially and will often have further changes in requirements during the main loop phase. A software developer will be able to identify exactly when their software will or will not require a particular resource and will be able to code their software to grant or deny access to that resource at runtime in order to provide better security. This solution is purely intended to be a facility that assists the end-user to harden their systems. Chapter 5

Implementation

To demonstrate this proposal, the solution will be built on top of an existing, well established security enhancement that permits users to define new policies. This minimises the amount of new code introduced and allows efforts to be focused on the core part of the proposed solution. To meet this requirement, grsecurity was chosen as the kernel security enhancement that this solution would be built on top of. It is one of the leading security enhancements for Linux that is not implemented as an LSM and provides security features that are not available in mainline kernel LSMs. Although grsecurity is not the most popular security enhancement and is not available in mainline Linux, it is installed by default in security orientated distributions of Linux based operating systems such as Alpine Linux and Subgraph OS [92] [93], whilst distributions like Debian and Gentoo provide facilities to install it [94] [95]. It provides protections for several classes of attacks that are not protected in mainline Linux. This means that grsecurity hardened kernels have not been affected by some known kernel vulnerabilities [96]. The solution takes into account the design aspects described the previous chapter. The result is an extension to grsecurity which implements policies to control system call filters and namespaces. At present it implements system call filters and three types of namespaces: network, IPC and UTS. The initial implementation supports the x86-64 architecture only.

5.1 Components

This section describes the components used and how they have been modified to achieve the desired solution.

5.1.1 Grsecurity Grsecurity is a comprehensive security enhancement for Linux. It adds several enhancements that are not included in the mainline Linux kernel and are missing from security modules such as SELinux and AppArmor [97]. It consists of memory page protections including PaX, an RBAC system, file system protections, kernel auditing, executable protections, network protections and physical protections for USB devices. The Kernel Self Protection Project aims to bring some of grsecurity’s protections to mainline Linux [98].

22 CHAPTER 5. IMPLEMENTATION 23

Grsecurity is available for download in several parts but this solution is only concerned with two of them. The main part is the kernel patch which may be applied to a range of kernels; the compatible range is defined in the patch’s filename. Kernel patches come in two series, stable and test. Since September 2015, the stable series is only made available to commercial customers. The proposed solution is built using the publicly available testing series. The other part of grsecurity is a user space program called gradm which serves as an RBAC administration utility. This solution has been developed on top of Linux version 4.4.6 with grsecurity patch named grsecurity-3.1-4.4.6-201604100830.patch and gradm version 3.1- 201603152148.

5.1.1.1 Gradm Gradm is an administration program for the grsecurity RBAC system. It is used in conjunction with a policy file which allows the end-user to define system security policies. The main tasks of gradm include authentication, parsing of the policy file and validation of policies, enabling and disabling of the RBAC system and enabling and disabling of learning mode. Gradm reads policies from the policy file, validates them and then passes them to the kernel. The implementation of gradm uses the programs Flex and Bison to define the policy file structure. Flex is used to generate lexical analysers that create tokenised input data to define the lexicon of the policy file. Bison is a parser generator which uses the tokenised data generated from Flex to define the language of the policy file. Both of these tools generate C code which is integrated into the gradm program.

5.1.1.2 Policy file structure Grsecurity’s RBAC system uses a policy file with rules that consist of role, subject and object abstractions:

• A role describes a traditional Unix user or group or a special role that is specific to grsecurity.

• A subject describes a directory, a program or a script.

• Objects define the resources that are to be applied to subjects. They may represent files, capabilities, resource restrictions, PaX flags or IP ACLs.

Roles and subjects may specify additional attributes. Roles, subjects and objects may also use modes to provide information pertinent to the rule. These rules establish the behaviour of a resource and who it applies to. A generalisation of the policy file structure is illustrated in Figure 5.1. Further information describing the policy file structure may be found in the official grsecurity documentation [99].

5.1.1.3 Gradm special roles One example of a gradm special role is the admin role. If an end-user wishes to perform an administrative task which would be inhibited by the active policies, they may authenticate to the admin special role which requires a password. This will allow them to perform the CHAPTER 5. IMPLEMENTATION 24 role subject / /