Universidad de Los Andes Tesis de Maestr´ıa Hardening Linux Processes Extending Grsecurity to Integrate System Call Filters and Namespaces David Derby Cardona Facultad de Ingenier´ıa Departamento de Ingenier´ıade Sistemas y Computaci´on June 2016 Universidad de Los Andes Tesis de Maestr´ıa Hardening Linux Processes Extending Grsecurity to Integrate System Call Filters and Namespaces David Derby Cardona Asesor: Sandra Rueda Rodr´ıguez Jurados: Rafael G´omezD´ıaz Fabian Molina Molina Facultad de Ingenier´ıa Departamento de Ingenier´ıade Sistemas y Computaci´on June 2016 Abstract The area of Linux sandboxing has seen various developments in recent years with the intro- duction of operating system containers and the ever present need to harden the security of applications. Two of the more prominent technologies that have been used when creating sandboxes are namespaces and system call filters. Whilst these technologies have been ef- fective for creating sandboxes, they are limited in that they require a developer to integrate them into their software. This work proposes to use these two technologies to enforce the Principle of Least Privilege on every process on a system. The solution extends a grsecurity hardened Linux kernel and allows the user to define security policies for each process which permit them to behave as intended. The presented results demonstrate the effectiveness of the extended Linux kernel and its impact on performance. The results provide a basis that may be built upon to deliver a comprehensive solution that would be appealing for use in real world environments. 1 Contents Abstract 1 Index of Figures 4 Index of Tables 5 1 Introduction 1 2 Context and Problem Description 3 2.1 Linux . .3 2.1.1 Processes . .3 2.1.2 Permissions . .4 2.1.3 Authorisation . .5 2.2 Least privilege . .5 2.3 Limitations of existing solutions . .6 2.3.1 Type enforcement solutions . .6 2.3.2 Other solutions . .6 2.3.2.1 Android . .7 2.4 The problem . .8 2.4.1 General objective . .8 2.4.2 Specific objectives . .8 3 Related Work 9 3.1 Integrated kernel tools . .9 3.2 User space tools . .9 3.3 Other isolation work . 10 3.4 Access control technologies . 10 3.5 Summary . 10 4 Proposal 16 4.1 Design . 16 4.1.1 Constrained access . 17 4.1.2 Enforcing constrained execution . 18 4.1.3 Default policies . 18 4.1.3.1 IPC namespaces . 18 4.1.3.2 Network namespaces . 19 2 CONTENTS 3 4.1.3.3 Mount namespaces . 19 4.1.3.4 PID namespaces . 19 4.1.3.5 User namespaces . 19 4.1.3.6 UTS namespaces . 20 4.1.3.7 Secure computing (seccomp) . 20 4.1.4 Policy creation . 20 4.2 Limitations . 21 5 Implementation 22 5.1 Components . 22 5.1.1 Grsecurity . 22 5.1.1.1 Gradm . 23 5.1.1.2 Policy file structure . 23 5.1.1.3 Gradm special roles . 23 5.2 Extending grsecurity . 24 5.2.1 Extension to gradm . 24 5.2.1.1 Implementation of new policy types . 26 5.2.2 Extension to the grsecurity hardened Linux kernel . 27 5.2.2.1 Implementation of namespaces . 27 5.2.2.2 Implementation of system call filters . 28 5.3 Evaluation . 28 5.3.1 System call filtering . 28 5.3.2 Namespaces . 29 5.3.3 Performance . 29 5.4 Summary . 30 6 Conclusions and Future Work 38 6.1 Conclusions . 38 6.2 Future work . 38 6.2.1 Feature completeness . 39 6.2.2 Blacklists . 39 6.2.3 Namespace bind mounts . 39 6.2.4 Further namespace customisation . 39 6.2.5 Further seccomp customisation . 40 6.2.6 Policy file separation . 40 6.2.7 Learning mode . 41 6.2.8 Code refactoring . 41 A Gradm patch 42 B Kernel patch 53 Bibliography 59 List of Figures 2.1 Execution of a command in Linux . .4 2.2 Access control representation . .4 2.3 The Android execution environment . .7 4.1 Creation of constrained processes . 17 5.1 Structure of the grsecurity policy file . 24 5.2 Flex code to match namespace and system call filter rules . 25 5.3 Bison code to handle tokens . 25 5.4 Example of a policy in the policy file . 26 5.5 Sequence of interaction of grsecurity's RBAC system . 31 5.6 The sequence of interaction between a process, execve() and stored policies . 32 5.7 The C source code of the \rootshell" program . 32 5.8 Demonstration of \rootshell" malware succeeding . 33 5.9 Finding setuid programs with the find command . 33 5.10 Demonstration of \rootshell" malware failing . 33 5.11 Demonstration of a network namespace . 34 5.12 Demonstration of an IPC namespace . 35 5.13 Demonstration of a UTS namespace . 35 5.14 Tests run on a standard grsecurity enhanced Linux kernel . 36 5.15 Tests run on a grsecurity Linux kernel including the extension presented in this work.......................................... 37 6.1 Policy rule for an IPC namespace with a bind mount . 39 6.2 Policy rule for a network namespace with a virtual network link . 40 6.3 Policy rule for a system call filter with a thread synchronisation flag . 40 4 List of Tables 3.1 Isolation through virtualisation of operating systems . 11 3.2 User space tools for sandboxing . 12 3.3 Linux Security Modules . 13 3.4 Other security facilities provided by mainline Linux . 14 3.5 Out-of-tree kernel patches . 15 5 Chapter 1 Introduction The use of Linux based operating systems has grown in server, desktop/laptop and mobile deployments [1]. Linux usage of the Internet's web servers is significant as it is the most popular operating system with hosting companies. Eight out of ten of the most reliable com- pany websites were hosted on Linux machines as reported by Netcraft in April 2016 [2]. The desktop and laptop market has seen growth with the Linux based ChromeOS, the operating system which runs on Chromebooks, recently outselling Apple's range of Macs for the first time in the United States of America [3] [4]. In the mobile sector, Linux remains the dominant player with the Linux based Android operating system holding 80% market share [5]. The significant presence that Linux holds in the industry, highlights the need to research new ways to keep these systems secure. Malicious programs have frequently been developed for Windows systems, however the increased popularity of other operating systems including Linux, has led to an increase in those being targeted too. This rise shows that attackers are not only interested in targeting single clients but also the infrastructure that they use [6]. Although the volume of attacks targeting Linux is lower than those targeting Windows, the number of attacks against Linux has considerably increased, meaning that Linux system administrators have an increased responsibility to consider additional defence mechanisms to protect their systems. Unix based systems traditionally use discretionary access control (DAC) as a means of restricting access to objects based on the identity of subjects and/or groups to which they belong [7]. When a program is executed, it uses default permission inheritance which means that the created process will run with the same permissions as the user that started it. The process has the same visibility and access rights to the system as the user would have, thus each process exposes a surface that would be attractive to an attacker who may take advantage by exploiting a vulnerability in a program or by means of malware executed by that user. A user does not need to have the same system access rights as a privileged user such as root for an attacker to cause damage or to obtain confidential information. Existing solutions which improve upon the traditional DAC restriction on computer pro- grams include the use of: role based access control (RBAC), security modules or enhancements which provide mandatory access control (MAC), control of operating system privileges, op- erating system capabilities, memory page protections, integrity protection and a variety of methods to create a sandbox. These solutions often present their own set of challenges which impact their widespread adoption. 1 CHAPTER 1. INTRODUCTION 2 Recent additions to the Linux kernel provide new facilities that allow a software developer to isolate processes and restrict their behaviour. Given that these isolation and restriction techniques are relatively new, much of the large back catalogue of software already available for Linux does not yet take advantage of them. In order to do so, each software package would have to be analysed and then modified. A software developer may not necessarily take the time and effort to secure the software that they have already written or that they may write in future. This work explores the security features provided by the Linux kernel and proposes an original solution that complements other methods of locking down the runtime environment of all Linux processes. It aims to help Linux distribution maintainers and system administrators to implement policies that restrict the environment of existing unmodified software. The proposed solution not only helps to prevent attacks that originate from vulnerabilities in authorised software, including zero-day threats, but can also prevent attacks from purpose- built malware. The rest of the document is organised as follows: Chapter 2 presents the problem in depth; Chapter 3 reviews related work; Chapters 4 and 5 propose a solution that addresses the limitations of previous work;.