Clean Application Compartmentalization with SOAAP Khilan Gudka Robert N.M. Watson Jonathan Anderson University of Cambridge University of Cambridge Memorial University Cambridge, UK Cambridge, UK St. Johns, NL, Canada [email protected] [email protected] [email protected] David Chisnall Brooks Davis Ben Laurie University of Cambridge SRI International Google UK Ltd. Cambridge, UK Menlo Park, CA, USA London, UK [email protected] [email protected] [email protected] Ilias Marinos Peter G. Neumann Alex Richardson University of Cambridge SRI International University of Cambridge Cambridge, UK Menlo Park, CA, USA Cambridge, UK [email protected] [email protected] [email protected] ABSTRACT ploitation of vulnerabilities. It seeks to employ the prin- Application compartmentalization, a vulnerability mitiga- ciple of least privilege by granting each compartment only tion technique employed in programs such as OpenSSH and the rights that it requires to operate [37]. It is used in the Chromium web browser, decomposes software into iso- software ranging from OpenSSH [35] to the Chromium web lated components to limit privileges leaked or otherwise browser [36] to limit rights available to an attacker follow- available to attackers. However, compartmentalizing appli- ing a successful exploit. Recently disclosed vulnerabilities cations { and maintaining that compartmentalization { is such as Heartbleed [29], Shellshock [31], and a symlink at- hindered by ad hoc methodologies and significantly increased tack vulnerability in Wget [30], illustrate the importance of programming effort. In practice, programmers stumble protecting software not just from known exploit techniques through (rather than overtly reason about) compartmental- (e.g., stack canaries mitigating buffer overflows), but also ization spaces of possible decompositions, unknowingly trad- from unknown future classes of vulnerabilities. ing off correctness, security, complexity, and performance. Application compartmentalization proves surprisingly We present a new conceptual framework embodied in an subtle: contrary to the implied suggestion of simplistic de- LLVM-based tool: the Security-Oriented Analysis of Appli- scriptions (\Application X has been privilege separated"), cation Programs (SOAAP) that allows programmers to rea- there are many possible compartmentalizations of most pro- son about compartmentalization using source-code annota- grams within a compartmentalization space that reflects dif- tions (compartmentalization hypotheses). We demonstrate ferent tradeoffs among correctness, security, complexity, and considerable benefit when creating new compartmentaliza- performance. Selecting a suitable point in this space is tions for complex applications, and analyze existing com- challenging, as implementing compartmentalization requires partmentalized applications to discover design faults and considerable effort and can disrupt code structure. This maintenance issues arising from application evolution. choice is crucial, as it can be difficult to change strategy after implementation: the non-trivial code changes made to application structure are often specific to the selected com- Categories and Subject Descriptors partmentalization. Design choices and tradeoffs are infre- D.4.6 [Operating Systems]: Security and Protection quently revisited { even as understanding of program secu- rity grows due to discovered vulnerabilities, or as program Keywords functionality evolves beyond its compartmentalization. Reasoning about compartmentalization tradeoffs is diffi- Security, Compartmentalization, Vulnerability Mitigation cult for many reasons. Information about past vulnerabil- ities { often a good predictor of future vulnerabilities { is 1. INTRODUCTION not easily accessible. Call graphs of compartmentalized ap- Application compartmentalization decomposes software plications are extremely complex, and simple control-flow into isolated but collaborating components to mitigate ex- analysis cannot follow manually encoded cross-domain ac- tions { typically via Inter-Process Communication (IPC) { Permission to make digital or hard copies of all or part of this work for personal or found in real-world applications. Even where call graphs classroom use is granted without fee provided that copies are not made or distributed can be extracted, control flow is only a starting point: in for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the the compartmentalization adversary model, naively written author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or code may be vulnerable, but it is exposure to malicious data republish, to post on servers or to redistribute to lists, requires prior specific permission that causes exploits { requiring reasoning about a combi- and/or a fee. Request permissions from [email protected]. nation of vulnerabilities, information flow from dangerous CCS’15 October 12–16, 2015, Denver, Colorado, USA. sources, and uncompartmentalized code. Application fail- Copyright is held by the owner/author(s). Publication rights licensed to ACM. ures due to compartmentalization are frequently mysterious: ACM 978-1-4503-3832-5/15/10$15.00 DOI: http://dx.doi.org/10.1145/2810103.2813611. the libraries they link against are often designed without the expectation of sandboxing, and failures may not be detected 2. CONCEPTUAL FRAMEWORK by testing. Finally, performance characteristics are critical, Application compartmentalization rests on the principle of with compartmentalization boundaries often selected that least privilege: compromised components yield fewer rights are not based on ideal security boundaries, but rather on ac- to attackers, and offer fewer opportunities to compromise the ceptable performance (e.g., in Capsicum where gzip rather rest of the system. Karger first proposed using access con- than zlib is sandboxed [41]). Even then, performance im- trol for vulnerability mitigation while working on capability pacts are difficult to predict. systems [18], and took as his threat model the trojan horse: Application developers would ideally be able to explore subverted software working on behalf of the adversary. Con- how new or changing compartmentalization would affect fac- temporary compartmentalization for vulnerability mitiga- tors such as security and performance without paying the full tion, sometimes referred to as privilege separation, saw its cost of implementation, letting them more easily identify a foundations in work by Provos et al. [35] and Kilpatrick [19], preferred spot in the compartmentalization space. For ex- and has been applied to complex, security-relevant programs ample, partitioning decompression from decryption can pre- such as OpenSSH and the Chromium web browser, both of vent decompression-related vulnerabilities from leaking key- which hold substantial effective privilege and perform com- ing material { but at great costs to both code complexity plex processing of untrustworthy, network-originated data. and run-time performance. The overhead of this compart- Compartmentalization adopts a de facto, and quite strong, mentalization may be too high for production use, requiring threat model in which attackers gain total control of com- another approach, such as having per-client compartments. promised compartments as a result of poorly crafted C code This exploration is comparable to the use of performance- (or other program weaknesses) being being exposed to ma- profiling tools to identify the hot spots in an application licious input. For example, buffer-overflow attacks triggered that would benefit most from optimization. Similarly, se- by network input might allow malicious code to be executed, curity profiling tools could enable the developer to deter- or program control flow to be fully subverted by the at- mine whether a particular compartmentalization was \worth tacker. We generalize this model, adding software supply- it", namely that it would give sufficient security gains while chain vulnerabilities involving explicitly trojaned software not exceeding tolerable performance overheads. Such an ap- without the need for malicious input { e.g., back doors be- proach would also allow continuous automatic reevaluation ing inserted following compromise of a software vendor's as application source code evolves, ensuring that, for exam- source-code repository. Weaker attacks, which fall short of ple, refactorings do not accidentally move vulnerable code arbitrary code execution, are also important: the OpenSSL to a more privileged context. To this end, this paper: Heartbleed vulnerability allows read access to victim mem- • Describes a conceptual framework for compartmental- ory, which might reveal keys allowing further access, but ization grounded in isolation and controlled communi- does not immediately allow arbitrary manipulation. cation. Possible program compartmentalizations, each Compartmentalization rests on three underpinnings: a with differing design tradeoffs in terms of complexity, strong Trusted Computing Base (TCB) able to protect its performance, security, and functionality, exist within own integrity [2]; compartment isolation (implemented by compartmentalization spaces. We also describe a set of the TCB), which provides strong prevention of interference compartmentalization design patterns reflecting com- between isolated program instances [10]; and controlled com- mon approaches