Telling Your Secrets Without Page Faults: Stealthy Page Table-Based Attacks on Enclaved Execution Jo Van Bulck, imec-DistriNet, KU Leuven; Nico Weichbrodt and Rüdiger Kapitza, IBR DS, TU Braunschweig; Frank Piessens and Raoul Strackx, imec-DistriNet, KU Leuven https://www.usenix.org/conference/usenixsecurity17/technical-sessions/presentation/van-bulck This paper is included in the Proceedings of the 26th USENIX Security Symposium August 16–18, 2017 • Vancouver, BC, Canada ISBN 978-1-931971-40-9 Open access to the Proceedings of the 26th USENIX Security Symposium is sponsored by USENIX Telling Your Secrets Without Page Faults: Stealthy Page Table-Based Attacks on Enclaved Execution Jo Van Bulck Nico Weichbrodt Rüdiger Kapitza imec-DistriNet, KU Leuven IBR DS, TU Braunschweig IBR DS, TU Braunschweig [email protected] [email protected] [email protected] Frank Piessens Raoul Strackx imec-DistriNet, KU Leuven imec-DistriNet, KU Leuven [email protected] [email protected] Abstract ware to make it relatively easy to run unmodified legacy applications within an enclave [3, 2, 41, 45]. Protected module architectures, such as Intel SGX, en- An essential aspect of enclaved execution is that the able strong trusted computing guarantees for hardware- hardware prevents privileged system software from read- enforced enclaves on top a potentially malicious operat- ing or writing a module’s private memory directly, or ing system. However, such enclaved execution environ- from tampering with its internal control flow. However, ments are known to be vulnerable to a powerful class of the OS remains in charge of allocating platform resources controlled-channel attacks. Recent research convincingly (memory pages and CPU time) to protected modules, such demonstrated that adversarial system software can extract that the platform can be protected against misbehaving sensitive data from enclaved applications by carefully or buggy enclaves. One consequence of this interaction revoking access rights on enclave pages, and recording between privileged system software and enclaves is an en- the associated page faults. As a response, a number of tirely new class of powerful, indirect attacks on enclaved state-of-the-art defense techniques has been proposed that applications. Xu et al. [48] first showed how a malicious suppress page faults during enclave execution. OS can use page faults as a noise-free controlled-channel This paper shows, however, that page table-based to extract rich information (full text and images) from threats go beyond page faults. We demonstrate that an a single run of a victim enclave. This is particularly untrusted operating system can observe enclave page ac- dangerous when legacy software is running within an cesses without resorting to page faults, by exploiting other enclave, as these applications have not been hardened side-effects of the address translation process. We con- against side-channel attacks. As a result, several authors tribute two novel attack vectors that infer enclaved mem- have expressed their concerns on side-channel vulnera- ory accesses from page table attributes, as well as from bilities in a PMA setting in general, and the page fault the caching behavior of unprotected page table memory. channel in particular [12, 9, 43, 39, 7]. We demonstrate the effectiveness of our attacks by recov- ering EdDSA session keys with little to no noise from the The research community has since proposed a num- popular Libgcrypt cryptographic software suite. ber of compile-time and hardware-enabled defense tech- niques [40, 10, 39] that hide enclave page accesses from the OS. We argue, however, that page faults are but one 1 Introduction side-effect of the address translation process that is ob- servable by untrusted system software. More specifically, Enclaved execution, or support for protected modules, is the main contribution of this paper is that we show that an a promising new security paradigm that makes it possible adversarial OS can infer page accesses from an enclaved to execute application code on a platform without having execution that never suffers a page fault. Our attacks ex- to trust the underlying operating system or hypervisor. ploit the key property that the SGX design leaves page With the advent of Intel SGX [32], support for Protected table memory under explicit control of the untrusted OS. Module Architectures (PMAs) is now available on main- As such, other side-effects of the page table walk in en- stream consumer hardware, and can be used to defend clave mode can be observed by the OS with very little to against malicious or compromised system software, both no noise. We identify and successfully exploit straightfor- in an untrustworthy cloud environment [3, 36] as well as ward effects such as the setting of “accessed” and “dirty” for desktop applications [18]. In particular, one line of bits, as well as less obvious effects such as the caching of research has developed techniques and supporting soft- page table memory itself. An important consequence is USENIX Association 26th USENIX Security Symposium 1041 that our novel attack vectors bypass recent defenses that Page ok Enclave no padrs in no focus exclusively on suppressing page faults [40, 39]. Allow In summary, the contributions of this paper are: walk? mode? PRM? yes • We advance the state-of-the-art by defeating recently fail yes proposed defense techniques, showing that we can vadrs Page fault in Abort page infer page accesses without resorting to page faults. enclave? no • We present a page table-based technique to precisely yes interrupt an enclave at instruction-level granularity. padrs in yes EPCM ok fail • We implement our novel attack vectors as an exten- EPC? checks? sion to Graphene-SGX’s untrusted runtime, facilitat- ing eavesdropping on unmodified applications. • We demonstrate the effectiveness of our attacks by Figure 1: Additional memory access checks performed by extracting private EdDSA session keys from the SGX for a virtual address vadrs that maps to a physical widely used Libgcrypt cryptographic library. address padrs. Our attack framework and evaluation scenarios are available as free software, licensed under GPLv3, at SGX enclaves are instantiated as part of the virtual ad- https://github.com/jovanbulck/sgx-pte. dress space of a conventional OS process. Since PRM is a limited system resource, untrusted system software is in charge of assigning protected memory pages to enclaves, 2 Background and is allowed to oversubscribe the EPC. At enclave cre- ation time, the OS can instruct the processor to initialize In this section, we provide the necessary background on newly allocated EPC pages with unprotected code or data. Intel SGX, refine the attacker model, and discuss previous After finalizing the enclave, and before it can be entered, research results on controlled-channel attacks. the hardware calculates a secure hash of the enclave’s initial state. This allows the integrity of the untrusted 2.1 Intel SGX loading process to be attested to a remote stakeholder [1]. SGX furthermore offers dedicated ring-zero instructions Recent Intel x86 processors from Skylake onwards are be- to securely evict and reload enclave pages between EPC ing shipped with Software Guard eXtensions (SGX) [32, memory and untrusted storage. 1, 23] that enable strong, hardware-enforced trusted com- An important design decision of SGX is that it leaves puting guarantees in an untrusted execution environment. page tables under explicit control of the untrusted oper- SGX extends the instruction set and memory access logic ating system. Instead, SGX implements an additional, of the Intel architecture to allow the execution of security- independent layer of access control on top of the legacy sensitive application logic in protected enclaves in isola- page table-based memory protection mechanism. Fig- tion from the remainder of the system, including privi- ure 1 summarizes the additional checks performed when leged OS or hypervisor. accessing enclave memory. First, in order to translate the provided virtual address to a physical one, the pro- Memory Protection. An SGX-enabled processor sets cessor traverses the OS-managed page tables, as well as aside a contiguous physical memory area, referred to as the extended page tables set up by the hypervisor, if any. Processor Reserved Memory (PRM). A hardware-level As usual, a page fault is signaled to the untrusted OS in memory encryption engine guarantees the confidentiality, case of a permission mismatch or missing page table en- integrity, and freshness of PRM memory while it resides try during address translation. Any attempt to access the outside of the processor package. The PRM region is PRM region in non-enclave mode results in abort page subdivided into two data structures: the Enclave Page semantics, i.e., read 0xFF and ignore writes. Likewise, in Cache (EPC) and the Enclave Page Cache Map (EPCM). enclave mode, the processor is allowed to reference all Protected 4 KB enclave code and data pages are allocated memory that falls outside of the executing enclave’s vir- from the EPC, while every EPC page has a shadow entry tual address range, but abort page semantics apply when in the EPCM to track ownership, type, address translation, such an address resolves into PRM memory. Further- and permission meta data. EPCM memory is exclusively more, a page fault is signaled to the untrusted OS for EPC managed by the processor, and is never directly accessible accesses that either do not belong to the currently execut- to software. ing enclave, are accessed through an unexpected virtual 1042 26th USENIX Security Symposium USENIX Association address, or do not comply with the read/write/execute flow to the instruction pointer specified in the SSA frame. permissions imposed by the EPCM. SGX allows an enclave to register trusted in-enclave ex- To speed up subsequent memory accesses, SGX em- ception handlers with a cooperative OS.