Foundations and Trends R in Electronic Design Automation Vol. 11, No. 3 (2017) 249–361 c 2017 V. Costan, I. Lebedev and S. Devadas DOI: 10.1561/1000000052 Secure Processors Part II: Intel SGX Security Analysis and MIT Sanctum Architecture Victor Costan, Ilia Lebedev and Srinivas Devadas [email protected], [email protected] and [email protected] Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology Contents 1 Introduction 250 1.1 The Case for Hardware Isolation . 251 1.2 Intel SGX is Not the Answer . 252 1.3 MIT Sanctum Processor . 253 2 An Analysis of Intel’s Software Guard Extensions (SGX) 255 2.1 SGX Implementation Overview . 256 2.2 SGX Memory Access Protection . 261 2.3 SGX Security Check Correctness . 268 2.4 Tracking TLB Flushes . 276 2.5 Enclave Signature Verification . 280 2.6 Key Hierarchy and Derivation . 285 2.7 SGX Security Properties . 288 3 The MIT Sanctum Processor 306 3.1 Threat Model . 307 3.2 Programming Model Overview . 309 3.3 Protection Boundaries . 315 3.4 Security Primitives . 315 3.5 Hardware Modifications . 317 3.6 Software Design . 324 ii iii 3.7 Security Analysis of Sanctum . 338 3.8 Work Related to Sanctum Mechanisms . 348 4 Conclusion 350 Acknowledgments 352 References 353 Abstract This manuscript is the second in a two part survey and analysis of the state of the art in secure processor systems, with a specific fo- cus on remote software attestation and software isolation. The first part established the taxonomy and prerequisite concepts relevant to an examination of the state of the art in trusted remote computation: attested software isolation containers (enclaves). This second part ex- tends Part I’s description of Intel’s Software Guard Extensions (SGX), an available and documented enclave-capable system, with a rigorous security analysis of SGX as a system for trusted remote computation. This part documents the authors’ concerns over the shortcomings of SGX as a secure system and introduces the MIT Sanctum processor developed by the authors: a system designed to offer stronger security guarantees, lend itself better to analysis and formal verification, and offer a more straightforward and complete threat model than the Intel system, all with an equivalent programming model. This two part work advocates a principled, transparent, and well- scrutinized approach to system design, and argues that practical guar- antees of privacy and integrity for remote computation are achievable at a reasonable design cost and performance overhead. V. Costan, I. Lebedev and S. Devadas. Secure Processors Part II: Intel SGX Security Analysis and MIT Sanctum Architecture. Foundations and Trends R in Electronic Design Automation, vol. 11, no. 3, pp. 249–361, 2017. DOI: 10.1561/1000000052. 1 Introduction Between the Snowden revelations and the seemingly unending series of high-profile hacks of the past few years, the public’s confidence in software systems has decreased considerably. At the same time, key initiatives such as cloud computing and the IoT (Internet of Things) are gaining popularity but require users to place much trust in the systems providing these services. We must therefore develop capa- bilities to build software systems with compelling security, and gain back our users’ trust. This manuscript is the second in a two part survey of the state of the art in secure processor systems, with a specific focus on remote software attestation and software isolation. Part I [Costan et al., 2017] established relevant background in computer system design (§ I.2) and security primitives (§ I.3), and surveyed relevant prior work (§ I.4). The same work discussed the attested software isolation container (en- clave): a modern primitive for modular secure software and trusted remote computation, as exemplified by Intel’s Software Guard Ex- tensions (§ I.5). This manuscript extends the discussion of enclaves and SGX by surveying the implementation and security properties of SGX (§ 2), 250 1.1. The Case for Hardware Isolation 251 and documents the authors’ concerns with its vulnerabilities to several classes of software attacks. Informed by the successes and shortcom- ings of SGX, this manuscript also discusses the MIT Sanctum proces- sor (§ 3): a secure processor that offers an equivalent programming model with strong security guarantees against an insidious software threat model including cache timing and memory access pattern at- tacks. With this work, we hope to enable a shift in discourse in secure hardware architecture away from plugging specific security holes to a principled approach to eliminating attack surfaces. 1.1 The Case for Hardware Isolation The best known practical method for securing a software system amounts to modularizing the system’s code in a way that minimizes code in the modules responsible for the system’s security. Formal ver- ification techniques are then applied to these modules, which make up the system’s trusted codebase (TCB). The method assumes that software modules are isolated, so the TCB must also include the mech- anism providing the isolation guarantees. Today’s systems rely on an operating system kernel, or a hypervi- sor (such as Linux or Xen, respectively) for software isolation. How- ever each of the last three years (2012-2014) witnessed over 100 new security vulnerabilities in Linux [cve, 2014a, Chen et al., 2011], and over 40 in Xen [cve, 2014b]. One may hope that formal verification methods can produce a se- cure kernel or hypervisor. Unfortunately, these codebases are far out- side our verification capabilities: Linux and Xen have over 17 million [Anthony, 2014] and 150,000 [xen, 2015] lines of code, respectively. In stark contrast, the seL4 formal verification effort [Klein et al., 2009] spent 20 man-years to cover 9,000 lines of code. Given Linux and Xen’s history of vulnerabilities and uncertain prospects for formal verification, a prudent system designer cannot in- clude either in a TCB (trusted computing base), and must look else- where for a software isolation mechanism. 252 Introduction Fortunately, Intel’s Software Guard Extensions (SGX) [McKeen et al., 2013, Anati et al., 2013] has brought attention to the alter- native of providing software isolation primitives in the CPU’s hard- ware. This avenue is appealing because the CPU is an unavoidable TCB component, and processor manufacturers have strong economic incentives to build correct hardware. 1.2 Intel SGX is Not the Answer Unfortunately, although the SGX design includes a vast array of de- fenses against a variety of software and physical attacks, it fails to offer meaningful software isolation guarantees. The SGX threat model pro- tects against all direct attacks, but excludes “side-channel attacks”, even if they can be performed via software alone. Alarmingly, cache timing attacks require only unprivileged software running on the victim’s host computer, and do not rely on any phys- ical access to the machine. This is particularly concerning in a cloud computing scenario, where gaining software access to the victim’s com- puter only requires a credit card [Ristenpart et al., 2009], whereas physical access is harder, requiring trespass, coercion, or social engi- neering on the cloud provider’s employees. Similarly, in many Internet of Things (IoT) scenarios, the process- ing units have some amount of physical security, but they run outdated software stacks that have known security vulnerabilities. For example, an attacker may exploit a vulnerability in an IoT lock’s Bluetooth stack and obtain software execution privileges, then mount a cache timing attack on its access-granting process, and obtain the crypto- graphic key that opens the lock. Furthermore, the analysis of SGX documentation as described in Part I of this work reveals that it is impossible for anyone but In- tel to reason about SGX’s security properties, because significant im- plementation details are not covered by the publicly available docu- mentation. This is a concern, as the myriad of security vulnerabilities [Wojtczuk and Rutkowska, 2011, 2009b, Wojtczuk et al., 2009, Duflot et al., 2006, Rutkowska and Wojtczuk, 2008, Wojtczuk and Rutkowska, 1.3. MIT Sanctum Processor 253 2009a, Wecherowski, 2009, Embleton et al., 2010] in TXT [Grawrock, 2009], Intel’s previous attempt at securing remote computation, show that securing the machinery underlying Intel’s processors is incredibly challenging, even in the presence of strong economic incentives. If a successor to SGX claimed to protect against cache timing at- tacks, substantiating such a claim would require an analysis of its hard- ware and microcode, and ensuring that no implementation detail is vulnerable to cache timing attacks. Barring a highly unlikely shift to open-source hardware from Intel, such analysis will never happen. A concrete example: the SGX documentation [Int, 2013, 2014] does not state where SGX stores the EPCM (enclave page cache map). If the EPCM is stored in cacheable RAM, page translation verification is subject to cache timing attacks. Interestingly, this detail is unnecessary for analyzing the security of today’s SGX implementation, as we know that SGX uses the operating system’s page tables, and page transla- tions are therefore vulnerable to cache timing attacks. The example does, however, demonstrate the fine nature of crucial details that are simply undocumented in today’s hardware security implementations. In summary, while the principles behind SGX have great potential, the SGX design does not offer meaningful isolation guarantees, and the SGX implementation is not open enough for independent researchers to be able to analyze its security properties. 1.3 MIT Sanctum Processor The Sanctum processor’s main contribution is a software isolation scheme that addresses the issues raised above: Sanctum’s isolation prov- ably defends against known software side-channel attacks, including cache timing attacks and passive address translation attacks. Sanctum is a co-design that combines minimal and minimally invasive hard- ware modifications with a trusted software security monitor that is amenable to rigorous analysis and does not perform cryptographic operations using keys.