One Bit Flips, One Cloud Flops: Cross-VM Row Hammer Attacks and Privilege Escalation Yuan Xiao, Xiaokuan Zhang, Yinqian Zhang, and Radu Teodorescu, The Ohio State University https://www.usenix.org/conference/usenixsecurity16/technical-sessions/presentation/xiao This paper is included in the Proceedings of the 25th USENIX Security Symposium August 10–12, 2016 • Austin, TX ISBN 978-1-931971-32-4 Open access to the Proceedings of the 25th USENIX Security Symposium is sponsored by USENIX One Bit Flips, One Cloud Flops: Cross-VM Row Hammer Attacks and Privilege Escalation Yuan Xiao Xiaokuan Zhang Yinqian Zhang Radu Teodorescu Department of Computer Science and Engineering The Ohio State University xiao.465, zhang.5840 @buckeyemail.osu.edu, yinqian, teodores @cse.ohio-state.edu { } { } Abstract ing the target memory regions, invalidate this assump- tion, raising broad security concerns. Row hammer attacks exploit electrical interactions be- Row hammer attacks exploit a vulnerability in the de- tween neighboring memory cells in high-density dy- sign of dynamic random-access memory (DRAM). Mod- namic random-access memory (DRAM) to induce mem- ern high-capacity DRAM has very high memory cell ory errors. By rapidly and repeatedly accessing DRAMs density which leads to greater electrical interaction be- with specific patterns, an adversary with limited privilege tween neighboring cells. Electrical interference from on the target machine may trigger bit flips in memory re- neighboring cells can cause accelerated leakage of ca- gions that he has no permission to access directly. In this pacitor charges and, potentially, data loss. Although paper, we explore row hammer attacks in cross-VM set- these so-called “disturbance errors” have been known tings, in which a malicious VM exploits bit flips induced for years, it has only recently been shown that these er- by row hammer attacks to crack memory isolation en- rors can be triggered by software. In particular, [23] has forced by virtualization. To do so with high fidelity, we demonstrated that malicious programs may issue spe- develop novel techniques to determine the physical ad- cially crafted memory access patterns, e.g., repeated and dress mapping in DRAM modules at runtime (to improve rapid activation of the same DRAM rows, to increase the effectiveness of double-sided row hammer attacks), the chances of causing a disturbance error in neighbor- methods to exhaustively hammer a large fraction of phys- ing rows. ical memory from a guest VM (to collect exploitable Row hammer vulnerabilities have been exploited in vulnerable bits), and innovative approaches to break security attacks shortly after its discovery [4, 10, 16, 20]. Xen paravirtualized memory isolation (to access arbi- In particular, Seaborn [4] demonstrated two privilege es- trary physical memory of the shared machine). Our study calation attacks that exploit row hammer vulnerabilities: also suggests that the demonstrated row hammer attacks One escaped from Google’s NaCl sandbox and the other are applicable in modern public clouds where Xen par- gained kernel memory accesses from userspace pro- avirtualization technology is adopted. This shows that grams running on Linux operating systems. Other stud- the presented cross-VM row hammer attacks are of prac- ies [10, 16, 20] aim to conduct row hammer attacks from tical importance. high-level programming languages, e.g., JavaScript, so that an adversary can induce memory errors and escalate 1 Introduction privileges remotely, by injecting malicious JavaScript code into the target’s web traffic (e.g., by hosting ma- Security of software systems is built upon correctly im- licious websites, cross-site scripting, man-in-the-middle plemented and executed hardware-software contracts. attacks, etc.). Violation of these contracts may lead to severe security In contrast to the client-side bit flip exploitations, breaches. For instance, operating system security re- server-side row hammer attacks are much less under- lies on the assumption that data and code stored in the stood. One particularly interesting scenario where memory subsystems cannot be altered without media- server-side row hammer attacks are of importance is tion by the software running with system privileges (e.g., in multi-tenant infrastructure clouds, where mutually- OS kernels, hypervisors, etc.). However, the recently distrusting cloud tenants (i.e., users of clouds) may co- demonstrated row hammer attacks [23], which are capa- locate their virtual machines (VM) on the same physical ble of inducing hardware memory errors without access- server, therefore sharing hardware resources, including USENIX Association 25th USENIX Security Symposium 19 DRAMs. Although server-grade processors and more Xen guest VMs. We first empirically study which row expensive DRAMs are believed to be less vulnerable to hammer attack methods (i.e., accessing memory with or row hammer attacks [23], studies have suggested that without mfence instructions, see Section 4) are most ef- even servers equipped with error correcting (ECC) mem- fective and lead to most bit flips. Then, in order to guar- ory are not immune to such attacks [12, 23]. antee that sufficient exploitable bit flips (i.e., located at In this paper, we aim to explore row hammer attacks specific memory locations and can be repeatedly induced in cross-VM settings, and shed some light on the secu- in row hammer attacks) are found, we conduct exhaus- rity, or lack thereof, in multi-tenant infrastructure clouds. tive row hammer attacks from a guest VM to test all The goal of this research is not to extensively study how DRAM rows that are accessible to the VM. Because each vulnerable the cloud servers are. Rather, we explore VM is limited to a small portion of the entire physical whether the isolation of cloud software systems—virtual memory, we also develop methods to explore more phys- machines and hypervisors—can be circumvented by row ical memory than assigned to our VM initially. In addi- hammer attacks (and if so, how?), should the underlying tion, we design a safe mode that makes bit flips induced hardware become vulnerable. by row hammer attacks less likely to crash the system. Towards this end, we demonstrate cross-VM row ham- Third, crack memory isolation enforced by virtual- mer attacks with high fidelity and determinism, which ization. Unlike prior work, which sprays large num- can be achieved in the following pipelined steps. bers of page tables and conducts random row hammer attacks hoping that bit flips will occur in a page table First, determine physical address mapping in DRAM. Double-sided row hammer attacks target a specific mem- entry (PTE) [4], in our approach (Section 5), we use hy- ory row by hammering its two neighboring rows to en- percalls to map page directories in the OS kernel of our hance the effectiveness of the attack [4, 23]. Conducting own VM to physical pages containing memory cells that such attacks, however, requires knowledge of the physi- are vulnerable to row hammer attacks. We then conduct cal memory mapping in DRAMs (i.e., bits in physical ad- row hammer attacks to deterministically flip the vulner- dresses that determine memory channels, DIMMs, ranks, able bit at anticipated positions in a page directory en- banks, and rows). This enables the identification of ad- try (PDE), making it point to a different page table. In dresses in neighboring rows of the same bank. How- the context of this paper, we call such attack techniques ever such information is not publicly available for In- page table replacement attacks to indicate that the orig- tel processors and memory controllers. Moreover, the inal page table has been replaced with a forged one. We same memory controller may map physical addresses to empirically demonstrate in Section 6 that such attacks al- DRAMs in different ways, depending on how DRAM low a Xen guest VM to have both read and write access modules are configured. to any memory pages on the machine. We demonstrate two examples to illustrate the power of the cross-VM row To address this issue, we developed a novel algo- hammer attacks: private key exfiltration from an HTTPS rithm to determine the memory mapping at runtime web server and code injection to bypass password au- (Section 3). Each bank in a DRAM chip has a row buffer thentication of an OpenSSH server. We emphasize that that caches the most recently used row in a bank. There- with the attack techniques we propose in this paper, the fore, by alternately accessing two rows in the same bank, attacker’s capability is only limited by imagination. we expect a higher memory access latency due to row buffer conflicts. The increase in access latency serves as We note our attacks primarily target Xen paravirtual- the basis for a timing channel which can be used to de- ized VMs, which, although are gradually superseded by termine if two physical memory addresses are mapped to hardware-assisted virtualization, are still widely used as the same DRAM bank. Building on the timing-channel cloud substrates in public cloud like Amazon EC2. This primitive, we developed a novel graph-based algorithm offers the adversary easy-to-break targets on servers with which models each bit in a physical address as a node in vulnerable hardware. Given the existing evidence of suc- a graph and establishes relationships between nodes us- cessful co-location attacks in public clouds [30, 32], we ing memory access latency. We show that the algorithm recommend discontinuing the use of such virtualization is capable of accurately detecting the row bits, column technology in cloud hosting services. bits and bank bits. We empirically show the algorithm Contributions. This paper makes the following contri- can accurately identify the DRAM mapping schemes au- butions to the field: tomatically within one or two minutes on the machines A novel graph-based algorithm incorporating timing- • we tested.