Throwhammer: Rowhammer Attacks over the Network and Defenses Andrei Tatar and Radhesh Krishnan Konoth, Vrije Universiteit Amsterdam; Elias Athanasopoulos, University of Cyprus; Cristiano Giuffrida, Herbert Bos, and Kaveh Razavi, Vrije Universiteit Amsterdam https://www.usenix.org/conference/atc18/presentation/tatar
This paper is included in the Proceedings of the 2018 USENIX Annual Technical Conference (USENIX ATC ’18). July 11–13, 2018 • Boston, MA, USA ISBN 978-1-939133-02-1
Open access to the Proceedings of the 2018 USENIX Annual Technical Conference is sponsored by USENIX. Throwhammer: Rowhammer Attacks over the Network and Defenses
Andrei Tatar Radhesh Krishnan Konoth Elias Athanasopoulos VU Amsterdam VU Amsterdam University of Cyprus Cristiano Giuffrida Herbert Bos Kaveh Razavi VU Amsterdam VU Amsterdam VU Amsterdam
Abstract never progressed beyond local privilege escalations or sandbox escapes. The attacker needs the ability to run Increasingly sophisticated Rowhammer exploits allow an code on the victim machine in order to flip bits in sen- attacker that can execute code on a vulnerable system sitive data. Hence, Rowhammer posed little threat from to escalate privileges and compromise browsers, clouds, attackers without code execution on the victim machines. and mobile systems. In all these attacks, the common In this paper, we show that this is no longer true and at- assumption is that attackers first need to obtain code tackers can flip bits by only sending network packets to execution on the victim machine to be able to exploit a victim machine connected to RDMA-enabled networks Rowhammer either by having (unprivileged) code exe- commonly used in clouds and data centers [1, 20, 45, 62]. cution on the victim machine or by luring the victim to a website that employs a malicious JavaScript applica- Rowhammer exploits today Rowhammer allows at- tion. In this paper, we revisit this assumption and show tackers to flip a bit in one physical memory location that an attacker can trigger and exploit Rowhammer bit by aggressively reading (or writing) other locations (i.e., flips directly from a remote machine by only sending hammering). As bit flips occur at the physical level, they network packets. This is made possible by increasingly are beyond the control of the operating system and may fast, RDMA-enabled networks, which are in wide use in well cross security domains. A Rowhammer attack re- clouds and data centers. To demonstrate the new threat, quires the ability to hammer memory sufficiently fast to we show how a malicious client can exploit Rowhammer trigger bit flips in the victim. Doing so is not always triv- bit flips to gain code execution on a remote key-value ial as several levels of caches in the memory hierarchy server application. To counter this threat, we propose often absorb most of the memory requests. To address protecting unmodified applications with a new buffer al- this hurdle, attackers resort to accessing cache eviction locator that is capable of fine-grained memory isolation buffers [12] or using direct memory access (DMA) [59] in the DRAM address space. Using two real-world ap- for hammering. But even with these techniques in place, plications, we show that this defense is practical, self- triggering a bit flip still requires hundreds of thousands contained, and can efficiently stop remote Rowhammer of memory accesses to specific DRAM locations within attacks by surgically isolating memory buffers that are tens of milliseconds. As a result, the current assumption exposed to untrusted network input. is that Rowhammer may only serve local privilege esca- lation, but not to launch attacks from over the network. 1 Introduction Remote Rowhammer attacks In this paper, we revisit this assumption. While it is true that millions of DRAM A string of recent papers demonstrated that the Rowham- accesses per second is harder to accomplish from across mer hardware vulnerability poses a growing threat to sys- the network than from code executing locally, today’s tem security. From a potential security hole in 2014 [36], networks are becoming very fast. Modern NICs are able it grew into an attack vector to mount end-to-end exploits to transfer large amounts of network traffic to remote in browsers [15, 29, 52], cloud environments [47, 51, 60], memory. In our experimental setup, we observed bit flips and smartphones [24, 59]. Recent work even generated when accessing memory 560,000 times in 64 ms, which Rowhammer-like bit flips on flash storage [17, 38]. Even translates to 9 million accesses per second. Even regular so, however advanced the attacks have become and how- 10 Gbps Ethernet cards can easily send 9 million packets ever worrying for the research community, these attacks per second to a remote host that end up being stored on
USENIX Association 2018 USENIX Annual Technical Conference 213 the host’s memory. Might this be enough for an attacker Contributions We make the following contributions: to effect a Rowhammer attack from across the network? In the remainder of this paper, we demonstrate that this • We describe Throwhammer, the first Rowhammer is the case and attackers can use these bit flips induced profiling tool that scans a host over the network for by network traffic to compromise a remote server appli- bit flips. We evaluate Throwhammer using differ- cation. To our knowledge, this is the first reported case ent configurations (i.e., different link speeds and of a Rowhammer attack over the network. Specifically, DIMMs). We then show how an attacker can use we managed to flip bits remotely using a commodity these bit flips to exploit a remote memcached server. 10 Gbps network. We rely on the commonly-deployed RDMA technology in clouds and data centers for reading • We design and implement ALIS, a new allocator from remote DMA buffers quickly to cause Rowhammer for safe allocation of network buffers. We show corruptions outside these untrusted buffers. These cor- that ALIS correctly finds guard rows at the DRAM ruptions allow us to compromise a remote memcached address space level, provided an address mapping server without relying on any software bug. that satisfies certain prerequisites. Furthermore, we show that ALIS is compatible with existing soft- Mitigating remote Rowhammer attacks It is un- ware. We further evaluate ALIS using microbench- clear whether existing hardware mitigations can protect marks and real-world applications to show that it in- against these dangerous network attacks. For instance, curs negligible performance and memory overhead. while clouds and data centers may (and sometimes do) use ECC memory to guard against bit flips, researchers • We release Throwhammer and ALIS as open-source have, from the first paper on Rowhammer [36], warned software in the URL that follows. that ECC may not be sufficient to protect against such attacks. Targeted Row Refresh, specifically designed https://vusec.net/projects/throwhammer to address Rowhammer is also shown not to be always Concerned parties can use Throwhammer to check effective [41, 59]. Unfortunately, existing software de- for remote bit flips and ALIS for protecting their fenses are also unprepared to deal with network attacks: applications against remote Rowhammer attacks. ANVIL [12] relies on performance counters that are not available for DMA, CATT [16] only protects the kernel from user-space attacks and VUsion [47] only protects 2 Background the memory deduplication subsystem. We make the observation that compared to local at- With software becoming increasingly more difficult due tackers, remote attackers can only target memory that is to a variety of defenses deployed in practice [11, 25, allocated for DMA buffers. Hence, instead of protecting 30, 39, 55, 57], security researchers have started ex- the entire memory, we only need to make sure that these ploring new directions for exploitation. For example, buffers cannot cause bit flips in the rest of the system. CPU caches can be abused to leak information [19, Specifically, we show that we can isolate the buffers for 26, 37, 42, 48, 61] or wide-spread reliability issues in fast network communication using a new memory allo- hardware [18, 36] can be abused to compromise soft- cation strategy that places CATT-like guard zones around ware [29, 52, 59]. These attacks require code execution them. These guard zones absorb any attacker-generated on the victim machine. In this paper, we show for the bit flips, protecting the rest of the system. Properly first time that this requirement is not strictly necessary, implementing this allocation strategy is not trivial: the and it is possible to trigger reliability issues in DRAM guard zones need to be placed in the DRAM address by merely sending network packets. We provide nec- space to effectively absorb the bit flips. Unfortunately, essary background on DRAM and its unreliabilities and the physical address space is not consecutively mapped high-speed networks that expose them remotely. to the DRAM address-space, unlike what is assumed in existing defenses [12, 16]. Memory controllers use 2.1 Unreliable DRAM complex functions to translate a physical address into a DRAM address. We therefore present a new alloca- DRAM Organization The basic unit of storage in tor, called ALIS (ALlocations ISolated), which uses a DRAM is cell made out of a capacitor used to hold the novel approach to translate between physical address- value of a single bit of information. Since capacitors space and DRAM address-space and safely allocates the leak charge over time, the memory controller should fre- DMA buffers and their guard zones. Since we need to quently (typically every 64 ms) recharge them in order to protect only a limited number of DMA buffers, doing so maintain the stored values, a process known as refresh- is inexpensive as we show using microbenchmarks and ing. DRAM cells are ganged together to form what is two real-world applications. known as a row (typically 1024 cells or columns wide).
214 2018 USENIX Annual Technical Conference USENIX Association 200 Infiniband flips in interesting offsets within a memory page and then Ethernet 180 Vulnerable DIMMs force the system to store sensitive information on that 160 140 memory page. For instance, the first known exploit by 120 100 Seaborn [52] finds a memory page with a bit flip that can Gbps 80 affect page table entries. It then frees that memory page 60 40 and sprays the system with page table pages. The hope Discovered Flips 20 is that the page that is now freed is used by a page table 0 page and the bit flip causes the page table entry to point 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 Year to another page table page, effectively giving the attacker control over all of physical memory. Similarly, Rowham- Figure 1: Trends in network performance and Rowhammer. mer.js [29], Drammer [59] and Xiao et al. [60] target page table pages but focus on browser, mobile and cloud Whenever a row is accessed the contents of that partic- environments respectively. Other attacks target crypto- ular row are put on a special buffer, called row buffer, graphic keys [51] or JavaScript objects [15, 24]. and the row is said to be activated. Once access is fin- ished, the activated row is written (i.e., recharged) with 2.2 Fast Networks the contents of the row buffer. Multiple rows along with a row buffer are stacked together to form a bank. There Figure 1 shows the evolution of network performance are multiple banks on a DRAM integrated circuit (IC). over time. Since 2010, the trend follows an exponen- Multiple DRAM ICs are laid out to for a DRAM rank. tial increase in the amount of available network band- The DRAM chips are accessed in parallel when reading width. This is putting a lot of pressure on other compo- a memory word. For example, with a DIMM that has 8 nents of the system, namely the CPU and the memory bit wide ICs, eight ICs are accessed in parallel to form a subsystem, and has forced systems engineers to rethink 64 bit memory word. their software stack in order to make use of all this band- width [22, 23, 33, 34, 43, 44]. DRAM addressing Addressing a memory word Figure 1 also shows that DIMMs with the Rowhammer within a DRAM rank is done by the system memory vulnerability have been produced since 2010 and their controller using three addresses: bank, row and column. production continues to date [40, 59]. As we will show Further parallelism can be added by having two ranks in §4, we observed bit flips with capacities available in on a single memory module (DIMM), adding multiple 10 Gbps or faster networks, suggesting that already back DIMMs on the same memory bus (also known as chan- in 2010, Rowhammer was exploitable over the network. nel), and providing multiple independent memory chan- While faster than 10 Gbps networks are very common nels. Hence, to address a specific word of memory, in data centers on bare metal [45, 53, 62], even today’s the memory controller uses a
USENIX Association 2018 USENIX Annual Technical Conference 215 cloud or data center machine that runs a server appli- Server Net Buffer Non-RDMA Networks Net Buffer Client User App App User cation. We assume that the target machine is vulnera- CPU CPU ble to Rowhammer bit flips [51, 60]. We further assume Kernel NIC NIC Kernel DMA Buffer DMA Buffer that the target system benefits from IOMMU protection. With IOMMU, the server’s NIC is not allowed to write Server Net Buffer RDMA-enabled Networks Net Buffer Client User App App User to memory pages that are not part of the pre-configured CPU CPU DMA areas by the server application. The end goal of the Kernel NIC NIC Kernel attacker is to bypass RDMA’s security model by modify- ing bits outside of memory areas that are registered for Figure 2: RDMA allows zero-copy network communication. DMA in order to compromise the system. ating system then signals the NIC that the packet is ready 4 Bit Flipping with Network Packets for network transfer. The NIC then reads the packet us- ing DMA and sends it over the wire. On the server side, To investigate the possibility of triggering bit flips over the server’s NIC receives the packet from the wire and the network, we built the first Rowhammer test tool that copies it to a DMA buffer that is pre-configured to the scans for bit flips by repeatedly sending or receiving NIC. It then signals the server’s CPU that a packet has packets to/from a remote machine, called Throwham- arrived. The CPU then copies the packet’s content to the mer. Throwhammer is implemented using 1831 lines of server application’s buffer. C code and runs entirely in user-space without requiring With RDMA, there is no need to involve the CPU on any special privileges. We will make this tool available both client and server for packet transfer. The server so that interested parties can check for remote bit flips. and client applications both configure DMA buffers to the NIC through interfaces that are provided by the oper- ating system. When the client application wants to send 4.1 Throwhammer’s Implementation a packet to a server application, it directly writes it to its buffer. It then signals its NIC that the packet is ready for Throwhammer makes use of RDMA capabilities for transfer. The NIC then sends the packet over the wire. transferring packets efficiently. Throwhammer has two On the server side, the NIC receives the packet and di- components: a server and a client process running on two rectly writes it to the buffer that has previously been con- nodes connected via an RDMA network. On the server figured by the server application. The server application side, we allocate a large virtually-contiguous buffer and can then be notified that a new packet has arrived or it configure it as a DMA buffer to the NIC. We set all bits can poll its own buffer. RDMA can boost existing pro- to one when checking for one-to-zero bit flips and do the tocols such as NFS [46] and new applications can use reverse when checking for zero-to-one bit flips. its functionalities to achieve better performance. Exam- On the client side, we repeatedly ask the server’s NIC ples include databases [43], distributed hash tables and to send us packets with data from various offsets within in-memory key-value stores [22, 23, 33, 34, 44]. this buffer. Given the remote nature of our attack, we cannot make any assumption on the physical addresses RDMA’s prevalence Data centers and cloud providers that map our target DMA buffers and cannot rely on side such as Google [45] and Microsoft [1, 62, 20] use RDMA channels for inferring this information [28]. Fortunately, to improve the performance of their clusters. Microsoft on the server side, the Linux kernel automatically turns very recently announced RDMA support for SMB file the memory backing our RDMA buffer into huge pages sharing in the workstation edition of Windows [5], sug- with its khugepaged daemon. This allows us to perform gesting RDMA-enabled networks are spreading into the double-sided Rowhammer similar to Rowhammer.js [29] workstation market. Cloud providers are already selling and Flip Feng Shui [51]. Periodically, we check the en- virtual machines with RDMA support. For example, Mi- tire buffer at the server for bit flips. crosoft Azure [3] and ProfitBricks [8] provide offerings To make the best possible use of the network capacity, with high-speed RDMA networks. we spawn multiple threads in Throwhammer. At each round, two aggressor addresses are chosen and all the threads send/receive packets that read from these two ad- 3 Threat Model dresses for a pre-defined number of times. We make no effort in synchronization between these threads, so mul- We consider attackers that generate and send legitimate tiple network packets may hit the row buffer. While we traffic through a high-speed network to a target server. leave potential optimizations for selecting aggressor ad- A common example is a client that sends requests to a dresses more carefully and better synchronization to fu-
216 2018 USENIX Annual Technical Conference USENIX Association 450 450 Hynix 40 Gbps - 52 mio pps 400 400 Corsair 33 Gbps - 42 mio pps 350 350 26 Gbps - 33 mio pps 300 300 19 Gbps - 23 mio pps 250 250 10 Gbps - 11 mio pps 200 200 150 150 Unique fl ips Unique fl ips 100 100 50 50 0 0 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Minutes Minutes
Figure 3: Number of unique Rowhammer bit flips over time Figure 4: Number of unique Rowhammer bit flips on Hynix using two sets of DIMMs over a 40 Gbps Ethernet network. DIMMs over time using different network configurations.
ture work, we show that Throwhammer already can trig- that matter) in between 40 Gbps and 10 Gbps. We can ger bit flips in 10 Gbps networks and above. however use fewer threads to emulate what the number of bit flips in networks that provide a bandwidth between 4.2 Results 10 Gbps and 40 Gbps (e.g., Amazon EC2 [2]). We mea- sure the pps for each configuration and use it to extrapo- Testbed We use two machines each with 8-core late the network bandwidth. Figure 4 shows the number Haswell i7-4790 processors connected using Mellanox of unique bit flips in different configurations as a function ConnectX-4 single port NICs as our evaluation testbed. of time. With 10 Gbps, we managed to trigger one bit flip Note that these cards are already old: at the time of after 700.7 seconds, showing that commodity networks this paper’s submission, two newer generations of these found in companies or university LANs are fast enough cards (ConnectX-5 and ConnectX-6) are available, but for triggering bit flips by transmitting network packets. we show that it is already possible to trigger bit flips with Starting with faster networks than 10 Gbps, Throwham- our older generation cards. We experiment with different mer can trigger many more bit flips during the 30 minutes DIMMs and varying network performance. window. Again, we believe a more optimized version of Throwhammer can potentially generate more bit flips, es- DIMMs We chose two pairs of DDR3 DIMMs con- pecially on 10 Gbps networks. figured in dual-channel mode, one from Hynix and one from Corsair. These DIMMs already show bit flips when we run the open source Rowhammer test [6]. We config- 5 Exploiting Bit Flips over the Network ured our NICs in 40 Gbps mode and ran Throwhammer for 30 minutes. Figure 3 shows the number of unique bit We now discuss how one can exploit remote bit flips flips as a function of time over these two sets of DIMMs caused by accessing RDMA buffers quickly. The ex- on the server triggered by transmitting packets. We could ploitation is similar to local Rowhammer exploits: the at- flip 464 unique bits on the Hynix DIMMs and 185 unique tacker needs to force the system to store sensitive data in bits on the Corsair DIMMs in 30 minutes. While these vulnerable memory locations before triggering Rowham- bit flips are already enough for exploitation, we believe mer to corrupt the data in order to compromise the sys- that it is possible to trigger many more bit flips with a tem. We exemplify this by building an end-to-end ex- more optimized version of Throwhammer. ploit against RDMA-memcached, a key-value store with RDMA support [32]. Network performance To understand how the net- work performance affects bit flips, we used the Hynix 5.1 Memcached architecture modules. We first configured our NICs in 10 Gbps Ether- net which can be saturated with 2 threads. We then con- Memcached stores key/value pairs as items within mem- figured our NICs in 40 Gbps Ethernet which can be satu- ory slabs of various sizes. By default, the smallest slab rated with 10 threads. The number of bit flips depends on class size is 96 bytes, with the largest being 1 MB. Mem- the number of packets that we can send over the network ory allocated is broken up into 1 MB sized chunks and (i.e., how many times we force a row to open) rather than then assigned into slab classes as necessary. For rapid the available bandwidth. For example, to trigger a bit retrieval of keys, memcached uses a hash table, with col- flip that happens by reading 300,000 times from each ag- liding items chained in a singly-linked list. gressor address in the refresh window of 64 ms locally, in The main data structure used for storing key/value perfect conditions (e.g., proper synchronization) we need items is called struct _stritem, as shown in Fig- 1000 to be able to transmit 64 ×300,000×2 = 9.375 million ure 5a. The first 50 bytes are used to store item metadata, packets per second (pps). Unfortunately our NICs do not while the remaining space is used to store the key and provide an option to reduce the bandwidth (or pps for the value. Items are chained together into two lists. A
USENIX Association 2018 USENIX Annual Technical Conference 217 Before *next ...... counterfeit item that we encode inside a legitimate item *prev h_next h_next h_next 50 ...... *h_next bytes — similar to Dedup Est Machina [15] and GLitch [24]. min. attacker attacker 96 ... controlled controlled Assuming we can reuse a 1 MB item for smaller items, bytes After key ...... we have to see which size class we should pick for our + h_next h_next h_next value ...... target items so that one of the items’ h_next pointer bit flip counterfeit struct _stritem item lands on a memory location with a bit flip. We do this (a) (b) analysis for every bit flip to see whether we can find a suitable size class for exploitation. Figure 5: Memcached Exploit. There are two challenges that we need to overcome for this strategy to work: first, we need to be able to chain first doubly linked list (LRU, identified by the next and many items together and second, we need to force mem- prev pointers) is updated during GET hits by relinking cached to reuse memory backing the 1 MB item with an recent items at the head, and is traversed when freeing exploitable bit flip for smaller items of the right size for unused items. A second singly linked list (hash chain, corrupting their h_next pointer. We discuss how we identified by the h_next pointer) is traversed when look- overcome these challenges next. ing up keys with colliding hashes. Another notable field is nbytes, the length of the value. Slabs come Memory massaging We first need to craft memcached in fixed number of classes decided at compile-time with items with different keys which hash to the same value. their metadata stored in a global data structure named Items with colliding keys make sure that the h_next slabclass. Crucially for our purposes, this data struc- pointer always points to an item that we control. Mem- ture contains slots, a pointer to (a list of) “free” items cached uses the 32 bit Murmur3 hash function on the key which are used for subsequent SET operations. to find the slot in a hashtable. This hash function is not cryptographically secure and we could easily generate 5.2 Exploiting memcached millions of colliding 8 byte keys. The simplest way to address the second challenge, is to The attack progresses in four steps. In the first step, we issue a DELETE request on the target 1 MB item. We pre- search for bit flips using GET requests. Once we find an viously calculated the exact size class that would allow exploitable bit flip, we perform memory massaging [51] us to land an h_next pointer on a location with a bit flip. to land a target struct _stritem on the memory loca- We can reassign the memory used by the deleted 1 MB tion with a bit flip. In the third step, we corrupt the hash item to the slab cache of the target item’s size class using chain to make a h_next value point to a counterfeit item the slabs reassign command from the memcached that we encode inside a valid item. Our counterfeit item client. Even without slabs reassign, we can easily provides us with limited read and write primitives, using trigger the reuse by just creating many items of the tar- program logic triggered by GET requests. Finally, we tar- get size. The LRU juggler component in the memcached get our limited write primitive towards the slabclass watches for free chunks in a slab class and reassigns any data structure, escalating it to arbitrary write and code free ones to the global page pool. execution. We describe each of the steps in more detail While it is possible to deterministically reuse 1 MB next. items after an exploitable bit flip in a complete im- plementation of RDMA-memcached, the current ver- Finding exploitable bit flips We spray the entire avail- sion of RDMA-memcached does not support DELETE or able memory with 1 MB sized key-value items with val- slabs reassign. In these cases, we can simply spray ues made out of binary value one (when looking for 1 to the memory with items with a size that maximizes the of 0 bit flips). Filling up all the available memory makes probability of corrupting the h_next pointer. We later sure that some key-value items eventually border on the report on the success rate of both attacks. initial 16 MB RDMA buffers. Our experiments show that this is always the case. The attacker now remotely ham- Corrupting the target item Once we land our target mers the initial 16 MB RDMA buffers to trigger bit flips item on the desired location in memory, we re-trigger the in the pages that belong to the adjacent rows where some bit flip by transmitting packets from the RDMA buffers. of the 1 MB items are now stored. After that, the attacker This causes the corruption of the target item’s h_next reads back the items with GET requests to find out which pointer. With the right corruption, the pointer will now bit offsets have been corrupted. We now discuss which point inside another item whose key/value contents we offsets are exploitable. control. By carefully crafting a counterfeit header inside Our target is corrupting the hash chain (i.e., the the value area, we gain either a limited read or write ca- h_next pointer of a struct _stritem). As shown in pability. Issuing a GET request on our counterfeit item Figure 5b, this allows us to pivot the h_next pointer to a will now retrieve nbytes contiguous bytes from mem-
218 2018 USENIX Annual Technical Conference USENIX Association cached’s address space, provided the memory is mapped. 6.1 Challenges of Finding Adjacent Rows Attempting to read unmapped memory is handled grace- fully with an error being returned to the client, preventing The main idea behind ALIS is simple: given that unwanted crashes. If our counterfeit item is not LRU- Rowhammer bit flips happen in victim rows adjacent to linked (i.e., next=prev=NULL), the GET handler returns aggressor rows, we need to make sure that all accessible without any additional side-effects. A linked item, how- rows in an isolated buffer are separated from the rest of ever, will trigger a relink operation on the next GET. By system memory by at least one guard row used to absorb controlling the next (n) and prev (p) pointers and tak- said bit flips. The implementation of this idea, however, ing advantage of the unlink step, we gain a limited write is not simple because finding all possible victim rows primitive, where *p=n and *(n+8)=p, if p and n are re- along with their neighbors is not straightforward. spectively non-NULL. Given that bit flips happen on the DRAM ICs, ALIS should isolate rows in the DRAM address space. Re- Escalation The RDMA-memcached binary is not call from §2.1 that the DRAM address space is defined compiled as a Position Independent Executable (PIE); using the
USENIX Association 2018 USENIX Annual Technical Conference 219 Physical Address Space DRAM Address Space Bank 0 Buffer+ +0 Physical Address Space DRAM Address Space Physical Address Space DRAM Address Space 1 1 Buffer +0 +64K 5 Buffer+ +0 16K rows 1 RANK 0 RANK 1 1 2 ... ++2M ++256 +128K CHAN 0 2 2 3 2 ++4M 1 1 ++512 +192K 3 6 ++6M 2 2 3 Row X 4 16K rows ++768 1 3 5 7 ... +256K ... 4 3 3 4 5 ++8M 4 4 ++1K +320K 5 5 3 +10M 5 8 +1.25K CHAN 1 6 7 +384K 6 6 7 6 16K rows ... +12M +1.5K 7 7 7 6 7 Row X 2 4 6 8 ... +448K +14M +1.75K 8 4 8 8 5 8 +512K 8 +16M +2K (a) (b) (c)
Figure 6: Examples of nonlinear DRAM address mappings taken from real systems [10, 49, 54].
6.2 Design end of this step, we have a complete view of the allo- cated buffer in DRAM address space. We now discuss how ALIS addresses these challenges. We define the row group of a page to be the set of rows Pass 1: Marking. ALIS iterates through the rows of that page maps onto. Similarly, the page group of a row the buffer in DRAM address order and marks all (allo- is the set of pages with portions mapped to that row. In cated) pages of a row as follows: Partially allocated rows addition, in the context of a user-space process, we say have their pages marked as UNSAFE. Completely allo- a page is allocated if it is mapped by the system’s MMU cated (i.e. full) rows preceded or followed by a partially into its virtual address space. Likewise, we say a row is allocated or empty row are marked EDGE. Due to the full if all pages in its page group are allocated, and partial fullness property we can be certain that any partial row if only a strict subset of these are allocated. If no page is groups have their pages marked UNSAFE at the first oc- allocated we call a row empty. currence of one of its members in the enumeration. We ALIS requires a physical to DRAM address mapping can therefore be certain that a row, once concluded safe, that satisfies the following condition: any two pages of will not be marked otherwise later. In addition, the enu- an arbitrary row’s page group have identical row groups meration property guarantees that if an edge row is found themselves. If this condition holds, we can prove the later in the pass, such as block 4 in Figure 6b, previous following two properties: rows containing the same pages will be correctly “back- (1) Enumeration property: To list all rows accessible marked” as EDGE. by owning pages in a page group, it is sufficient to list any such page’s row group. Pass 2: Gathering. ALIS now makes a second pass (2) Fullness property: Rows that share page groups over the DRAM rows, searching for contiguous row have the same allocation status — a row is full, partial or blocks of unmarked pages, bordered on each side by rows empty if and only if all rows in its pages’ row group are with marked EDGE. We add each of these row blocks respectively full, partial or empty. to a list while marking all their pages as USED. At the For the interested reader we present a formal descrip- end of this step we have a complete list of all guard- tion of these properties, along with sufficiency criteria able memory areas immediately available for allocation. and proof that they hold for common architectures in [7]. Note these row blocks are isolated from each other and Assuming a mapping where these properties hold, we all other system memory by a padding of at least one now discuss how ALIS allocates isolated RDMA buffers. guard row.
6.3 Allocation Algorithm Pass 3: Pruning. In this final cleanup pass, ALIS it- erates through allocated pages, unmapping and returning Preparatory Steps. Initially, ALIS reserves a buffer and to the OS pages that aren’t marked as USED, freeing up locks it in memory, so that any virtual memory map- any non-essential memory locked by the previous steps. pings are not changed through the course of allocation or usage. Subsequently, ALIS translates the physical Reservation and Mapping. ALIS can now use the data pages that back the reserved buffer into to their respective structure obtained in the previous steps to allocate iso- DRAM addresses. Translating from physical addresses lated buffers using one or more row blocks. Applications to DRAM addresses is performed using mapping func- can map the (physical) pages in these buffers into the tions either available through manufacturer documenta- virtual address space at desired locations to satisfy the tion [10] or previously reverse-engineered [49]. At the allocation request.
220 2018 USENIX Annual Technical Conference USENIX Association 6.4 Implementation own statically-linked implementation. We hence needed to perform a simple binary instrumentation to instead We have implemented ALIS on top of Linux as a user- jump to our implementation of posix_memalign which space library using 2518 lines of C code. ALIS reserves allocates isolated buffers. memory by mapping a file descriptor associated with anonymous shared memory (i.e., a memfd on Linux). ALIS uses the /proc/self/pagemap interface [35], to 7.2 HERD translate the buffer’s virtual addresses to physical ad- dresses. Finally, ALIS maps particular page frames into HERD [34] is a key-value store that leverages RDMA the process’ virtual address space using the mmap system to deliver low latency and high throughput. Unlike sim- call by providing specific offsets into the memfd to spe- ilar systems, HERD has been designed with RDMA in cific virtual addresses using the MAP FIXED flag. These mind. The system offers clean RDMA primitives, and mechanisms allow ALIS to seamlessly replace memory heavily relies on RDMA to reduce round-trip times, re- allocation routines used by applications with an isolated duce latency, and maximize throughput. As a case-study, version. ALIS supports translation between the physical HERD is ideal, since simply turning RDMA off is not an address space to the DRAM address space for the mem- option; the system is primarily designed around the con- ory controller of all major CPU architectures. cept of RDMA. Thus, if anyone needs to take advantage of the performance of HERD, they additionally need to secure its RDMA buffer, otherwise its users run the se- 7 Protecting Applications with ALIS curity risks of remote Rowhammer attacks. HERD’s initializer process uses shmget to allocate the In this section, we show how we used ALIS to protect RDMA buffer and share it with its worker process. While two popular applications that provide distributed key- we could extend ALIS to support shmget, instead we value services, namely memcached [32] and HERD [34] opted to declare a global variable in HERD’s initializer against remote Rowhammer attacks. One key observa- process to store allocated the RDMA buffer address and tion is that there are a few different ways to allocate pass it to the worker process. This required modifying space for the RDMA buffer. Since we are interested in 10 lines of code in HERD. isolating the memory used as the RDMA buffer for con- taining RDMA bit flips, it is crucial to understand the way each application manages memory for its RDMA buffers. Our allocator is capable of handling common 8 Evaluation cases such as when the memory is allocated with mmap or posix_memalign while it can be extended to support We use the same testbed that we used in §4 for our evalu- additional constructs. We now discuss the specifics of ation. Firstly we made sure ALIS was indeed protecting the applications which we tried with our custom alloca- our system from bit flips. We modified Throwhammer’s tor. We evaluate the performance of both systems when client to allocate isolated RDMA buffers using ALIS. deployed using our custom allocator in §8.2. Hammering remotely for extended periods of time with different strategies did not generate any bit flips outside the RDMA buffers unlike previously, thus enforcing the 7.1 Memcached RDMA security model (§3). Many large-scale Internet services use DRAM-based We now evaluate in detail the memory overhead and key-value caches like memcached. Usually, memcached performance impact of protecting applications. serves as a cache in front of a back-end database. Mem- cached with RDMA support [32] provides lower latency and higher throughput compared to the original mem- 8.1 Allocation Overhead cached. This is done by introducing traditional RDMA- To calculate the overhead of ALIS, we wrote a simple enabled set and get APIs. Given that memcached is a test program that allocates an isolated buffer of a given popular application, exploitable bit flips caused over the size and reports how long it took for the allocation to suc- network as we discussed in §5 can affect many users. ceed. Note that in most cases, we only pay this (modest) RDMA-memcached is not open-source. We hence overhead once at application initialization time. Given reverse engineered its binary to discover that it uses that ALIS pools these allocations, in cases where an ap- posix_memalign for allocating the RDMA buffers. plication re-allocates these buffers (e.g., [9, 46]), the sub- Total size of these RDMA-buffers is approximately sequent allocations of the same size will be fast. We also 5 MB. Unfortunately, instead of using the standard libc- collected statistics from our allocator on the number of provided posix_memalign, memcached-rdma uses its extra pages that we had to allocate for guard rows.
USENIX Association 2018 USENIX Annual Technical Conference 221 100 100 100 100 100 100 Time Time Time Memory Memory Memory 80 80 80 80 80 80
60 60 60 60 60 60
Time (s) 40 40 Time (s) 40 40 Time (s) 40 40
20 20 Memory overhead (MB) 20 20 Memory overhead (MB) 20 20 Memory overhead (MB)
0 0 0 0 0 0 4 8 4 8 4 8 16 32 64 16 32 64 16 32 64 128 256 512 128 256 512 128 256 512 1024 2048 4096 1024 2048 4096 1024 2048 4096 8192 MB MB MB (a) Two 4 GB (single rank, dual channel). (b) Single 8 GB (two ranks). (c) Four 4 GB (single rank, dual channel).
100 100 100 100 100 100 Time Time Time Memory Memory Memory 80 80 80 80 80 80
60 60 60 60 60 60 Time (s) Time (s) 40 40 Time (s) 40 40 40 40 Memory overhead (MB) Memory overhead (MB) Memory overhead (MB) 20 20 20 20 20 20
0 0 0 0 0 0 4 8 16 32 64 4 8 4 8 128 256 512 16 32 64 16 32 64 1024 2048 4096 8192 128 256 512 128 256 512 16384 1024 2048 4096 8192 1024 2048 4096 8192 MB MB MB (d) Two 8 GB (two ranks, single channel). (e) Two 8 GB (two ranks, dual channel). (f) Four 8 GB (two ranks, dual channel). Figure 7: The allocation time and memory overhead of isolating RDMA buffers of various sizes in different configurations.
70 Configurations We experimented with all possible memcached SET memcached secured SET 60 memcached GET configurations including multiple DRAM modules, memcached secured GET channels, and ranks. We assume that up to half of the 50 memory can be used for RDMA buffers, but nothing 40 30
stops us from increasing this limit (e.g., 80%). We run Latency (us) 20 each measurement 5 times and report the mean value. 10
Figure 7 shows two general expected trends in all con- 0 1 2 4 8 16 32 64 128 256 512 4096 8192 figurations: 1. the allocation time increases as we request 1024 2048 16384 32768 65536 131072 262144 524288 a larger allocation due to the required extra computation, Object Size (bytes) 2. the amount of extra memory that our allocator needs for guard rows increases only modestly as we allocate Figure 8: Secured memcached performance. larger safe buffers. In fact, the relative overhead becomes much smaller as we allocate larger buffers. 8.2 RDMA Performance We also make a number of other observations: We now report on the performance implications of using 1. The size of installed memory does not affect the al- ALIS on RDMA-memcached and HERD (§7). We use location performance (Figure 7a vs. Figure 7c), the same testbed that we used in our remote bit flip study (§4) and use the benchmarks provided by the applica- 2. Increasing the number of ranks and channels in- tions. The benchmark included in RDMA-memcached creases the allocation time. measures the latency of SET and GET requests with 3. The number of ranks increases the allocation time varying value sizes. Figure 8 shows that our custom allo- more than the number of channels (Figure 7a vs. cator only introduces negligible performance overhead in Figure 7b and Figure 7c vs. Figure 7d). Given that memcached-rdma. This is expected because ALIS only column address bits slice the DRAM address space introduces a small overhead during initialization. into finer chunks than the channel bits, our alloca- The benchmark included with HERD reports the tor requires more computation to find safe memory throughput of HERD in terms of number of requests pages when rank mirroring is active. per second. Our measurements show that isolating RDMA buffers in HERD reduces the performance by In general, allocating larger buffers slightly increases 0.4% which is negligible. The original HERD paper [34] the amount of memory required for isolating the buffers achieves the throughput of 26 million requests per sec- given that our allocator stitches multiple safe blocks to- ond by using multiple client machines and a server ma- gether to satisfy the requested size. Interestingly, as we chine with two processors. The authors of HERD ver- allocate more memory, the allocator requires fewer areas ified that our throughput baseline is expected with our and in some cases, this reduces the amount of memory testbed. Hence, we conclude that ALIS does not incur required for guard rows (Figure 7a and Figure 7b). any runtime overhead while isolating RDMA buffers.
222 2018 USENIX Annual Technical Conference USENIX Association 9 Related work Another software-based solution, ANVIL [12] also lacks the translation information for implementing a Attacks Rowhammer was initially conceived in 2014 proper protection. It relies on Intel’s performance moni- when researchers experimentally demonstrated flipping toring unit (PMU) that can capture precisely which phys- bits in DDR3 for x86 processors by just accessing other ical addresses cause many cache misses. By access- parts of memory [36]. Since then, researchers have pro- ing the neighboring rows of these physical addresses, posed increasingly sophisticated Rowhammer exploita- ANVIL manually recharges victim rows to avoid bits tion techniques, on browsers [52, 15, 29], clouds [14, 51, to flip. An improved version of ANVIL with proper 60], and ARM-based mobile platforms [59]. There have physical to DRAM translation can be an ideal software also been reports of bit flips on DDR4 modules [41, 59]. defense against remote Rowhammer attacks. Unfortu- Finally, recent attacks have focused on bypassing state- nately, Intel’s PMU (or AMD’s) does not capture pre- of-the-art Rowhammer defenses [27, 56]. cise address information when memory accesses bypass In all of these instances, the attacker needs to find a the cache through DMA. Hence, our allocator can pro- way to trigger the right bit flips that can alter critical data vide the necessary protection for remote DMA attacks (page tables, cryptographic keys, etc.) and thus affect (or even local DMA attacks [59]) while processor ven- the security of the system. All these cases assume that dors extend the capabilities of their PMUs. the attacker has local code execution. In this paper, we showed how an adversary can induce bit flips by merely sending network packets. 10 Conclusion
Thus far, Rowhammer has been commonly perceived as Defenses Although we have plenty of advanced at- a dangerous hardware bug that allows attackers capable tacks exploiting bit flips, defenses are still behind. We of executing code on a machine to escalate their privi- stress here that for some of the aforementioned attacks leges. In this paper, we have shown that Rowhammer is that affect real products, vendors often disable software much more dangerous and also allows for remote attacks features. Linux kernel disabled unprivileged access to in practical settings. Remote Rowhammer attacks place pagemap [35] in response to Seaborn’s attack [52], Mi- different demands on both the attackers and the defend- crosoft disabled deduplication [21] in response to the ers. Specifically, attackers should look for new ways to Dedup Est Machina attack [15], Google disabled the massage memory in a remote system. Meanwhile, de- ION contiguous heap [58] in response to the Dram- fenders can no longer prevent Rowhammer by banning mer attack [59] and further disabled high-precision GPU local execution of untrusted code. We showed how an timers [4] in response to the GLitch attack [24]. A similar attacker can exploit remote bit flips in memcached to ex- reaction to Throwhammer could be potentially disabling emplify a remote Rowhammer attack. We further pre- RDMA, which (a) in not realistic, and (b) does not solve sented a novel defense mechanism that physically iso- the problem entirely. Therefore, we presented ALIS, a lates the RDMA buffers from the rest of the system. This custom allocator that isolates a vulnerable RDMA buffer shows that while it may be hard to prevent Rowham- (and can in principle isolate any vulnerable to hammer- mer bit flips altogether without wide-scale hardware up- ing buffer in memory). ALIS is quite practical, since, grades, it is possible to contain their damage in software. compared to other proposals [12, 16], it is completely im- plemented in user-space, compatible with existing soft- ware, and does not require special hardware features. Disclosure More precisely, CATT [16] can only protect kernel memory against Rowhammer attacks. We showed, how- We have cooperated with the National Cyber Security ever, that it is possible to target user applications with Centre in the Netherlands to coordinate disclosure of the Rowhammer over the network. Furthermore, CATT re- vulnerabilities to the relevant parties. quires kernel modification which introduce deployment issues (especially in the case of data centers). In particu- lar, it applies a static partitioning between memory used Acknowledgements by the kernel and the user-space. The kernel, however, often needs to move physical memory between different We would like to thank the anonymous reviewers for zones depending on the currently executing workload. In their valuable feedback. This work was supported in comparison, our proposed allocator is flexible, does not part by the MALPAY project and in part by the Nether- require modification to the kernel, and unlike CATT, can lands Organisation for Scientific Research through grants safely allocate memory by taking the physical to DRAM NWO 639.023.309 VICI “Dowsing”, NWO 639.021.753 address space translation into account. VENI “PantaRhei”, and NWO 629.002.204 “Parallax”.
USENIX Association 2018 USENIX Annual Technical Conference 223 References [20]C OSTA, P., BALLANI,H.,RAZAVI,K., AND KASH, I. R2C2: A Network Stack for Rack-scale Computers. SIGCOMM’15. [1] About H-series and compute-intensive A-series VMs for [21] CVE-2016-3272. Microsoft Security Bulletin MS16-092 Windows. https://docs.microsoft.com/en-us/azure/ - Important. https: // technet. microsoft. com/ en-us/ virtual-machines/windows/a8-a9-a10-a11-specs, Re- library/ security/ ms16-092. aspx (2016). trieved 31.05.2018. ´ [2] Amazon EC2 Instance Types. https://aws.amazon.com/ [22]D RAGOJEVIC,A.,NARAYANAN,D.,HODSON,O., AND CAS- ec2/instance-types, Retrieved 31.05.2018. TRO, M. FaRM: Fast Remote Memory. NSDI’14. ´ [3] Compute Intensive Instances. https://docs.microsoft. [23]D RAGOJEVIC,A.,NARAYANAN,D.,NIGHTINGALE,E.B., com/en-us/azure/virtual-machines/windows/ RENZELMANN,M.,SHAMIS,A.,BADAM,A., AND CASTRO, sizes-hpc, Retrieved 31.05.2018. M. No Compromises: Distributed Transactions with Consis- tency, Availability, and Performance. SOSP’15. [4] GLitch vulnerability status. http://www.chromium.org/ chromium-os/glitch-vulnerability-status Retrieved [24]F RIGO, P., GIUFFRIDA,C.,BOS,H., AND RAZAVI, K. Grand 31.05.2018. pwning unit: Accelerating microarchitectural attacks with the gpu. SP’18. [5] Microsoft announces Windows 10 Pro for Workstations. https://blogs.windows.com/business/2017/08/10/ [25]G EORGE, V., PIAZZA, T., AND JIANG, H. Technology insight: microsoft-announces-windows-10-pro-workstations/, Intel R next generation microarchitecture codename ivy bridge. Retrieved 31.05.2018. In Intel Developer Forum (2011). [6] Program for testing for the DRAM “rowhammer” problem. [26]G RAS,B.,RAZAVI,K.,BOSMAN,E.,BOS,H., AND GIUF- https://github.com/google/rowhammer-test, Retrieved FRIDA, C. ASLR on the Line: Practical Cache Attacks on the 31.05.2018. MMU. NDSS’17. [7] Properties of a Physical-to-DRAM Address Mapping. https:// [27]G RUSS,D.,LIPP,M.,SCHWARZ,M.,GENKIN,D.,JUFFIN- www.vusec.net/download/?t=papers/dram-formal.pdf. GER,J.,O’CONNELL,S.,SCHOECHL, W., AND YAROM, Y. Another flip in the wall of rowhammer defenses. In S&P’18. [8] Technical Info: A Deep Dive Into ProfitBricks. https://www. profitbricks.com/technical-info Retrieved 31.05.2018. [28]G RUSS,D.,MAURICE,C.,FOGH,A.,LIPP,M., AND MAN- GARD, S. Prefetch Side-Channel Attacks: Bypassing SMAP and [9] MPI One-Sided Communication, 2014. https: Kernel ASLR. CCS’16. //software.intel.com/en-us/blogs/2014/08/06/ one-sided-communication, Retrieved 31.05.2018. [29]G RUSS,D.,MAURICE,C., AND MANGARD, S. Rowham- mer.js: A Remote Software-Induced Fault Attack in JavaScript. [10]A DVANCED MICRO DEVICES. BIOS and Kernel Developers DIMVA’16. Guide (BKDG) for AMD Family 15h Models 60h-6Fh Proces- sors. May 2016. [30]I NTEL, I. Intel-64 and ia-32 architectures software developer’s manual. Volume 3A: System Programming Guide, Part 1, 64 [11]A NDERSEN,S., AND ABELLA, V. Data execution prevention. (2013). changes to functionality in microsoft windows xp service pack 2, part 3: Memory protection technologies, 2004. [31]J EONG, E. Y., WOO,S.,JAMSHED,M.,JEONG,H.,IHM,S., HAN,D., AND PARK, K. mTCP: A Highly Scalable User-level [12]A WEKE,Z.B.,YITBAREK, S. F., QIAO,R.,DAS,R.,HICKS, TCP Stack for Multicore Systems. NSDI’14. M.,OREN, Y., AND AUSTIN, T. ANVIL: Software-Based Pro- tection Against Next-Generation Rowhammer Attacks. ASP- [32]J OSE,J.,SUBRAMONI,H.,LUO,M.,ZHANG,M.,HUANG,J., LOS’16. WASI-UR RAHMAN,M.,ISLAM,N.S.,OUYANG,X.,WANG, H.,SUR,S., AND PANDA, D. K. Memcached Design on High [13]B ELAY,A.,PREKAS,G.,KLIMOVIC,A.,GROSSMAN,S., Performance RDMA Capable Interconnects. ICPP’11. KOZYRAKIS,C., AND BUGNION, E. IX: A Protected Data- plane Operating System for High Throughput and Low Latency. [33]K ALIA,A.,KAMINSKY,M., AND ANDERSEN, D. G. FaSST: OSDI’14. Fast, Scalable and Simple Distributed Transactions with Two- sided (RDMA) Datagram RPCs. OSDI’16. [14]B HATTACHARYA,S., AND MUKHOPADHYAY, D. Curious Case of Rowhammer: Flipping Secret Exponent Bits Using Timing [34]K ALIA,A.,KAMINSKY,M., AND ANDERSEN, D. G. Using Analysis. CHESS’16. RDMA Efficiently for Key-value Services. SIGCOMM’14. [15]B OSMAN,E.,RAZAVI,K.,BOS,H., AND GIUFFRIDA,C. [35] KERNEL,L. https://www.kernel.org/doc/ Dedup Est Machina: Memory Deduplication as an Advanced Ex- Documentation/vm/pagemap.txt, Retrieved 31.05.2018. ploitation Vector. SP’16. [36]K IM, Y., DALY,R.,KIM,J.,FALLIN,C.,LEE,J.H.,LEE,D., [16]B RASSER, F., DAVI,L.,GENS,D.,LIEBCHEN,C., AND WILKERSON,C.,LAI,K., AND MUTLU, O. Flipping Bits in SADEGHI, A.-R. CAn’t Touch This: Software-only Mitigation Memory Without Accessing Them: An Experimental Study of against Rowhammer Attacks targeting Kernel Memory. SEC’17. DRAM Disturbance Errors. ISCA’14. [17]C AI, Y., GHOSE,S.,LUO, Y., MAI,K.,MUTLU,O., AND [37]K OCHER, P., GENKIN,D.,GRUSS,D.,HAAS, W., HAMBURG, HARATSCH, E. F. Vulnerabilities in MLC NAND flash mem- M.,LIPP,M.,MANGARD,S.,PRESCHER, T., SCHWARZ,M., ory programming: experimental analysis, exploits, and mitiga- AND YAROM, Y. Spectre attacks: Exploiting speculative execu- tion techniques. HPCA’17. tion. arXiv preprint arXiv:1801.01203 (2018). [18]C AI, Y., GHOSE,S.,LUO, Y., MAI,K.,MUTLU,O., AND [38]K URMUS,A.,IOANNOU,N.,PAPANDREOU,N., AND PAR- HARATSCH, E. F. Vulnerabilities in MLC NAND Flash Memory NELL, T. From random block corruption to privilege esca- Programming: Experimental Analysis, Exploits, and Mitigation lation: A filesystem attack vector for rowhammer-like attacks. Techniques. HPCA’17. WOOT’17. [19]C OCK,D.,GE,Q.,MURRAY, T., AND HEISER, G. The Last [39]K UZNETSOV, V., SZEKERES,L.,PAYER,M.,CANDEA,G., Mile: An Empirical Study of Timing Channels on seL4. CCS’14. SEKAR,R., AND SONG, D. Code-pointer Integrity. OSDI’14.
224 2018 USENIX Annual Technical Conference USENIX Association [40]L ANTEIGNE, M. A Tale of Two Hammers: A Brief Rowham- [53]S INGH,A.,ONG,J.,AGARWAL,A.,ANDERSON,G.,ARMIS- mer Analysis of AMD vs. Intel. http://www.thirdio.com/ TEAD,A.,BANNON,R.,BOVING,S.,DESAI,G.,FELDER- rowhammera1.pdf, May 2016. MAN,B.,GERMANO, P., KANAGALA,A.,PROVOST,J.,SIM- MONS,J.,TANDA,E.,WANDERER,J.,HOLZLE¨ ,U.,STUART, [41]L ANTEIGNE, M. How Rowhammer Could Be Used to Exploit S., AND VAHDAT, A. Jupiter Rising: A Decade of Clos Topolo- Weaknesses in Computer Hardware. http://www.thirdio. gies and Centralized Control in Google’s Datacenter Network. com/rowhammer.pdf, March 2016. SIGCOMM’15. [42]L IPP,M.,SCHWARZ,M.,GRUSS,D.,PRESCHER, T., HAAS, W., MANGARD,S.,KOCHER, P., GENKIN,D., [54] STANDARD, J. DDR3 SDRAM. JESD79-3C, Nov 2008. YAROM, Y., AND HAMBURG, M. Meltdown. arXiv preprint [55]T ANG,J., AND TEAM, T. M. T. S. Exploring con- arXiv:1801.01207 (2018). trol flow guard in windows 10. http: // blog. [43]L IU, F., YIN,L., AND BLANAS, S. Design and Evaluation of trendmicro. com/ trendlabs-security-intelligence/ an RDMA-aware Data Shuffling Operator for Parallel Database exploring-control-flow-guard-in-windows-10 , Re- Systems. EuroSys’17. trieved 31.05.2018.
[44]M ITCHELL,C.,GENG, Y., AND LI, J. Using One-sided RDMA [56]T ATAR,A.,RAZAVI,K.,BOS,H., AND GIUFFRIDA, C. Defeat- Reads to Build a Fast, CPU-efficient Key-value Store. USENIX ing software mitigations against rowhammer: a surgical precision ATC’13. hammer. In RAID’18. [45]M ITTAL,R.,LAM, V. T., DUKKIPATI,N.,BLEM,E.,WASSEL, [57]T ICE,C.,ROEDER, T., COLLINGBOURNE, P., CHECKOWAY, H.,GHOBADI,M.,VAHDAT,A.,WANG, Y., WETHERALL,D., S.,ERLINGSSON,U.,LOZANO,L., AND PIKE, G. Enforc- AND ZATS, D. TIMELY: RTT-based Congestion Control for the ing Forward-edge Control-flow Integrity in GCC and LLVM. Datacenter. SIGCOMM’15. SEC’14. [46]N ORONHA,R.,CHAI,L.,TALPEY, T., AND PANDA, D. K. De- signing NFS with RDMA for Security, Performance and Scala- [58]T JIN, P. android-7.1.0 r7 (Disable ION HEAP TYPE SYSTEM bility. ICPP’07. CONTIG). https: // android. googlesource. com/ device/ google/ marlin-kernel/ +/ android-7. 1. 0_ r7 [47]O LIVERIO,M.,RAZAVI,K.,BOS,H., AND GIUFFRIDA,C. (2016). Secure Page Fusion with VUsion. SOSP’17. [59] VAN DER VEEN, V., FRATANTONIO, Y., LINDORFER,M., [48]O REN, Y., KEMERLIS, V. P., SETHUMADHAVAN,S., AND GRUSS,D.,MAURICE,C.,VIGNA,G.,BOS,H.,RAZAVI,K., KEROMYTIS, A. D. The Spy in the Sandbox: Practical Cache AND GIUFFRIDA, C. Drammer: Deterministic Rowhammer At- Attacks in JavaScript and their Implications. CCS’15. tacks on Mobile Platforms. CCS’16.
[49]P ESSL, P., GRUSS,D.,MAURICE,C.,SCHWARZ,M., AND [60]X IAO, Y., ZHANG,X.,ZHANG, Y., AND TEODORESCU,R. MANGARD, S. DRAMA: Exploiting DRAM Addressing for One Bit Flips, One Cloud Flops: Cross-VM Row Hammer At- Cross-CPU Attacks. SEC’16. tacks and Privilege Escalation. SEC’16.
[50]P ETER,S.,LI,J.,ZHANG,I.,PORTS,D.R.K.,WOOS,D., [61]Y AROM, Y., AND FALKNER, K. FLUSH+RELOAD: A High KRISHNAMURTHY,A.,ANDERSON, T., AND ROSCOE, T. Ar- Resolution, Low Noise, L3 Cache Side-channel Attack. SEC’14. rakis: The Operating System is the Control Plane. OSDI’14. [62]Z HU, Y., ERAN,H.,FIRESTONE,D.,GUO,C.,LIPSHTEYN, [51]R AZAVI,K.,GRAS,B.,BOSMAN,E.,PRENEEL,B.,GIUF- M.,LIRON, Y., PADHYE,J.,RAINDEL,S.,YAHIA,M.H., FRIDA,C., AND BOS, H. Flip Feng Shui: Hammering a Needle AND ZHANG, M. Congestion Control for Large-Scale RDMA in the Software Stack. SEC’16. Deployments. SIGCOMM’15. [52]S EABORN,M., AND DULLIEN, T. Exploiting the DRAM Rowhammer Bug to Gain Kernel Privileges. BHUS’15.
USENIX Association 2018 USENIX Annual Technical Conference 225