DOCSLIB.ORG

Mqsim: a Framework for Enabling Realistic Studies of Modern Multi-Queue SSD Devices

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Mqsim: a Framework for Enabling Realistic Studies of Modern Multi-Queue SSD Devices MQSim: A Framework for Enabling Realistic Studies of Modern Multi-Queue SSD Devices Arash Tavakkol, Juan Gómez-Luna, and Mohammad Sadrosadati, ETH Zürich; Saugata Ghose, Carnegie Mellon University; Onur Mutlu, ETH Zürich and Carnegie Mellon University https://www.usenix.org/conference/fast18/presentation/tavakkol This paper is included in the Proceedings of the 16th USENIX Conference on File and Storage Technologies. February 12–15, 2018 • Oakland, CA, USA ISBN 978-1-931971-42-3 Open access to the Proceedings of the 16th USENIX Conference on File and Storage Technologies is sponsored by USENIX. MQSim: A Framework for Enabling Realistic Studies of Modern Multi-Queue SSD Devices Arash Tavakkol†, Juan Gomez-Luna´ †, Mohammad Sadrosadati†, Saugata Ghose‡, Onur Mutlu†‡ †ETH Zurich¨ ‡Carnegie Mellon University Abstract sponse time, and decreasing cost, SSDs have replaced traditional magnetic hard disk drives (HDDs) in many Solid-state drives (SSDs) are used in a wide array of datacenters and enterprise servers, as well as in consumer computer systems today, including in datacenters and en- devices. As the I/O demand of both enterprise and con- terprise servers. As the I/O demands of these systems sumer applications continues to grow, SSD architectures continue to increase, manufacturers are evolving SSD ar- are rapidly evolving to deliver improved performance. chitectures to keep up with this demand. For example, manufacturers have introduced new high-bandwidth in- For example, a major innovation has been the intro- terfaces to replace the conventional SATA host–interface duction of new host interfaces to the SSD. In the past, protocol. These new interfaces, such as the NVMe proto- many SSDs made use of the Serial Advanced Technology col, are designed specifically to enable the high amounts Attachment (SATA) protocol [67], which was originally of concurrent I/O bandwidth that SSDs are capable of designed for HDDs. Over time, SATA has proven to be delivering. inefficient for SSDs, as it cannot enable the fast I/O ac- While modern SSDs with sophisticated features such cesses and millions of I/O operations per second (IOPS) as the NVMe protocol are already on the market, exist- that contemporary SSDs are capable of delivering. New ing SSD simulation tools have fallen behind, as they do protocols such as NVMe [63] overcome these barriers not capture these new features. We find that state-of-the- as they are designed specifically for the high through- art SSD simulators have three shortcomings that prevent put available in SSDs. NVMe enables high throughput them from accurately modeling the performance of real and low latency for I/O requests through its use of the off-the-shelf SSDs. First, these simulators do not model multi-queue SSD (MQ-SSD) concept. While SATA ex- critical features of new protocols (e.g., NVMe), such as poses only a single request port to the OS, MQ-SSD pro- their use of multiple application-level queues for requests tocols provide multiple request queues to directly expose and the elimination of OS intervention for I/O request applications to the SSD device controller. This allows processing. Second, these simulators often do not accu- (1) an application to bypass OS intervention for I/O re- rately capture the impact of advanced SSD maintenance quest processing, and (2) the SSD controller to schedule algorithms (e.g., garbage collection), as they do not prop- I/O requests based on how busy the SSD’s resources are. erly or quickly emulate steady-state conditions that can As a result, the SSD can make higher-performance I/O significantly change the behavior of these algorithms in request scheduling decisions. real SSDs. Third, these simulators do not capture the As SSDs and their associated protocols evolve to keep full end-to-end latency of I/O requests, which can incor- pace with changing system demands, the research com- rectly skew the results reported for SSDs that make use munity needs simulation tools that reliably model these of emerging non-volatile memory technologies. By not new features. Unfortunately, state-of-the-art SSD simu- accurately modeling these three features, existing sim- lators do not model a number of key properties of mod- ulators report results that deviate significantly from real ern SSDs that are already on the market. We evaluate SSD performance. several real modern SSDs, and find that state-of-the-art In this work, we introduce a new simulator, called simulators do not capture three features that are critical MQSim, that accurately models the performance of to accurately model modern SSD behavior. both modern SSDs and conventional SATA-based SSDs. First, these simulators do not correctly model the MQSim faithfully models new high-bandwidth protocol multi-queue approach used in modern SSD protocols. In- implementations, steady-state SSD conditions, and the stead, they implement only the single-queue approach full end-to-end latency of requests in modern SSDs. We used in HDD-based protocols such as SATA. As a result, validate MQSim, showing that it reports performance re- existing simulators do not capture (1) the high amount of sults that are only 6%-18% apart from the measured ac- request-level parallelism and (2) the lack of OS interven- tual performance of four real state-of-the-art SSDs. We tion in modern SSDs. show that by modeling critical features of modern SSDs, Second, many simulators do not adequately model MQSim uncovers several real and important issues that steady-state behavior within a reasonable amount of sim- were not captured by existing simulators, such as the per- ulation time. A number of fundamental SSD main- formance impact of inter-flow interference. We have re- tenance algorithms, such as garbage collection [11– leased MQSim as an open-source tool, and we hope that 13, 23], are not executed when an SSD is new (i.e., no it can enable researchers to explore directions in new and data has been written to the drive). As a result, manufac- different areas. turers design these maintenance algorithms to work best when an SSD reaches the steady-state operating point 1 Introduction (i.e., after all of the pages within the SSD have been Solid-state drives (SSDs) are widely used in today’s written to at least once) [71]. However, simulators that computer systems. Due to their high throughput, low re- cannot capture steady-state behavior (within a reasonable USENIX Association 16th USENIX Conference on File and Storage Technologies 49 simulation time) perform these maintenance algorithms down each flow unequally) in modern SSDs. This is on a new SSD. As such, many existing simulators do a major concern, as fairness is a first-class design goal not adequately capture algorithm behavior under realistic in modern computing platforms [4, 17, 19, 31, 37, 56– conditions, and often report unrealistic SSD performance 60, 66, 73–76, 80, 84, 88]. Unfairness reduces the pre- results (as we discuss in Section 3.2). dictability of the I/O latency and throughput for each Third, these simulators do not capture the full end-to- flow, and can allow a malicious flow to deny or delay end latency of performing I/O requests. Existing sim- I/O service to other, benign flows. ulators capture only the part of the request latency that We have made MQSim available as an open source takes place during intra-SSD operations. However, many tool to the research community [1]. We hope that emerging high-speed non-volatile memories greatly re- MQSim enables researchers to explore directions in sev- duce the latency of intra-SSD operations, and, thus, the eral new and different areas. uncaptured parts of the latency now make up a signif- We make the following key contributions in this work: icant portion of the overall request latency. For exam- • We use real off-the-shelf SSDs to show that state- ple, in Intel Optane SSDs, which make use of 3D XPoint of-the-art SSD simulators do not adequately capture memory [9, 25], the overhead of processing a request and three important properties of modern SSDs: (1) the transferring data over the system I/O bus (e.g., PCIe) is multi-queue model used by modern host–interface much higher than the memory access latency [16]. By protocols such as NVMe, (2) steady-state SSD behav- not capturing the full end-to-end latency, existing simu- ior, and (3) the end-to-end I/O request latency. lators do not report the true performance of SSDs with • We introduce MQSim, a simulator that accurately new and emerging memory technologies. models both modern NVMe-based and conventional Based on our evaluation of real modern SSDs, we find SATA-based SSDs. To our knowledge, MQSim is that these three features are essential for a simulator to the first publicly-available SSD simulator to faithfully capture. Because existing simulators do not model these model the NVMe protocol. We validate the results re- features adequately, their results deviate significantly ported by MQSim against several real state-of-the-art from the performance of real SSDs. Our goal in this multi-queue SSDs. work is to develop a new SSD simulator that can faith- • We demonstrate how MQSim can uncover important fully model the features and performance of both modern issues in modern SSDs that existing simulators cannot multi-queue SSDs and conventional SATA-based SSDs. capture, such as the impact of inter-flow interference To this end, we introduce MQSim, a new simulator that on fairness and system performance. provides an accurate and flexible framework for evaluat- ing SSDs. MQSim addresses the three shortcomings we 2 Background found in existing simulators, by (1) providing detailed In this section, we provide a brief background on multi- models of both conventional (e.g., SATA) and modern queue SSD (MQ-SSD) devices.
Load more

Recommended publications
  • Bull SAS: Novascale B260 (Intel Xeon Processor 5110,1.60Ghz)
    SPEC CINT2006 Result spec Copyright 2006-2014 Standard Performance Evaluation Corporation Bull SAS SPECint2006 = 10.2 NovaScale B260 (Intel Xeon processor 5110,1.60GHz) SPECint_base2006 = 9.84 CPU2006 license: 20 Test date: Dec-2006 Test sponsor: Bull SAS Hardware Availability: Dec-2006 Tested by: Bull SAS Software Availability: Dec-2006 0 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.0 11.0 12.0 13.0 14.0 15.0 16.0 17.0 18.0 12.7 400.perlbench 11.6 8.64 401.bzip2 8.41 6.59 403.gcc 6.38 11.9 429.mcf 12.7 12.0 445.gobmk 10.6 6.90 456.hmmer 6.72 10.8 458.sjeng 9.90 11.0 462.libquantum 10.8 17.1 464.h264ref 16.8 9.22 471.omnetpp 8.38 7.84 473.astar 7.83 12.5 483.xalancbmk 12.4 SPECint_base2006 = 9.84 SPECint2006 = 10.2 Hardware Software CPU Name: Intel Xeon 5110 Operating System: Windows Server 2003 Enterprise Edition (32 bits) CPU Characteristics: 1.60 GHz, 4MB L2, 1066MHz bus Service Pack1 CPU MHz: 1600 Compiler: Intel C++ Compiler for IA32 version 9.1 Package ID W_CC_C_9.1.033 Build no 20061103Z FPU: Integrated Microsoft Visual Studio .NET 2003 (lib & linker) CPU(s) enabled: 1 core, 1 chip, 2 cores/chip MicroQuill SmartHeap Library 8.0 (shlW32M.lib) CPU(s) orderable: 1 to 2 chips Auto Parallel: No Primary Cache: 32 KB I + 32 KB D on chip per core File System: NTFS Secondary Cache: 4 MB I+D on chip per chip System State: Default L3 Cache: None Base Pointers: 32-bit Other Cache: None Peak Pointers: 32-bit Memory: 8 GB (2GB DIMMx4, FB-DIMM PC2-5300F ECC CL5) Other Software: None Disk Subsystem: 73 GB SAS, 10000RPM Other Hardware: None
    [Show full text]
  • Comparing Filesystem Performance: Red Hat Enterprise Linux 6 Vs
    COMPARING FILE SYSTEM I/O PERFORMANCE: RED HAT ENTERPRISE LINUX 6 VS. MICROSOFT WINDOWS SERVER 2012 When choosing an operating system platform for your servers, you should know what I/O performance to expect from the operating system and file systems you select. In the Principled Technologies labs, using the IOzone file system benchmark, we compared the I/O performance of two operating systems and file system pairs, Red Hat Enterprise Linux 6 with ext4 and XFS file systems, and Microsoft Windows Server 2012 with NTFS and ReFS file systems. Our testing compared out-of-the-box configurations for each operating system, as well as tuned configurations optimized for better performance, to demonstrate how a few simple adjustments can elevate I/O performance of a file system. We found that file systems available with Red Hat Enterprise Linux 6 delivered better I/O performance than those shipped with Windows Server 2012, in both out-of- the-box and optimized configurations. With I/O performance playing such a critical role in most business applications, selecting the right file system and operating system combination is critical to help you achieve your hardware’s maximum potential. APRIL 2013 A PRINCIPLED TECHNOLOGIES TEST REPORT Commissioned by Red Hat, Inc. About file system and platform configurations While you can use IOzone to gauge disk performance, we concentrated on the file system performance of two operating systems (OSs): Red Hat Enterprise Linux 6, where we examined the ext4 and XFS file systems, and Microsoft Windows Server 2012 Datacenter Edition, where we examined NTFS and ReFS file systems.
    [Show full text]
  • Hypervisors Vs. Lightweight Virtualization: a Performance Comparison
    2015 IEEE International Conference on Cloud Engineering Hypervisors vs. Lightweight Virtualization: a Performance Comparison Roberto Morabito, Jimmy Kjällman, and Miika Komu Ericsson Research, NomadicLab Jorvas, Finland [email protected], [email protected], [email protected] Abstract — Virtualization of operating systems provides a container and alternative solutions. The idea is to quantify the common way to run different services in the cloud. Recently, the level of overhead introduced by these platforms and the lightweight virtualization technologies claim to offer superior existing gap compared to a non-virtualized environment. performance. In this paper, we present a detailed performance The remainder of this paper is structured as follows: in comparison of traditional hypervisor based virtualization and Section II, literature review and a brief description of all the new lightweight solutions. In our measurements, we use several technologies and platforms evaluated is provided. The benchmarks tools in order to understand the strengths, methodology used to realize our performance comparison is weaknesses, and anomalies introduced by these different platforms in terms of processing, storage, memory and network. introduced in Section III. The benchmark results are presented Our results show that containers achieve generally better in Section IV. Finally, some concluding remarks and future performance when compared with traditional virtual machines work are provided in Section V. and other recent solutions. Albeit containers offer clearly more dense deployment of virtual machines, the performance II. BACKGROUND AND RELATED WORK difference with other technologies is in many cases relatively small. In this section, we provide an overview of the different technologies included in the performance comparison.
    [Show full text]
  • Towards Better Performance Per Watt in Virtual Environments on Asymmetric Single-ISA Multi-Core Systems
    Towards Better Performance Per Watt in Virtual Environments on Asymmetric Single-ISA Multi-core Systems Viren Kumar Alexandra Fedorova Simon Fraser University Simon Fraser University 8888 University Dr 8888 University Dr Vancouver, Canada Vancouver, Canada [email protected] [email protected] ABSTRACT performance per watt than homogeneous multicore proces- Single-ISA heterogeneous multicore architectures promise to sors. As power consumption in data centers becomes a grow- deliver plenty of cores with varying complexity, speed and ing concern [3], deploying ASISA multicore systems is an performance in the near future. Virtualization enables mul- increasingly attractive opportunity. These systems perform tiple operating systems to run concurrently as distinct, in- at their best if application workloads are assigned to het- dependent guest domains, thereby reducing core idle time erogeneous cores in consideration of their runtime proper- and maximizing throughput. This paper seeks to identify a ties [4][13][12][18][24][21]. Therefore, understanding how to heuristic that can aid in intelligently scheduling these vir- schedule data-center workloads on ASISA systems is an im- tualized workloads to maximize performance while reducing portant problem. This paper takes the first step towards power consumption. understanding the properties of data center workloads that determine how they should be scheduled on ASISA multi- We propose that the controlling domain in a Virtual Ma- core processors. Since virtual machine technology is a de chine Monitor or hypervisor is relatively insensitive to changes facto standard for data centers, we study virtual machine in core frequency, and thus scheduling it on a slower core (VM) workloads. saves power while only slightly affecting guest domain per- formance.
    [Show full text]
  • I.MX 8Quadxplus Power and Performance
    NXP Semiconductors Document Number: AN12338 Application Note Rev. 4 , 04/2020 i.MX 8QuadXPlus Power and Performance 1. Introduction Contents This application note helps you to design power 1. Introduction ........................................................................ 1 management systems. It illustrates the current drain 2. Overview of i.MX 8QuadXPlus voltage supplies .............. 1 3. Power measurement of the i.MX 8QuadXPlus processor ... 2 measurements of the i.MX 8QuadXPlus Applications 3.1. VCC_SCU_1V8 power ........................................... 4 Processors taken on NXP Multisensory Evaluation Kit 3.2. VCC_DDRIO power ............................................... 4 (MEK) Platform through several use cases. 3.3. VCC_CPU/VCC_GPU/VCC_MAIN power ........... 5 3.4. Temperature measurements .................................... 5 This document provides details on the performance and 3.5. Hardware and software used ................................... 6 3.6. Measuring points on the MEK platform .................. 6 power consumption of the i.MX 8QuadXPlus 4. Use cases and measurement results .................................... 6 processors under a variety of low- and high-power 4.1. Low-power mode power consumption (Key States modes. or ‘KS’)…… ......................................................................... 7 4.2. Complex use case power consumption (Arm Core, The data presented in this application note is based on GPU active) ......................................................................... 11 5. SOC
    [Show full text]
  • Software Performance Engineering Using Virtual Time Program Execution
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Spiral - Imperial College Digital Repository IMPERIAL COLLEGE LONDON DEPARTMENT OF COMPUTING Software Performance Engineering using Virtual Time Program Execution Nikolaos Baltas Submitted in part fullment of the requirements for the degree of Doctor of Philosophy in Computing of Imperial College London and the Diploma of Imperial College London To my mother Dimitra for her endless support The copyright of this thesis rests with the author and is made available under a Creative Commons Attribution Non-Commercial No Derivatives licence. Re- searchers are free to copy, distribute or transmit the thesis on the condition that they attribute it, that they do not use it for commercial purposes and that they do not alter, transform or build upon it. For any reuse or redistribution, researchers must make clear to others the licence terms of this work. 4 Abstract In this thesis we introduce a novel approach to software performance engineering that is based on the execution of code in virtual time. Virtual time execution models the timing-behaviour of unmodified applications by scaling observed method times or replacing them with results acquired from performance model simulation. This facilitates the investigation of \what-if" performance predictions of applications comprising an arbitrary combination of real code and performance models. The ability to analyse code and models in a single framework enables performance testing throughout the software lifecycle, without the need to to extract perfor- mance models from code. This is accomplished by forcing thread scheduling decisions to take into account the hypothetical time-scaling or model-based performance specifications of each method.
    [Show full text]
  • Connectcore® 8X Performance and Power Benchmarking Report
    ConnectCore® 8X Performance and Power Benchmarking Report Application Note Revision history—90002448 Revision Date Description A March 2021 Initial release. Trademarks and copyright Digi, Digi International, and the Digi logo are trademarks or registered trademarks in the United States and other countries worldwide. All other trademarks mentioned in this document are the property of their respective owners. © 2021 Digi International Inc. All rights reserved. Disclaimers Information in this document is subject to change without notice and does not represent a commitment on the part of Digi International. Digi provides this document “as is,” without warranty of any kind, expressed or implied, including, but not limited to, the implied warranties of fitness or merchantability for a particular purpose. Digi may make improvements and/or changes in this manual or in the product(s) and/or the program(s) described in this manual at any time. Feedback To provide feedback on this document, email your comments to [email protected] Include the document title and part number (ConnectCore® 8X Performance and Power, 90002448 A) in the subject line of your email. ConnectCore® 8X Performance and Power 2 Contents Introduction Power architecture Measurement conditions Hardware used 7 Software used 7 Digi Embedded Yocto 7 MCA firmware 7 Benchmark packages 8 Host requirements 8 General conditions 8 Location and environment 8 Instrumentation 9 SOM power measurements 9 How to calculate SOM power 9 Resistor swap 9 Console cable 9 Measure points 10 Formula 10
    [Show full text]
  • PIC Licensing Information User Manual
    Oracle® Communications Performance Intelligence Center Licensing Information User Manual Release 10.1 E56971 Revision 3 April 2015 Oracle Communications Performance Intelligence Center Licensing Information User Manual, Release 10.1 Copyright © 2003, 2015 Oracle and/or its affiliates. All rights reserved. This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following notices are applicable: U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs.
    [Show full text]
  • Fair Benchmarking for Cloud Computing Systems
    Fair Benchmarking for Cloud Computing Systems Authors: Lee Gillam Bin Li John O’Loughlin Anuz Pranap Singh Tomar March 2012 Contents 1 Introduction .............................................................................................................................................................. 3 2 Background .............................................................................................................................................................. 4 3 Previous work ........................................................................................................................................................... 5 3.1 Literature ........................................................................................................................................................... 5 3.2 Related Resources ............................................................................................................................................. 6 4 Preparation ............................................................................................................................................................... 9 4.1 Cloud Providers................................................................................................................................................. 9 4.2 Cloud APIs ...................................................................................................................................................... 10 4.3 Benchmark selection ......................................................................................................................................
    [Show full text]
  • Filesystem Benchmark Tool
    Masaryk University Faculty of Informatics Filesystem Benchmark Tool Bachelor’s Thesis Tomáš Zvoník Brno, Spring 2019 Masaryk University Faculty of Informatics Filesystem Benchmark Tool Bachelor’s Thesis Tomáš Zvoník Brno, Spring 2019 This is where a copy of the official signed thesis assignment and a copy ofthe Statement of an Author is located in the printed version of the document. Declaration Hereby I declare that this paper is my original authorial work, which I have worked out on my own. All sources, references, and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source. Tomáš Zvoník Advisor: RNDr. Lukáš Hejtmánek, Ph.D. i Acknowledgements I would like to thank my thesis leader Lukáš Hejtmánek for guiding me through the process of creating this work. I would also like to thank my family and friends for being by my side, providing support, laughs and overall a nice environment to be in. Lastly I would like to thank my parents for supporting me throughout my whole life and giving me the opportunity to study at a university. ii Abstract In this thesis I have created a filesystem benchmark tool that com- bines best features of already existing tools. It can measure read/write speeds as well as speed of metadata operations. It can run on multi- ple threads and on multiple network connected nodes. I have then used my benchmark tool to compare performance of different storage devices and file systems. iii Keywords benchmark, file, system, iozone, fio, bonnie++ iv Contents Introduction 1 1 Existing benchmarks 3 1.1 IOzone ............................
    [Show full text]
  • Efficient Online Memory Error Assessment and Circumvention For
    Int. J. Critical Computer-Based Systems, Vol. 4, No. 3, 227{247 1 Efficient Online Memory Error Assessment and Circumvention for Linux with RAMpage Horst Schirmeier*, Ingo Korb, Olaf Spinczyk and Michael Engel Department of Computer Science 12, Technische Universit¨atDortmund, Otto-Hahn-Str. 16, 44221 Dortmund, Germany E-mail: [email protected] E-mail: [email protected] E-mail: [email protected] E-mail: [email protected] *Corresponding author Abstract: Memory errors are a major source of reliability problems in computer systems. Undetected errors may result in program termination or, even worse, silent data corruption. Recent studies have shown that the frequency of permanent memory errors is an order of magnitude higher than previously assumed and regularly affects everyday operation. To reduce the impact of memory errors, we designed RAMpage, a purely software-based infrastructure to assess and circumvent permanent memory errors in a running commodity x86-64 Linux-based system. We briefly describe the design and implementation of RAMpage and present new results from an extensive qualitative and quantitative evaluation. These results show the efficiency of our approach { RAMpage is able to provide a smooth graceful degradation in the presence of permanent memory errors while requiring only a small overhead in terms of CPU time, energy, and memory space. Keywords: Memory errors; Software-based fault tolerance; DRAM chips; Silent data corruption; Operating systems; Reliable operation Biographical notes: Horst Schirmeier received his Diploma in Computer Science from Friedrich-Alexander-Universit¨atErlangen, Germany. He worked as a researcher in the System Software group at FAU Erlangen, and is currently working in the field of resilient infrastructure software and fault resilience assessment for the DanceOS project in the Embedded System Software group at Technische Universit¨atDortmund, Germany.
    [Show full text]
  • Binary Compatibility for SGX Enclaves
    Binary Compatibility For SGX Enclaves Shweta Shindey Jinhua Cui Satyaki Sen Pinghai Yuan Prateek Saxena UC Berkeley National University of Singapore & National University of Singapore National University of Defense Technology Abstract—Enclaves, such as those enabled by Intel SGX, mance to specific versions of library interfaces provided by offer a powerful hardware isolation primitive for application library OSes [6], [17], [58], [61], [64]. partitioning. To become universally usable on future commodity While these approaches are promising and steadily matur- OSes, enclave designs should offer compatibility with existing software. In this paper, we draw attention to 5 design decisions ing, none of them offer binary compatibility with existing soft- in SGX that create incompatibility with existing software. These ware. In existing approaches, applications are expected to be represent concrete starting points, we hope, for improvements relinked against specific versions of libraries (e.g., musl, libc, in future TEEs. Further, while many prior works have offered glibc), ported to a customized OS, or containerized. Such partial forms of compatibility, we present the first attempt to offer modifications require significant changes to the complex build binary compatibility with existing software on SGX. We present RATEL, a system that enables a dynamic binary translation engine systems in place, often demanding developer involvement and inside SGX enclaves on Linux. Through the lens of RATEL, even access to source code. More importantly, most prior we expose the fundamental trade-offs between performance and works have enabled sufficient SGX compatibility to handle complete mediation on the OS-enclave interface, which are rooted specific applications [38], [52], [53], standard libraries, or in the aforementioned 5 SGX design restrictions.
    [Show full text]