A Comparison Study of Intel SGX and AMD Memory Encryption Technology

Saeid Mofrad, Fengwei Zhang Weidong Shi (Larry) Shiyong Lu University of Houston Wayne State University [email protected] {saeid.mofrad, Fengwei, Shiyong}@wayne.edu

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 1 Outline ➢Introduction ➢Technology Background ➢Comparison and Results ➢Conclusions and Future Work

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 2 Outline ➢Introduction ➢Technology Background ➢Comparison and Results ➢Conclusions and Future Work

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 3 Trusted Execution Environment

➢Trusted Execution Environment can be achieved with isolation. APP APP APP APP ➢Isolation through Virtual Machine is a common approach to achieve security at runtime. OS OS OS OS ➢Downsides of software only virtualization: 1) Virtualization uses OS and Hypervisor and puts them in the TCB. VM VM VM VM 2) OS or Hypervisor contains thousands of lines of code and may have security flaws. 3) Many OS and Hypervisor exploits have been HYPERVISOR reported. 4) Increased TCB means less security. HARDWARE

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 4 Hardware-Assisted Trusted Execution Environment ➢Hardware-Assisted TEE couples hardware with TEE technology mitigates the downsides of the software only TEEs. ➢Hardware-Assisted TEE is faster since it uses dedicated hardware. ➢Hardware-Assisted TEE exposes small TCB and smaller TCB means better security. ➢Early Hardware-Assisted TEE: Intel ME, AMD PSP, and x86 SMM.

➢Two general purpose Hardware-Assisted TEE have been proposed recently in x86 architecture: 1. Intel Software Guard eXtensions (SGX). (HASP 2013) 2. AMD Memory Encryption Technology. (White Paper 2016)

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 5 Outline ➢Introduction ➢Technology Background ➢Comparison and Results ➢Conclusions and Future Work

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 6 Background: Intel Software Guard eXtensions (SGX)

➢SGX Application execution flow: 1) App is built with trusted and untrusted App part. Untrusted part of App Enclave (Trusted part of App) 2) Untrusted part creates and executes the 1- App creates an Call Gates enclave. 3- The trusted enclave that is placed in the encrypted and 2- App calls a trusted function processes the trusted memory referred to as EPC. function. security-sensitive data. 5- App continues its 4- The trusted 3) Trusted function is called and the normal execution. function returns. execution is transferred into the enclave where all data will be in clear text, then the security-sensitive data is processed. Privileged System Software, OS, Hypervisor, SMM, and BIOS 4) Trusted function returns. 5) App continues its normal execution.

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 7 Background: AMD Memory Encryption Technology

➢Addresses the physical access and the system software class of attacks in the public cloud. APP APP APP APP ➢Introduces SME, TSME, and SEV 1) Secure Memory Encryption (SME) and Transparent Secure Memory Encryption (TSME) protect against VM VM VM VM the physical access attacks. 2) Secure Encrypted Virtualization (SEV) protects against system software class of attacks. OS/Hypervisor 3) SME and TSME encrypt the system memory providing the confidentiality for illegally physical access attempts. 4) SEV encrypts VM’s memory image with an unique Memory Encryption Engine (AES-128) key to the VM providing confidentiality for VM’s memory image and protects against system RAM software class of attacks.

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 8 Outline ➢Introduction ➢Technology Background ➢Comparison and Results ➢Conclusions and Future Work

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 9 Testbeds Configuration

Testbed Machine Intel AMD CPU Model Core i7-6700 EPYC-7251 CPU Physical Core 4 8 Number CPU Logical Core 8 16 Number CPU Base Clock 3.4 GHz 2.1 GHz CPU Max Clock 4.0 GHz 2.9 GHz Cache Type Smart Cache L3 Cache Size 8MB 32MB Motherboard DELL OptiPlex 7040 GIGABYTE MZ31-AR0 Memory 8GB DDR4 No-ECC 32GB DDR4-ECC Storage 1TB 7200 RPM HDD 512GB SSD Operating System Linux 16.04 LTS Linux 16.04 LTS OS, Hypervisor kernel 4.15.7-041507-generic 4.15.0-rc1-kvm VM Kernel N/A 4.14.0-rc5-tip TEE SDK Version SGX SDK Ver 2.00 N/A

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 10 SGX VS SEV

TEE Highest Memory Size SDK Software Platform Protection Level Technology Access Limits Change Attestation Level Mechanism Intel SGX Ring 3 Up to 128MB Provided Required Attested Confidentiality EPC through Intel and Integrity of Remote the Code and Attestation Data in the Protocol and Enclave IAS At Runtime AMD SEV Ring 0 Up to Available Not Only Attested Confidentiality System Ram Required Hypervisor through AMD of the VM’s and VM’s Secure Memory Image Kernel Processor At Runtime

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 11 Function and Use Cases Comparison

Intel SGX AMD Memory Encryption Technology (SEV) Initial design targeted microservices and small Initial design targeted cloud and Infrastructure as a workload. (small amount of secure memory and Service. (Large amount of secure memory featured was featured mainly in mobile and desktop family in server family processors) processors) Requires major software changes and code Does not require software changes and code refactoring. (Not suitable for securing legacy refactoring. (Suitable for securing legacy applications) applications) SGX works with ring 3 and is not suitable for SEV works with ring 0 and is suitable for broader workloads with many system calls. range of workloads especially those with many system calls. SGX is suitable for small but security-sensitive SEV is suitable for securing legacy, large and workload. (SGX has small TCB) enterprise level application. (SEV has large TCB)

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 12 Security and Vulnerability Comparison

Intel SGX AMD Memory Encryption Technology (SME, SEV) Provides Memory Integrity Protection. Does Not Provide Memory Integrity Protection.

Vulnerable to Memory Side Channels. Vulnerable to Memory Side Channels. Vulnerable to Denial of Service Attacks. (OS Vulnerable to Denial of Service Attacks. (Hypervisor Handles System Calls) Handles VM Requests) Small TCB. (TCB is CPU package) Large TCB. (VM’s OS is located inside TCB) Vulnerable to Synchronization Attacks. AMD Secure Processor Firmware Bug Discovered. (TOCTTOU, Use-After-Free) (MASTERKEY and FALLOUT)

Intel SGX carefully separates the trusted and untrusted environments, provides a narrow and protected enclave gateway, enforces memory access control, and applies memory integrity protection, thus making it a suitable TEE for protecting workloads that interact with security-sensitive data.

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 13 Floating Point Intensive Workload Comparison ➢Measures the execution performance of the Platform

TEE. Driver program calculates the ➢Methodology: elapsed time 1) Codebase is identical for SGX and SEV. Start High-Resolution Timer Stop High-Resolution Timer 2) SGX uses different random number Trusted Execution generator (Provided by SGX SDK). Environment

Random 3) Datapoints are generated inside the TEE. datapoint is generated 4) Benchmark applies floating point intensive Floating point primitives and calculates the elapsed time. intensive primitive is executed

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 14 Floating Point Intensive Workload Comparison Results

Intel SGX Enclave Protected Workload Intel SGX VS AMD SEV Performance Intel Unprotected Workload Comparison AMD SEV Protected VM Workload 25.00 AMD Unprotected VM Workload 19.31

140.90 20.00

140.15

135.31

134.18 134.16 133.87

15.00

10.00

NANOSECONDS 5.00

Average Relative Slowdown Relative Average

27.44 27.34

14.64 1.00

11.46

11.42

10.04

9.40

9.33

9.29

9.23

8.70

8.67

8.63

8.52

6.11

6.09

5.58 5.58 0.00 SIN() COS() TAN() ARCTAN() EXP() SQRT() Floating Point Intensive Workload FLOATING POINT INTENSIVE OPERATIONS Intel SGX Slowdown AMD SEV Slowdown

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 15 Memory Encryption Engine Performance Comparison

➢Measures the MEE performance of the TEE. Platform

➢Codebase is identical for SGX and SEV. Driver program generates a large buffer with randomly Methodology: generated numbers 1) Large buffer is generated outside of the TEE. Start High-Resolution Timer Stop High-Resolution Timer

Trusted Execution 2) Large buffer is sent and copied inside the Environment TEE. • Trusted buffer is created inside the TEE 3) Elapsed time is calculated. • Untrusted buffer is marshalled and copied into the trusted buffer

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 16 Memory Encryption Engine Performance Comparison Results

MEE Performance with 2.8GB Input MEE Performance Comparison Buffer 12.00 12000 9.45 10.00 10018 10000 8.00 8000 6.00 5.53

6000 5575 Relative Slowdown Relative 4.00 Milliseconds 4000

2.00 2000 1009 1060

0 0.00 AMD SEV- AMD Intel SGX Intel MEE Intensive Workload Protected VM Unprotected Unproteted VM Intel SGX Slowdown AMD SEV Slowdown

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 17 Comprehensive Workload Comparison

➢Measures the performance of the TEE while a secure Platform

protocol for public cloud data provisioning and Driver program workload is followed. generates a large encrypted buffer with randomly generated ➢Codebase is identical for SGX and SEV. numbers ➢Task performed: Quicksort and MD5 message digest. ➢SEV simulates enclave model. Start High-Resolution Timer Stop High-Resolution Timer Trusted Execution Methodology: Environment • Trusted buffer is created inside the 1) Driver program generates and encrypts datapoints TEE, and untrusted with a key known to the enclave. buffer is marshalled and copied into the trusted buffer. 2) Encrypted data is sent and copied inside the TEE. • Trusted buffer is decrypted and 3) Inside the enclave the received buffer is decrypted, intended task is executed. task is executed, result is encrypted, and returns to • Result is encrypted the driver program. and returns to the driver program. 4) Elapsed time is calculated.

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 18 Comprehensive Workload Comparison Results

Comprehensive Workload Comparison Comprehensive Workload Comparison With Quicksort Task And Different Input 6.00 5.00 Size 5.00 250 3.99 200 4.00 3.19 150 3.00 100 Seconds 50 2.00 0 1.06 1.01 1.01 0 250 500 750 1000 1250 1500 1750 2000 1.00 Input Size in MB 0.00 Average Relative Slowdown Relative Average Quicksort with 1.9GB MD5 (OpenSource) MD5 (SGXSSL) For Intel SGX Enclave Protected Workload Input Data With 256 Bytes Input SGX (OpenSource) Intel Unrotected Workload Data For SEV With 256 AMD SEV Protected VM Bytes Input Data AMD Unprotected VM Intel SGX Slowdown AMD SEV Slowdown

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 19 Outline ➢Introduction ➢Technology Background ➢Comparison and Results ➢Conclusions and Future Work

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 20 Conclusions and Future Work

➢This paper is the first comparison study between AMD Memory Encryption Technology and Intel Software Guard eXtensions (SGX). ➢This paper illustrates comparison information regarding the functionality and use cases, security, and performance of Intel SGX and AMD Memory Encryption Technology. ➢We conclude that Intel SGX is suited for highly security-sensitive but small workloads since it enforces the memory integrity protection and has a limited amount of secure resources. ➢AMD SME and SEV do not provide memory integrity protection. However, providing a greater amount of secure resources to applications, performing faster than Intel SGX (when an application requires a large amount of secure memory), and no code refactoring, make them more suitable for complex or legacy applications and services. ➢Future work: SEV-ES and SGX2.

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 21 Thank You! Email: [email protected] Technical Report of this work will be available at: http://compass.cs.wayne.edu/compass/publications.html

WAYNE STATE UNIVERSITY & UNIVERSITY OF HOUSTON COMPASS LAB (HTTP://COMPASS.CS.WAYNE.EDU) 22