Systems and Internet Infrastructure Security
Network and Security Research Center Department of Computer Science and Engineering Pennsylvania State University, University Park PA
CMPSC 497 Hardware Security
Trent Jaeger Systems and Internet Infrastructure Security (SIIS) Lab Computer Science and Engineering Department Pennsylvania State University
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 1 Security Problems
• We have discussed lots of security problems ‣ Attacks on memory errors ‣ Return-oriented attacks ‣ Compromised software ‣ Compromised operating systems, etc. • Is there any way new hardware features could prevent some attack vectors?
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 2 Hardware Features
‣ Enforce control-flow integrity via hardware • Intel Memory Protection Extensions (MPX)
‣ Check and enforce memory bounds • ARM TrustZone
‣ Prevent code injection in compromised operating systems • Intel SGX
‣ Isolate sensitive applications from compromised operating systems
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 3 Control-Flow Integrity
• What do you need to do to enforce control flow integrity?
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 4 Control-Flow Integrity
• What do you need to do to enforce control flow integrity? • Forward edges (indirect calls and jumps) ‣ For each indirect control transfer (source), ensure that the chosen target complies with the program’ CFG for that source (Fine-grained CFI) • Backward edges (returns) ‣ For each return, ensure that the target is associated with the originating call site (Shadow Stack)
• May be exceptions, but handle exceptionally
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 5 Intel Processor Trace
• A new hardware feature that enables efficient recording of control-flow and timing information about software execution (3-5% overhead) ‣ Initially available on the Broadwell processor ‣ Fully implemented on the Skylake processor • At each control choice, record a packet in memory ‣ Conditional branches ‣ Indirect call ‣ Returns • Enough to reconstruct the actual control flow
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 6 Intel PT Example
Trace Packets Basic Blocks
A F PGE A jmp D syscall TNT B Taken jcc E Not Taken
End C TIP F call *rax D PGD 0 jcc B
E
ret
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 8 When to Check?
• Since we are using Intel PT to log the program’s execution, we are naturally running behind ‣ Is this sufficient to enforce CFI? ‣ A forward or backward edge may already have been exploited
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 9 When to Check?
• Since we are using Intel PT to log the program’s execution, we are naturally running behind ‣ Is this sufficient to enforce CFI? ‣ A forward or backward edge may already have been exploited • While an exploit may be underway, the exploit cannot really have an impact until a system call occurs ‣ Modify unauthorized data persistently (except for memory-mapped files) ‣ Leak sensitive data to others
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 10 System Overview
User Space Kernel Space
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 11 System Overview
User Space Kernel Space
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 12 System Overview
User Space Kernel Space
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 13 System Overview
User Space Kernel Space
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 14 System Overview
User Space Kernel Space
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 15 System Overview
User Space Kernel Space
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 16 System Overview
User Space Kernel Space
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 17 System Overview
User Space Kernel Space
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 18 What To Do?
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 19 CFI Policies
• Coarse-grained Policy ‣ Check if the targets of indirect control transfers are valid ‣ Requires decoding the trace packets to find each target • Fine-grained Policy ‣ Check if the source and destination are a legitimate pair ‣ Requires control-flow recovery to identify source • Stateful Policy ‣ Check if an indirect control transfer is legitimate based on the program state (e.g., shadow stack) ‣ Requires sequential processing if state spans trace buffers
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 21 Using Intel PT for CFI
• What do you need to do to leverage an Intel PT trace to enforce fine-grained CFI?
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 22 Using Intel PT for CFI
• What do you need to do to leverage an Intel PT trace to enforce fine-grained CFI? ‣ Need to collect the source and target of each indirect call
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 23 Using Intel PT for CFI
• What do you need to do to leverage an Intel PT trace to enforce fine-grained CFI? ‣ Need to collect the source and target of each indirect call • How do you find these from Intel PT trace? ‣ Target is recorded in a packet ‣ But how do we find the source?
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 24 Using Intel PT for CFI
• What do you need to do to leverage an Intel PT trace to enforce fine-grained CFI? ‣ Need to collect the source and target of each indirect call • How do you find these from Intel PT trace? ‣ Target is recorded in a packet ‣ But how do we find the source? • Reconstruct the control flow from the trace identifies the sources ‣ Then can perform authorization
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 25 Fine-Grained CFI
• Recover the control flow from the trace buffer and the program binaries to identify sources ‣ Disassemble the binary online in basic blocks ‣ Traverse basic blocks using the trace buffer to find sources of indirect control transfers
• Authorize each indirect control transfer target against that program’s fine-grained policy for source ‣ For each indirect control transfer found in the trace ensure that the destination is in the legal target set of the corresponding source
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 26 CFI-Focused Logging
• Could you further optimize the hardware logging for CFI enforcement?
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 32 CFI-Focused Logging
• Could you further optimize the hardware logging for CFI enforcement? ‣ Can we eliminate need for control-flow recovery to enforce fine-grained CFI policies?
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 33 CFI-Focused Logging
• Could you further optimize the hardware logging for CFI enforcement? ‣ Can we eliminate need for control-flow recovery to enforce fine-grained CFI policies? • Intel PT could record the source in a packet as well as the target packet ‣ And ignore recording other information not necessary for fine-grained CFI – taken/not-taken
• This combination of changes reduces overhead for checking by over 90% on average
• But, not clear what impact on hardware overhead
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 34 Intel CET
• Intel Control-Flow Enforcement Technology (CET) aims to enforce shadow stack defenses in hardware ‣ Announced in June 2017 • Shadow Stack on backward edge ‣ Exception on failure – for handler to deal with • Indirect Branch Tracking on forward edge ‣ Restrict indirect calls/jumps to valid targets
• Issue: precision of these restrictions ‣ Weak – Single class of valid targets for all calls (coarse)
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 35 Prevent Memory Errors
• How do we prevent memory errors in our programs?
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 36 Prevent Memory Errors
• How do we prevent memory errors in our programs? ‣ Bounds checking • Can hardware help us in bounds checking? ‣ Intel Memory Protection Extensions (MPX)
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 37 Intel MPX
• Bounds checking instructions added to the ISA • If your program is compiled to use such instructions, it can enforce buffer bounds
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 38 Intel MPX
• Bounds checking instructions added to the ISA • If your program is compiled to use such instructions, it can enforce buffer bounds • Suppose you have the code
Obj* a[10];
for (i=0; i
}
• How would you check bounds?
Systems and Internet Infrastructure Security (SIIS) Laboratory Page 39 Intel MPX
• Bounds checking instructions added to the ISA • If your program is compiled to use such instructions, it can enforce buffer bounds Obj* a[10]; a_b = bndmk a, a+79; // set bounds – pointers are 8 bytes for (i=0; i • Bounds checking instructions added to the ISA • If your program is compiled to use such instructions, it can enforce buffer bounds • How would you check bounds using Intel MPX? ‣ Set the bounds for each pointer at allocation ‣ Load bounds when pointer is used ‣ Check start and end for each access wrt bounds Systems and Internet Infrastructure Security (SIIS) Laboratory Page 41 Intel MPX • Bounds checking instructions added to the ISA • If your program is compiled to use such instructions, it can enforce buffer bounds • How would you check bounds using Intel MPX? ‣ Set the bounds for each pointer at allocation ‣ Load bounds when pointer is used ‣ Check start and end for each access wrt bounds • Intel MPX is available, so support is happening for GCC and LLVM Systems and Internet Infrastructure Security (SIIS) Laboratory Page 42 Kernel Code Injection • Preventing code injection is a key defense ‣ We prevent code injection in user space using W xor X ‣ Which is implemented by the kernel • What if the kernel itself is compromised? ‣ Not only can we turn off W xor X ‣ But, code injection in kernel itself is trivial • Can we prevent kernel code injection – even should the kernel be compromised? Systems and Internet Infrastructure Security (SIIS) Laboratory Page 43 Lifetime Kernel Code Integrity 0xFFFFFFFF kernel space .bss R/W .data .text R/X .init 0xC0000000 0xBFFFFFFF R/W 0x00000000 user space Systems and Internet Infrastructure Security (SIIS) Laboratory Page 44 Attack on Permissions • Tamper with permissions 0xFFFFFFFF kernel space .bss R/W/X .data .text R/X/W .init 0xC0000000 0xBFFFFFFF R/W/X 0x00000000 user space Systems and Internet Infrastructure Security (SIIS) Laboratory Page 45 Attack on Mappings • Tamper with mappings virtual pages data page #m physical frames code page #n code page #n-1 Systems and Internet Infrastructure Security (SIIS) Laboratory Page 46 Goal Systems and Internet Infrastructure Security (SIIS) Laboratory Page 47 Background: TrustZone • Resources are partitioned into two distinct worlds ‣ Physical memory, interrupts, peripherals, etc. • Each world has its autonomy over its own resources • Secure world can access normal world resources, but not vice versa • Run in time-sliced fashion Systems and Internet Infrastructure Security (SIIS) Laboratory Page 48 SARMPROBE TrustZone Placement • Main limitation is that Trusted Computing technologies are • designedRecall the only specific to build attacksproofs of system boot ‣ Change to a different set of page tables that are under attacker’s control • instrument all instructions that can be potentially used to switch the page table root ‣ Modify page table entries in place • write-protect the whole page tables and instrument the first instruction in page fault handler Systems and Internet Infrastructure SecuritySecurity (SIIS)Laboratory Laboratory (SIIS) PagePage 49 13 SPROBES Invariants • S1: Execution of user space code from the kernel must never be allowed. • S2: W⊕X protection employed by the operating system must always be enabled. • S3: The page table base address must always correspond to a legitimate page table. • S4: Any modification to the page table entry must not make a kernel code page writable or make a kernel data page executable. • S5: MMU must be kept enabled to ensure all existing memory protections function properly. Systems and Internet Infrastructure Security (SIIS) Laboratory Page 50 SPROBE Mechanism • We need an instrumentation mechanism that enables the secure world to be notified upon events of its choice in the normal world normal world secure world push {r1-r3} sprobe_handler() stmia sp!,r10 { ... check_kernel(); movsmc #0 pc,lr restore_insn(); return_to_ns(); } Systems and Internet Infrastructure Security (SIIS) Laboratory Page 51 Evaluation • Setup ‣ Linux 2.6.38 in the normal world ‣ Fast Models 8.1 for emulation • 12 SPROBES are inserted into the Linux kernel ‣ 6 for enforcing W⊕X protection and MMU Enable (S2+S5) ‣ 4 for monitoring changes to page table root (S3) ‣ 1 for monitoring changes to page table configuration (S4) ‣ 1 for monitoring modifications to page table entries (S4) ‣ Reject page table entries with wrong user/kernel bits (S1) Systems and Internet Infrastructure Security (SIIS) Laboratory Page 53 A Little Bit More… • Samsung has implemented the same idea and deployed this technique on millions of devices [CCS 2014] Systems and Internet Infrastructure Security (SIIS) Laboratory Page 55 Performance overheads of Samsung’s implementation 9.00% 8.00% 7.65% 7.55% 7.00% 6.80% 6.00% 5.00% 4.00% 3.00% 3.24% 2.61% 2.00% 1.51% 1.43% 1.40% 1.00% 0.85% 0.39% 0.00% Systems andQuadrant Internet InfrastructureAntutu SecurityGL (SIIS) Frame LaboratoryLinpack RL HTML5 Metal CF-Bench Smartbench Geekbench Page 56 Eliminate Trust in OS • The OS may not be secure itself ‣ Hard to achieve reference monitor concept ‣ Especially, verification • What if you want to eliminate trust in the OS altogether? Systems and Internet Infrastructure Security (SIIS) Laboratory Page 57 Intel Software Guard Ext • What if we only want to run one high-integrity user- Intel® Software Guard Extensions (SGX) process? [McKeen et al, Hoekstra et al., Anati et al., HASP’13] Security critical code isolated in enclave Only CPU is trusted Transparent memory encryption APP1 APP2 Enclave Security 18 new instructions Service Enclaves cannot harm the system Only unprivileged code (CPU ring3) Memory protection Designed for Multi-Core systems Operating System Multi-threaded execution of enclaves Parallel execution of enclaves and SGX CPU Hardware untrusted code Enclaves are interruptible Programming Reference available Trusted Untrusted SYSTEM SECURITY A.-R. Sadeghi ©TU Darmstadt, 2007-2014 Slide Nr. 2, Lecture Embedded System Security, SS 2014 Trusted Execution Environments / Intel SGX LAB Systems and Internet Infrastructure Security (SIIS) Laboratory Page 58 Intel Software Guard Ext • What if we only want to run one high-integrity user- process? SGX Enclaves Enclaves are isolated memory regions of code and data One part of physical memory (RAM) is reserved for enclaves It is called Enclave Page Cache (EPC) EPC memory is encrypted in the main memory (RAM) Trusted hardware consists of the CPU-Die only EPC is managed by OS/VMM RAM: Random Access Memory OS: Operating System VMM: Virtual Machine Monitor (also known as Hypervisor) SYSTEM SECURITY A.-R. Sadeghi ©TU Darmstadt, 2007-2014 Slide Nr. 3, Lecture Embedded System Security, SS 2014 Trusted Execution Environments / Intel SGX LAB Systems and Internet Infrastructure Security (SIIS) Laboratory Page 59 Intel Software Guard Ext • What if we only want to run one high-integrity user- process?SGX Memory Access Control Access control in two direction From enclaves to “outside“ Isolating malicious enclaves Enclaves needs some means to communicate with the outside world, e.g., their “host applications” From “outside“ to enclaves Enclave memory must be protected from Applications Privileged software (OS/VMM) Other enclaves OS: Operating System VMM: Virtual Machine Monitor (also known as Hypervisor) SYSTEM SECURITY A.-R. Sadeghi ©TU Darmstadt, 2007-2014 Slide Nr. 14, Lecture Embedded System Security, SS 2014 Trusted Execution Environments / Intel SGX LAB Systems and Internet Infrastructure Security (SIIS) Laboratory Page 60 Intel Software Guard Ext • What if we only want to run one high-integrity user- SGXprocess? MAC from enclaves to “outside“ From enclaves to “outside“ All memory access has to conform to segmentation and paging policies by the OS/VMM Enclaves cannot manipulate those policies, only unprivileged instructions inside an enclave Code fetches from inside an enclave to a linear address outside that enclave will results in a #GP(0) exception MAC: Memory Access Control #GP(0): General Protection Fault SYSTEM SECURITY A.-R. Sadeghi ©TU Darmstadt, 2007-2014 Slide Nr. 15, Lecture Embedded System Security, SS 2014 Trusted Execution Environments / Intel SGX LAB Systems and Internet Infrastructure Security (SIIS) Laboratory Page 61 Intel Software Guard Ext • What if we only want to run one high-integrity user- process?SGX MAC “outside” to enclaves From “outside“ to enclaves Non-enclave accesses to EPC memory results in abort page semantics Direct jumps from outside to any linear address that maps to an enclave do not enable enclave mode and result in a about page semantics and undefined behavior Hardware detects and prevents enclave accesses using logical-to-linear address translations which are different than the original direct EA used to allocate the page. Detection of modified translation results in #GP(0) MAC: Memory Access Control EA: Enclave Access #GP(0): General Protection Fault SYSTEM SECURITY A.-R. Sadeghi ©TU Darmstadt, 2007-2014 Slide Nr. 16, Lecture Embedded System Security, SS 2014 Trusted Execution Environments / Intel SGX LAB Systems and Internet Infrastructure Security (SIIS) Laboratory Page 62 Intel Software Guard Ext • What if we SGXonly want – Create to run oneEnclave high-integrity user- process? 1 3 2 Loader Enclave5 User space Client SGX driver Operating system SK/PK SGX Hardware 1. Create App 2. Create app certificate (includes HASH(App) and Client PK) 3. Upload App to Loader Trusted UntrustedSYSTEM SECURITY A.-R. Sadeghi ©TU Darmstadt, 2007-2014 Slide Nr. 19, Lecture Embedded System Security, SS 2014 Trusted Execution Environments / Intel SGX LAB Systems and Internet Infrastructure Security (SIIS) Laboratory Page 63 Intel Software Guard Ext • What if weSGX only want– Create to run Eonenclave high-integrity user- process? 1 3 2 4 Loader Enclave5 User space 5 Client SGX driver Operating system SK/PK SGX 7 Hardware 1. Create App 2. Create app certificate (includes HASH(App) and Client PK) 3. Upload App to Loader 4. Create enclave 5. Allocate enclave pages Trusted UntrustedSYSTEM SECURITY A.-R. Sadeghi ©TU Darmstadt, 2007-2014 Slide Nr. 20, Lecture Embedded System Security, SS 2014 Trusted Execution Environments / Intel SGX LAB Systems and Internet Infrastructure Security (SIIS) Laboratory Page 64 Intel Software Guard Ext • What if we SGXonly want – Create to run oneEnclave high-integrity user- process? 6 1 3 2 4 Loader Enclave5 User space 5 Client SGX driver Operating system SK/PK SGX 7 Hardware 1. Create App 2. Create app certificate (includes HASH(App) and Client PK) 3. Upload App to Loader 4. Create enclave 5. Allocate enclave pages 6. Load & Measure App 7. Validate certificate and enclave integrity Trusted UntrustedSYSTEM SECURITY A.-R. Sadeghi ©TU Darmstadt, 2007-2014 Slide Nr. 21, Lecture Embedded System Security, SS 2014 Trusted Execution Environments / Intel SGX LAB Systems and Internet Infrastructure Security (SIIS) Laboratory Page 65 Intel Software Guard Ext • What if we onlySGX want – Create to run one Enclave high-integrity user- process? 6 1 3 2 4 Loader Enclave5 User space 5 Client SGX driver Operating system SK/PK SGX 7 8 K Hardware 1. Create App 2. Create app certificate (includes HASH(App) and Client PK) 3. Upload App to Loader 4. Create enclave 5. Allocate enclave pages 6. Load & Measure App 7. Validate certificate and enclave integrity 8. Generate enclave K key 9. Protect enclave Trusted UntrustedSYSTEM SECURITY A.-R. Sadeghi ©TU Darmstadt, 2007-2014 Slide Nr. 22, Lecture Embedded System Security, SS 2014 Trusted Execution Environments / Intel SGX LAB Systems and Internet Infrastructure Security (SIIS) Laboratory Page 66 Take Away • Lots of efforts in exploring hardware features to improve security ‣ CFI enforcement via Intel PT • Hardware may need to be optimized further ‣ Software bounds checking MPX ‣ Isolate code from untrusted kernel – SGX and TZ • However, there are also security issues with hardware ‣ Meltdown and Spectre ‣ Hardware Trojans Systems and Internet Infrastructure Security (SIIS) Laboratory Page 73