SGX-LKL: a Linux-Based Runtime System for SGX Enclaves
Large-Scale Data & Systems Group
SGX-LKL: A Linux-Based Runtime System for SGX Enclaves
Peter Pietzuch
Imperial College London http://lsds.doc.ic.ac.uk
Joint work with Microsoft Research
SGX Community Workshop – July 2020 Design Space for Enclave Execution
(a) SDK (b) Syscalls (c) LibOS (d) Full OS kernel
Application code Unmodifed Application Application (trusted) application Shim layer System libraries System libraries System libraries Internal Internal Narrow Shim layer syscall syscall call handling handling interface Wide system call Library OS Full OS kernel interface Narrow/ Application code Narrow (untrusted) wide hypervisor Loader/starter host interface interface Loader/starter Loader/starter
Host OS Host OS Host OS Host OS
We want a complete Library OS to get a hypervisor host interface
Peter Pietzuch - Imperial College London 2 Why is making a Library OS complete hard?
Evolution of the Linux system call interface 500 450 400 350 300 250 200 150
Number of Linux system calls Number of Linux system 100 50 0 1 2 2.2 2.4 3.1 3.5 3.8 4.3 4.5 4.8 5.1 5.3 2.6.02.6.3 3.14 3.17 3.19 4.11 4.15 2.0.28 2.4.8.12.4.132.5.36 2.6.102.6.132.6.172.6.192.6.232.6.272.6.302.6.332.6.362.6.39
Peter Pietzuch - Imperial College London Linux kernel version 3 SGX-LKL Design
SGX enclave
Mounted Root Application binary root file file Application files system system System libraries Paravirtualised host interface
Initialisation, Linux kernel threading, attestation SGX-LKL Virtual I/O device (virtio) layer
ocalls Virtualized Host disk/network devices SGX-LKL launcher Virtio
Network Root disk device image
Peter Pietzuch - Imperial College London 4 SGX-LKL Implementation
SGX enclave Unmodified application
Application binary Dynamic library Dynamic library User space
Standard C library (musl)
System call layer (implemented as function calls) Linux Kernel Library (LKL) Page Cache Work queues Network stack Filesystem stack Encryption and integrity protection Subsystems Crypto Wireguard Device mapper No MMU Standard Linux VirtIO drivers architecture LKL host interface Console Network Block device Timer Kernel space driver driver driver Setup Attestation
SGX-LKL Memory Threading Signals
Open Enclave Open Enclave SDK
SGX-LKL launcher
Synchronous ocalls Shared memory transport Paravirtualised host interface Page Signal Virtual Virtual console Virtual net Virtual block Idle protection handling timer device device device device
Peter Pietzuch - Imperial College London 5 Paravirtualised Host Interface
• Host layer is minimal hypervisor
• Narrow host interface: Enclave Shared memory (untrusted) Host – 4 hypercalls Block data Virtual block – 3 upcalls device ring buffer device – Virtio devices driver interrupts event channel
• Follows OASIS Virtual I/O device Network Virtual net driver specification device – virtio-block – virtio-net Console Virtual console – virtio-console driver device • Ring buffer per device • Event channels for notifications
Peter Pietzuch - Imperial College London 6 Disk protection using Linux Volume Encryption
• Applications and data provided via encrypted ext4 disk images – Encryption/integrity protection at block level – Mounted by LKL inside enclave
• Uses standard Linux device mapper API: Root disk – dm-crypt for encryption of file systems image – dm-integrity for integrity of read-only file systems – dm-verity for integrity protection for read/write file systems – Merkle tree for disk block verification – Leaf nodes contain hashes of disk blocks
• Support for any Linux file systems – e.g. overlayfs for read-only volumes
Peter Pietzuch 7 Network protection using Linux in-kernel VPN
• In-enclave trusted Linux network stack – Protects arbitrary application traffic Network device • TUN/TAP interface to send/receive packets via host – Public unencrypted network device – Wireguard VPN network device
• Wireguard to create VPN between enclaves – Layer 3 VPN protocol – Protects enclave <-> enclave communication
Peter Pietzuch 8 Comparison of TCB Sizes
TEE runtime system Component LoC (in 1000s) Total LoC (in 1000s) Panoply 20 20 GNU C library 1,292 Graphene-SGX Library OS 34 1,348 Shield module 22 Drawbridge library OS 5,505 Haven 5,528 Shield module 23 LKL 598 SGX-LKL Musl C library 88 736 Core 50
Peter Pietzuch - Imperial College London 9 SGX-LKL Roadmap
• Support for layered attestation
• Support for GNU standard C library
• Porting of other OS kernels to enclaves
• Hypervisor-based host launcher implementation
• More efficient kernel-bypass I/O (DPDK, SPDK)
Peter Pietzuch - Imperial College London 10 SGX-LKL is available on GitHub
Peter Pietzuch - Imperial College London 11 Summary: Linux in Enclaves with SGX-LKL
• Supporting a narrow host interface is key!
• SGX-LKL: Running OS kernels inside SGX enclaves – Benefits from a mature and popular POSIX implementation (Linux) – We encourage people to experiment with other OS kernels
• Trade-off between TCB size and POSIX functionality
Thank You — Any Questions? Peter Pietzuch https://lsds.doc.ic.ac.uk — [email protected]
Peter Pietzuch - Imperial College London 12