Large-Scale Data & Systems Group SGX-LKL: A Linux-Based Runtime System for SGX Enclaves Peter Pietzuch Imperial College London http://lsds.doc.ic.ac.uk <[email protected]> Joint work with Microsoft Research SGX Community Workshop – July 2020 Design Space for Enclave Execution (a) SDK (b) Syscalls (c) LibOS (d) Full OS kernel Application code Unmodifed Application Application (trusted) application Shim layer System libraries System libraries System libraries Internal Internal Narrow Shim layer syscall syscall call handling handling interface Wide system call Library OS Full OS kernel interface Narrow/ Application code Narrow (untrusted) wide hypervisor Loader/starter host interface interface Loader/starter Loader/starter Host OS Host OS Host OS Host OS We want a complete Library OS to get a hypervisor host interface Peter Pietzuch - Imperial College London 2 Why is making a Library OS complete hard? 500 450 400 350 300 250 Evolution of the Linux system call interface 200 150 Number of Linux system calls 100 50 Peter Pietzuch 0 1 - Imperial College London 2 2.0.28 2.2 2.4 2.4.8.1 2.4.13 2.5.36 2.6.0 2.6.3 2.6.10 2.6.13 2.6.17 2.6.19 2.6.23 2.6.27 Linux kernel version 2.6.30 2.6.33 2.6.36 2.6.39 3.1 3.5 3.8 3.14 3.17 3.19 4.3 4.5 4.8 4.11 4.15 5.1 5.3 3 SGX-LKL Design SGX enclave Mounted Root Application binary root file file Application files system system System libraries Paravirtualised host interface Initialisation, Linux kernel threading, attestation SGX-LKL Virtual I/O device (virtio) layer ocalls Virtualized Host disk/network devices SGX-LKL launcher Virtio Network Root disk device image Peter Pietzuch - Imperial College London 4 SGX-LKL Implementation SGX enclave Unmodified application Application binary Dynamic library Dynamic library User space Standard C library (musl) System call layer (implemented as function calls) Linux Kernel Library (LKL) Page Cache Work queues Network stack Filesystem stack Encryption and integrity protection Subsystems Crypto Wireguard Device mapper No MMU Standard Linux VirtIO drivers architecture LKL host interface Console Network Block device Timer Kernel space driver driver driver Setup Attestation SGX-LKL Memory Threading Signals Open Enclave Open Enclave SDK SGX-LKL launcher Synchronous ocalls Shared memory transport Paravirtualised host interface Page Signal Virtual Virtual console Virtual net Virtual block Idle protection handling timer device device device device Peter Pietzuch - Imperial College London 5 Paravirtualised Host Interface • Host layer is minimal hypervisor • Narrow host interface: Enclave Shared memory (untrusted) Host – 4 hypercalls Block data Virtual block – 3 upcalls device ring buffer device – Virtio devices driver interrupts event channel • Follows OASIS Virtual I/O device Network Virtual net driver specification device – virtio-block – virtio-net Console Virtual console – virtio-console driver device • Ring buffer per device • Event channels for notifications Peter Pietzuch - Imperial College London 6 Disk protection using Linux Volume Encryption • Applications and data provided via encrypted ext4 disk images – Encryption/integrity protection at block level – Mounted by LKL inside enclave • Uses standard Linux device mapper API: Root disk – dm-crypt for encryption of file systems image – dm-integrity for integrity of read-only file systems – dm-verity for integrity protection for read/write file systems – Merkle tree for disk block verification – Leaf nodes contain hashes of disk blocks • Support for any Linux file systems – e.g. overlayfs for read-only volumes Peter Pietzuch 7 Network protection using Linux in-kernel VPN • In-enclave trusted Linux network stack – Protects arbitrary application traffic Network device • TUN/TAP interface to send/receive packets via host – Public unencrypted network device – Wireguard VPN network device • Wireguard to create VPN between enclaves – Layer 3 VPN protocol – Protects enclave <-> enclave communication Peter Pietzuch 8 Comparison of TCB Sizes TEE runtime system Component LoC (in 1000s) Total LoC (in 1000s) Panoply 20 20 GNU C library 1,292 Graphene-SGX Library OS 34 1,348 Shield module 22 Drawbridge library OS 5,505 Haven 5,528 Shield module 23 LKL 598 SGX-LKL Musl C library 88 736 Core 50 Peter Pietzuch - Imperial College London 9 SGX-LKL Roadmap • Support for layered attestation • Support for GNU standard C library • Porting of other OS kernels to enclaves • Hypervisor-based host launcher implementation • More efficient kernel-bypass I/O (DPDK, SPDK) Peter Pietzuch - Imperial College London 10 SGX-LKL is available on GitHub Peter Pietzuch - Imperial College London 11 Summary: Linux in Enclaves with SGX-LKL • Supporting a narrow host interface is key! • SGX-LKL: Running OS kernels inside SGX enclaves – Benefits from a mature and popular POSIX implementation (Linux) – We encourage people to experiment with other OS kernels • Trade-off between TCB size and POSIX functionality Thank You — Any Questions? Peter Pietzuch https://lsds.doc.ic.ac.uk — [email protected] Peter Pietzuch - Imperial College London 12.