Nested Kernel: An Operating System Architecture for Intra-Kernel Privilege Separation Nathan Dautenhahn1, Theodoros Kasampalis1, Will Dietz1, John Criswell2, and Vikram Adve1 1University of Illinois at Urbana-Champaign, 2University of Rochester fdautenh1, kasampa2, wdietz2,
[email protected],
[email protected] Abstract 1. Introduction Monolithic operating system designs undermine the security Critical information protection design principles, e.g., fail- of computing systems by allowing single exploits anywhere safe defaults, complete mediation, least privilege, and least in the kernel to enjoy full supervisor privilege. The nested common mechanism [34, 40, 41], have been well known kernel operating system architecture addresses this problem for several decades. Unfortunately, monolithic commodity by “nesting” a small isolated kernel within a traditional operating systems (OSes), like Windows, Mac OS X, Linux, monolithic kernel. The “nested kernel” interposes on all and FreeBSD, lack sufficient protection mechanisms with updates to virtual memory translations to assert protections which to adhere to these design principles. As a result, on physical memory, thus significantly reducing the trusted these OS kernels define and store access control policies in computing base for memory access control enforcement. We main memory which any code executing within the kernel incorporated the nested kernel architecture into FreeBSD can modify. The impact of this default, shared-everything on x86-64 hardware while allowing the entire operating environment is that the entirety of the kernel, including system, including untrusted components, to operate at the potentially buggy device drivers [12], forms a single large highest hardware privilege level by write-protecting MMU trusted computing base (TCB) for all applications on the translations and de-privileging the untrusted part of the system.