Lecture 6 From L3 to seL4: What Have We Learnt in 20 Years of L4 ?

Kevin Elphinstone and Gernot Heiser

Operating Systems Practical

12 November, 2014

OSP Lecture 6, L4 Microkernels 1/42 Contents

Introduction and design principles

Brief history of microkernels

L4: Basic abstractions

L4: Design and implementation choices

Keywords

Questions

OSP Lecture 6, L4 Microkernels 2/42 Outline

Introduction and design principles

Brief history of microkernels

L4: Basic abstractions

L4: Design and implementation choices

Keywords

Questions

OSP Lecture 6, L4 Microkernels 3/42 Context and terminology

I

I Kernel

I Monolithic kernel

I

OSP Lecture 6, L4 Microkernels 4/42 Operating system

I abbrv. OS

I Software (collection) to interface hardware with user I Components:

I Kernel: Linux, FreeBSD, Windows NT, XNU, L4, ... I Services/daemons: sysvinit, CUPS print server, udev, ... I Utilities: ls, Windows Commander, top I Other applications

OSP Lecture 6, L4 Microkernels 5/42 Kernel

I Components directly interfacing with hardware

I Examples? I “Core” of OS

I No general definition of “core”

OSP Lecture 6, L4 Microkernels 6/42 Monolithic vs. Micro-kernel

Application Syscall User VFS Mode Unix File Server Device Server IPC, Application Driver

Scheduler, virtual memory Kernel Mode Device drivers, dispatcher IPC, virtual memory IPC

Hardware Hardware

Source: http://www.cse.unsw.edu.au/

OSP Lecture 6, L4 Microkernels 7/42 Monolithic vs. Micro-kernel

Monolithic kernel Microkernel

I IPC, scheduling, I IPC, scheduling, memory management memory management

I File systems I API closer to the

I Drivers hardware

I Higher-level API

OSP Lecture 6, L4 Microkernels 8/42 Microkernel principles: minimality

I If it’s not critical, leave it out of the kernel I Pros:

I Small code base I Easy to debug I Trusted Computing Base, feasible for formal verification I Cons:

I Harder to find the “right” API design I Harder to optimize for high-performance

OSP Lecture 6, L4 Microkernels 9/42 Microkernel principles: user-level services

I Drivers, file systems, etc. as user space services I Pros:

I Isolation ⇒ limited attack surface I High availability, fault tolerance I Componentization, reusability I Cons:

I Performance: IPC is a bottleneck

OSP Lecture 6, L4 Microkernels 10/42 Microkernel principles: policy freedom

I Kernel provides mechanisms, not policies

I Policy definition is left up to the user space application I Pros:

I Flexibility I Cons:

I Hard to achieve, .g. for scheduling I May lead to application bloat

I Example: kernel provides user with memory, allocation algorithm depends on app

I Example: cache maintenance is explicitly exposed to user space, to improve performance

OSP Lecture 6, L4 Microkernels 11/42 Outline

Introduction and design principles

Brief history of microkernels

L4: Basic abstractions

L4: Design and implementation choices

Keywords

Questions

OSP Lecture 6, L4 Microkernels 12/42 0th Generation (1970s)

I Nucleus [Brinch Hansen ’70]

I [Wulf et al ‘74] I Issues

I Lack of hardware support I Bad performance

OSP Lecture 6, L4 Microkernels 13/42 1st Generation (1980s)

I

I Chorus I Issues

I Stripped-down monolithic kernels I Big I Bad performance: 100µs IPC

OSP Lecture 6, L4 Microkernels 14/42 2nd Generation (1990s, early 2000s)

I

I L3, L4 [Lietdke ’95] I Performance-oriented

I From scratch design I Architecture-dependent optimizations, e.g. reduced cache footprint I L3 was fully implemented in assembly I Issues

I Security

OSP Lecture 6, L4 Microkernels 15/42 L4 family tree

Verifie CCC d

L4-embed. seL4 Assember

Caps OKL4-µKernel

L4/MIPS OKL4-Microvisor

L4/Alpha Portable Codezero

Hazelnu L3→L4 “X” Pistachio t

Fiasco Fiasco.OC

GMD/IBM/Karlsruhe Nova UNSW/NICTA P4→ PikeOS Dresden

93 94 95 96 97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13

Source: http://www.cse.unsw.edu.au/

OSP Lecture 6, L4 Microkernels 16/42 L4 family tree

Verifie C d

L4-embed. seL4 Assember Asm+C

Caps OKL4-µKernel

L4/MIPS OKL4-Microvisor

L4/Alpha Portable Codezero

Hazelnu L3→L4 “X” Pistachio t

Fiasco Fiasco.OC

GMD/IBM/Karlsruhe C++ Nova UNSW/NICTA P4→ PikeOS Dresden

93 94 95 96 97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13

Source: http://www.cse.unsw.edu.au/

OSP Lecture 6, L4 Microkernels 17/42 3rd Generation (2007+)

I OKL4 Microvisor [Heiser and Leslie ’10]

I Microkernel and hypervisor

I Replaces some of the mechanisms with hypervisor mechanisms

I Deployed in older Motorola phones

OSP Lecture 6, L4 Microkernels 18/42 3rd Generation (2007+)

I seL4 [Elphinstone et al ’07, Klein et al ’09] I Security-oriented

I Capability-based access control I Strong isolation I Memory management policy fully exported to user space

I Kernel objects are first class citizens I All memory is explicitly allocated

I Formally verified [Klein et al ’09]

OSP Lecture 6, L4 Microkernels 19/42 Outline

Introduction and design principles

Brief history of microkernels

L4: Basic abstractions

L4: Design and implementation choices

Keywords

Questions

OSP Lecture 6, L4 Microkernels 20/42 What mechanisms to abstract?

I Bare minimum:

I Processor I Memory I Interrupts/exceptions I Must replace memory isolation with communication protocols

I Communication (IPC) I Synchronization

OSP Lecture 6, L4 Microkernels 21/42 Hypervisor vs. microkernel

Resource Hypervisor Microkernel Memory Virtual MMU (vMMU) Address space CPU Virtual CPU (vCPU) Thread or scheduler activation Interrupt Virtual IRQ (vIRQ) IPC message or signal Communication Virtual NIC Message-passing IPC Synchronization Virtual IRQ IPC message

Source: http://www.cse.unsw.edu.au/

OSP Lecture 6, L4 Microkernels 22/42 Abstracting memory

I Address space, fundamentally:

I A collection of virtual → physical mappings I Ways to expose this to user:

I Array of (physical) frames or (virtual) pages to be mapped I Cache for mappings which might vanish (Virtual TLB)

OSP Lecture 6, L4 Microkernels 23/42 Abstracting execution

I Threads, vCPUs

I What defines a thread? I Migrating threads

I Thread might be moved to different address space

OSP Lecture 6, L4 Microkernels 24/42 Abstracting execution

I Scheduling: map threads to CPUs

I What is the scheduling policy?

I Simple round-robin

I Policy-free scheduling?

OSP Lecture 6, L4 Microkernels 25/42 Communication abstraction

I Inter-Process Communication (IPC)

I Synchronous, asynchronous 6= blocking, non-blocking

I Traditional L4 IPC is fully synchronous I Asynchronous notification

I Sender asynchronous, receiver blocking and synchronous I Similar to Unix’s select

OSP Lecture 6, L4 Microkernels 26/42 Interrupt abstraction

I Hardware faults are abstracted through IPC

I Synchronous exceptions, page faults, etc. I Interrupts are asynchronous notifications

I Thread must register as a pagefault/exception/interrupt handler

OSP Lecture 6, L4 Microkernels 27/42 Access control

How do we specify objects? I IDs in a global list

I Provably insecure I Can DDoS, create covert channels, etc.

I IDs in per-address space lists

I Capabilities

OSP Lecture 6, L4 Microkernels 28/42 Access control: capabilities

I Developed in KeyKOS, Coyotos, Amoeba, L4 Pistachio, OKL4, seL4, ... I A token

I owned by the subject (e.g. a thread) I as proof that it has access rights to an object (e.g. a kernel object)

I All inter-domain accesses are mediated by capabilities

OSP Lecture 6, L4 Microkernels 29/42 Outline

Introduction and design principles

Brief history of microkernels

L4: Basic abstractions

L4: Design and implementation choices

Keywords

Questions

OSP Lecture 6, L4 Microkernels 30/42 Implementation language

I Initial L3 and L4: 100% x86 assembly

I Pistachio, OKL4 microkernel: C, C++, assembly

I OKL4 Microvisor, seL4: C

I seL4: Haskell prototype for correctness proof

OSP Lecture 6, L4 Microkernels 31/42 Inter-Process Communication

I seL4, OKL4: “Endpoints” as IPC targets

I Decouple target from actual service

I Fully signal-like asynchronous IPC (OKL4 Microvisor)

OSP Lecture 6, L4 Microkernels 32/42 Access control and resource management

I seL4: access control based on delegable capabilities

I Take-grant model I Provable security

I Information leaks are impossible I ... if the policy is correct I ... and the implementation is correct I ... and the compiler is correct I ... and the hardware isn’t faulty

OSP Lecture 6, L4 Microkernels 33/42 Access control and resource management

I seL4: resources are exposed as capabilities to physical memory I May be:

I Mapped I Delegated to children domains I Delegated to kernel: “retyped” into kernel objects

OSP Lecture 6, L4 Microkernels 34/42 Preemption in the kernel

I Interrupts are disabled when running in kernel

I Microkernel is in general non-preemptable

I Preemption points for long-running operations

OSP Lecture 6, L4 Microkernels 35/42 Scheduling

I Scheduling contexts (Fiasco.OC)

I Separate scheduling parameters from threads I Allow implementing hierarchical scheduling [Lackorzy´nskiet al ’12]

I Policy-free scheduling still unresolved

OSP Lecture 6, L4 Microkernels 36/42 Multi-processors

I Initial L4 design is uniprocessor

I seL4: same, due to formal verification constraints

I Possible approach: multikernels [M Von Tessin ’12]

OSP Lecture 6, L4 Microkernels 37/42 Outline

Introduction and design principles

Brief history of microkernels

L4: Basic abstractions

L4: Design and implementation choices

Keywords

Questions

OSP Lecture 6, L4 Microkernels 38/42 Keywords

I microkernel I inter-process communication

I l4 I access control

I thread I capability

I address space I preemption

OSP Lecture 6, L4 Microkernels 39/42 Resources

I http://dl.acm.org/citation.cfm?id=224075 I http://www.cse.unsw.edu.au/~cs9242/13/lectures/ I http://os.inf.tu-dresden.de/L4/

I http://ssrg.nicta.com.au/projects/seL4/

I http://os.inf.tu-dresden.de/fiasco/

I http://www.ok-labs.com/products/okl4-microvisor

OSP Lecture 6, L4 Microkernels 40/42 Outline

Introduction and design principles

Brief history of microkernels

L4: Basic abstractions

L4: Design and implementation choices

Keywords

Questions

OSP Lecture 6, L4 Microkernels 41/42 Questions

?

OSP Lecture 6, L4 Microkernels 42/42