ARTIST Summer School in Europe 2010 Autrans (near Grenoble), France September 5-10, 2010
The L4 Microkernel
Invited Speaker: Prof. Hermann Härtig Technische Universität Dresden
http://www.artist-embedded.org/
L4
http://www.artist-embedded.org/ COTS - SW
Applet
Firefox Flash X11 Linux! Kernel
Keyboard
Hermann Härtig L4 Microkernel 3 MICRO
Native Native Microkernel Microkernel App App
Window File IP Stack Server System
Framebuffer! Disk Network Driver Driver Driver
L4 Microkernel
Hermann Härtig L4 Microkernel 4 MICRO
Native Native Microkernel Microkernel App App Virtualization! Container for Window File IP Stack Legacy OS Server System
Framebuffer! Disk Network Driver Driver Driver
L4 Microkernel
Hermann Härtig L4 Microkernel 5 OUTLINE
■ Using a small kernel – Hermann Härtig ■ Motivation, Cost & Benefit ■ Case studies ■ A short history of L4 ■ L4 Kernel interface ■ Capability system design – Michael Roitzsch ■ Mobile use cases – Adam Lackorzynski
Hermann Härtig L4 Microkernel 6 Department of Computer Science Institute of System Architecture, Operating Systems Group
USING A SMALL KERNEL
HERMANN HÄRTIG WHY MICRO
Native Native Microkernel Microkernel App App Virtualization! Container for Window File IP Stack Legacy OS Server System
Framebuffer! Disk Network Driver Driver Driver
L4 Microkernel
Hermann Härtig L4 Microkernel 8 COST & BENEFIT
■ Performance
■ (Failure) Isolation ■ Openness ■ Small (Minimal) Trusted Computing Base
Hermann Härtig L4 Microkernel 9 BENEFITS
■ Flexibility? SW engineering ./. microkernels ■ Difficulty to build? can be harder to build
Hermann Härtig L4 Microkernel 10 ISOLATION
■ Separate address spaces for components ■ Message passing interfaces ■ Communication controlled by capabilities
■ Immediate: I/O Drivers tamed ■ Base for fault containment
■ fault recovery?
Hermann Härtig L4 Microkernel 11 ALTERNATIVE Virtual machines ■ provide separate machines ■ requires ■ emulation of physical HW interface ■ an operating system in each VM ■ large!grained components
more later
Hermann Härtig L4 Microkernel 12 ALTERNATIVE Languages with component!support ■ Use one language or language family ■ Fine!grained components (modules, objects, …) ■ Compiler and runtime enforce isolation ■ Closed systems ■ Examples: Burroughs 7700, B extended Algol, Espol, Concurrent Pascal, Modula, Oberon, various Java systems
Hermann Härtig L4 Microkernel 13 OPENNESS
Microkernels ■ Minimal kernel and hardware enforce separation ■ Only kernel runs in CPU privileged mode ■ Components are user!level processes ■ No restrictions on component software ■ Reuse of legacy software
Hermann Härtig L4 Microkernel 14 MINIMAL TCB
Trusted Computing Base „A small amount of software and hardware that security depends on and that we distinguish from a much larger amount that can misbehave without affecting security.“ — Lampson et al.
Hermann Härtig L4 Microkernel 15 MINIMAL TCB
Trusted Computing Base „A small amount of software and hardware that * depends on and that we distinguish from a much larger amount that can misbehave without affecting * .“ In this lecture: * : security, real!time, fault tolerance, ... TCB is application specific
Hermann Härtig L4 Microkernel 16 MINIMAL TCB
■ General Approach: ■ Divide system into uncritical and (minimal) critical parts ■ Include minimal set of components into TCB ■ offload uncritical parts, e.g. into legacy SW ■ Split critical part into isolated components ■ Benefit: ■ small application!specific TCB
Hermann Härtig L4 Microkernel 17 CASE STUDIES
Hermann Härtig L4 Microkernel 18 L4LINUX
App App
X11
L4Linux Server
L4 Microkernel Fiasco.OC
Hermann Härtig L4 Microkernel 19 COST !1997
Time- Sharing (Härtig, Hohmuth, Liedtke, Schönberg, Applicatio Wolter: ns The Performance of !-Kernel based Systems, SOSP 1997)"
L4Linux jobs per minute perminute jobs
simulated load L4
Hermann Härtig L4 Microkernel COST !1997
Time- Sharing Applicatio ns
L4Linux
L4
Hermann Härtig L4 Microkernel L4LINUX
App App App App
X11 X11
L4Linux Server L4Linux Server
L4 Microkernel Fiasco.OC
Hermann Härtig L4 Microkernel 22 L4RE
App App App App
X11 X11
Resource L4Linux Server Management and L4Linux Server Virtualization Support Layer
moe ned io rtc mag
L4 Microkernel Fiasco.OC
Hermann Härtig L4 Microkernel 23 L4RE
App App App App
X11
L4Symbian Server L4Linux Server
moe ned io rtc mag
L4 Microkernel Fiasco.OC
Hermann Härtig L4 Microkernel 24 L4LINUX
App App App App
X11 X11
L4Linux Server L4Linux Server
L4 Microkernel Fiasco.OC
Hermann Härtig L4 Microkernel 25 L4RE
App App App App
X11 X11
Resource L4Linux Server Management and L4Linux Server Virtualization Support Layer
moe ned io rtc mag
L4 Microkernel Fiasco.OC
Hermann Härtig L4 Microkernel 26 L4RE
App App App App
X11
L4Symbian Server L4Linux Server
moe ned io rtc mag
L4 Microkernel Fiasco.OC
Hermann Härtig L4 Microkernel 27 NATIVE APPS
App App
X11
L4Linux Server Security!Sensitive Application
moe ned io rtc mag
L4 Microkernel Fiasco.OC
Hermann Härtig L4 Microkernel 28 HYBRID APPS
Helper App
X11 Security!Sensitve Application
L4Linux Server Secure Application Core
moe ned io rtc mag
L4 Microkernel Fiasco.OC
Hermann Härtig L4 Microkernel 29 CASE STUDY
Micro!SINA VPN Box ■ security goals: ■ connect sets of trusted machines ■ ensure confidentiality and integrity ■ non goal: ■ availability
Hermann Härtig L4 Microkernel 30 USE CASES
eth0 eth1
L4Linux Server L4Linux Server
IP Viaduct
moe ned io rtc mag
L4 Microkernel Fiasco.OC
Hermann Härtig L4 Microkernel 31 USE CASES
DDE DDE eth0 IP Viaduct eth1
moe ned io rtc mag
L4 Microkernel Fiasco.OC
Hermann Härtig L4 Microkernel 32 HYBRID APPS
Slide Loader
X11 Presentation Application
L4Linux Server Presenter
moe ned io rtc mag
L4 Microkernel Fiasco.OC
Hermann Härtig L4 Microkernel 33 HYBRID APPS
E!Mail Client
X11
L4Linux Server E!Mail Signing
moe ned io rtc mag
L4 Microkernel Fiasco.OC
Hermann Härtig L4 Microkernel 34 HYBRID APPS
Address Book ROBIN Demo Scenario
X11
L4Linux Server Dialer with Filter for Premium!rate Numbers
moe ned io rtc mag
L4 Microkernel Fiasco.OC
Hermann Härtig L4 Microkernel 35 HYBRID APPS
Browser Nizza Demo Scenario
X11
L4Linux Server Secure Transactions for Home!Banking
moe ned io rtc mag
L4 Microkernel Fiasco.OC
Hermann Härtig L4 Microkernel 36 NUMBERS
Original AppCore Reduc! Scenario tion kLOC kMCC kLOC kMCC Factor e!commerce 978 151 10 15 100" Browser VPN Gateway 155 25 74 10 2.1" FreeS/WAN Email signer 250 45 54 11 4.6" Thunderbird TCB 1485 238 100 14 14" Linux + X11
Reducing TCB Complexity for Security!Sensitive Applications: Three Case Studies Lenin Singaravelu, Calton Pu, Hermann Härtig, Christian Helmuth, EuroSys 2006
Hermann Härtig L4 Microkernel 37 REAL-TIME
App App
X11
L4Linux Server Real!Time Application
moe ned io rtc mag
L4 Microkernel Fiasco.OC
Hermann Härtig L4 Microkernel 38 REAL-TIME
Legacy App Hybrid App Real!Time App
RT!File RT!Net L4Linux Server Stubs
RT!Disk RT!NIC DOpE
moe ned io rtc mag
L4 Microkernel Fiasco.OC
Hermann Härtig L4 Microkernel 39 ORTHOGONAL
Microkernel Virtual Machine
Isolation Rehosting Light!Weight Microkernels ■ Componentization of operating system ■ Split applications ■ Critical part on microkernel ■ Uncritical on commodity OS ■ Small Trusted Computing Bases
Hermann Härtig L4 Microkernel 40 ORTHOGONAL
Microkernel Virtual Machine
Isolation Rehosting
Virtual Machines ■ Provide virtual hardware interface ■ Reuse of COTS operating systems and applications with no modification ■ At the price of complexity
Hermann Härtig L4 Microkernel 41 NOVA State of the Art: Monolithic Hypervisors
guest mode VM VM VM
Monolithic Hypervisor
Storage Management x86 Virtualization> 100,000 lines of code Network Device Drivers
host mode
Monolithic hypervisor is single point of failure
UdoHermann Steinberg Härtig L4 MicrokernelNOVA 42 4 NOVA OS Virtualization ArchitectureNOVA
guest mode VM VM VM
20,000 LOC VMM VMM VMM
7,000 LOC Applications Partition Manager Device Drivers !"#$ %#$&#' 9,000 LOC Microhypervisor host mode
Udo Steinberg NOVA 7
Hermann Härtig L4 Microkernel 43 SHORT HISTORY
• Eumel, L3, BirliX 1995 • first version of L4 1997 • L4Linux, the first major application 1998 • Fiasco, the first HLL implementation • PikeOS: first commercial derivative • real!time systems based on Fiasco • Pistachio (Uni Karlsruhe)
Hermann Härtig L4 Microkernel 44 SHORT HISTORY
2000 • First commercial usage (Fiasco on a DRM product) 2005 • Qualcomm adopts L4 kernel from NICTA (Pistachio derivative) 2009 • Full formal verification of implementation in Haskell (NICTA) 2009 • Fiasco.OC, L4Re 2009 • NOVA
Hermann Härtig L4 Microkernel 45 L4 KERNEL ABSTRACTIONS
Hermann Härtig L4 Microkernel 46 ABSTRACTIONS
■ Task (Address space: memory & capabilities) ■ Thread ■ Communication (IPC)
Hermann Härtig L4 Microkernel 47 MAIN MEMORY
■ Management of Physical Memory at user"level? ■ only kernel can access page tables?
Hermann Härtig L4 Microkernel 48 MAIN MEMORY
Task C Task D
Task B Page Fault transformed into message to Task A handler
Hermann Härtig L4 Microkernel 49 MAIN MEMORY
Task C Task D
Task B Handler returns message with PTE as payload, kernel adds to Task A address space
Hermann Härtig L4 Microkernel 50 MAIN MEMORY
Task C Task D
Task B
Task A
Hermann Härtig L4 Microkernel 51 MAIN MEMORY
Task C Task D ! !
Task B revoke: kernel maintains ! data structure for Task A revoking
Hermann Härtig L4 Microkernel 52 USES OF IPC
■ data ■ exceptions ■ interrupts ■ memory references (page fault handling) ■ capabilities
Hermann Härtig L4 Microkernel 53 NEXT …
■ Using a small kernel – Hermann Härtig ■ Capability system design – Michael Roitzsch ■ Mobile use cases – Adam Lackorzynski
Hermann Härtig L4 Microkernel 54 Department of Computer Science Institute of System Architecture, Operating Systems Group
DESIGN OF A CAPABILITY SYSTEM
MICHAEL ROITZSCH SYSTEM DESIGN
Applications
Services
Kernel
Michael Roitzsch Design of a Capability System 2 DESIGN GOALS
■ application!centric interfaces ■ object!based design ■ easy setup and destruction of subsystems ■ object invocation by message passing ■ uniform security model ■ all services virtualizable ■ flexible and efficient support for multicore
Michael Roitzsch Design of a Capability System 3 EXAMPLE
Worker A Worker B
Manager
Service
Michael Roitzsch Design of a Capability System 4 GOOGLE CHROME
■ separate processes ■ chrome parent ■ sandboxes for tabs ■ implementation on Linux: glorious mix of chroot(), clone() and setuid() ■ there must be a better way…
Michael Roitzsch Design of a Capability System 5 TWO WORLDS
POSIX POLA
operations allowed nothing allowed by by default default
some limited every right must restrictions apply be granted
ambient authority explicit authority
Michael Roitzsch Design of a Capability System 6 L4RE
L4Re — the L4 Runtime Environment set of libraries and system services on top of the Fiasco.OC microkernel
Microkernel L4Re
Michael Roitzsch Design of a Capability System 7 CAPABILITIES
■ Fiasco.OC and L4Re form an object!capability system ■ actors in the system are objects ■ objects have local state and behavior ■ capabilities are references to objects ■ object interaction requires a capability ■ capabilities cannot be forged
Michael Roitzsch Design of a Capability System 8 CAPABILITIES
Task A Task B
1 1 2 2 3 3 4 4 5 5 Capability Table Capability Table
A B C D E Fiasco.OC
Michael Roitzsch Design of a Capability System 9 HOW TO USE?
■ invocation of any object requires a capability to that object ■ no global names ■ no sophisticated rights representation beyond capability ownership ■ just four rights bits on objects ■ C++ language integration ■ capabilities passed as message payload
Michael Roitzsch Design of a Capability System 10 CAP TRANSFER
Task A Task B
A
Michael Roitzsch Design of a Capability System 11 CAP TRANSFER
Task A Task B
A
1 2 3 4 5 1 2 3 4 5
Michael Roitzsch Design of a Capability System 11 EXAMPLE
Worker A Worker B
Manager
Service
Michael Roitzsch Design of a Capability System 12 ANSWERING
How do you send an answer to a client? ■ Always include a backward capability in every request? ■ Establish backward capability once and cache? ■ call!return!semantics as the standard case ■ implicit reply capability ■ use!once, cannot be passed on
Michael Roitzsch Design of a Capability System 13 EXAMPLE
Worker A Worker B
Manager
mag
Michael Roitzsch Design of a Capability System 14 MAG ■ factory for new framebuffer sessions Manager ■ session object ■ backing store memory ■ view: visible rectangle on the backing store Factory S S mag ■ metadata, refresh method ■ How does it appear on the screen?
Michael Roitzsch Design of a Capability System 15 MAG
Factory S S ■ hardware framebuffer is mag memory with side effect ■ all memory is initially fb!drv mapped to the roottask ■ framebuffer driver
moe ■ find framebuffer memory ■ wrap in FB!interface
Memory ■ same interface as mag’s
Michael Roitzsch Design of a Capability System 16 INTERFACES
■ L4Re uses one interface per resource ■ low!level system resources are managed by the kernel ■ CPU, memory, IRQ ■ minimal policy ■ user!level servers can reimplement and augment interfaces ■ virtualizable interfaces
Michael Roitzsch Design of a Capability System 17 EXAMPLE
Worker B
Manager
? Servicemag
Michael Roitzsch Design of a Capability System 18 SUBSYSTEMS
Subsystem Life ■ subsystems are opaque ■ parents can restrict the resources ■ parents cannot restrict their sub!structure Subsystem Death ■ How to deallocate resources in servers? ■ notify all servers used by the subsystem? ■ garbage collection
Michael Roitzsch Design of a Capability System 19 CONCLUSION
! coherent per!resource interfaces " ! all services provided as objects " ! garbage collection for server resources " ! invocation is the only system call " ! object!capability system " ! all interfaces can be interposed " ! see next talk "
Michael Roitzsch Design of a Capability System 20 Department of Computer Science Institute of System Architecture, Operating Systems Group
MOBILE USE CASES
ADAM LACKORZYŃSKI OUTLINE
■ ICT!eMuCo project ■ Multi!Cores and Load Balancing ■ Virtualization
Adam Lackorzynski Mobile Use Cases 2 ICT-EMUCO
■ Embedded Multicore Computing ■ FP7 Project, STREP ■ Feb 2008 – Jan 2010 ■ Partners: ■ ARM, Infineon, Ruhr! Uni Bochum, IBM, Uni of Timisoara, Uni York, TU!Dresden
Adam Lackorzynski Mobile Use Cases 3 ARCHITECTURE
■ Microkernel based system ■ ARM11MPCore ■ 4 cores ■ Modem Stack ■ App!OS
Adam Lackorzynski Mobile Use Cases 4 THE EMUCO OS
■ Isolation ■ Secure communication ■ Timing properties ■ Multi!core capable Protocol Stack ■ Power Management L4Re Runtime Environment ■ Embedded systems Fiasco.OC Microkernel ■ Flexible & usable Hardware Platform
Adam Lackorzynski Mobile Use Cases 5 OUTLINE
■ ICT!eMuCo project ■ Multi!Cores and Load Balancing ■ Virtualization
Adam Lackorzynski Mobile Use Cases 6 MULTI-CORES
■ Shared memory multi!processor systems ■ Model: ■ Cross CPU tasks ■ Cross CPU notifications ■ Cross CPU IPC ■ Shared memory ■ Local scheduling ■ Fixed!prio, round!robin scheduler on each core
Adam Lackorzynski Mobile Use Cases 7 DISTRIBUTION
How to Distribute Work? ■ Kernel provides migration mechanism ■ No automatic migration, decision is policy App2 App1 App3 ?
CPU1 CPU2 CPU3 CPU4
Adam Lackorzynski Mobile Use Cases 8 L4::SCHEDULER
■ L4::Scheduler interface ■ Run/Migrate a thread with parameters ■ Priority, CPU set, budget, ... ■ scheduler.run_thread(thread, sched_param); ■ Kernel implementation ■ Singleton, has all resources (all CPUs, all CPU time) ■ Chooses one CPU from CPU set, fixed
Adam Lackorzynski Mobile Use Cases 9 LOAD BALANCING ■ Load balancer component ■ Implements L4::Scheduler interface ■ Hides platform details from application ■ Implements policy App2 App1 App3 Load Balancer?
CPU1 CPU2 CPU3 CPU4
Adam Lackorzynski Mobile Use Cases 10 LOAD BALANCER
■ Implements balancing strategy Protocol Stack ■ Application specific scheduler instances 3 CPUs 2 CPUs ■ Enforces scheduling LB Scheduler Policy Scheduler policies ■ Combines client Fiasco.OC Scheduler policies CPU CPU CPU CPU
Adam Lackorzynski Mobile Use Cases 11 USE CASES ■ Multi!threaded program ■ Distribute and balance ■ Sophisticated real!time application ■ No migration by load balancer ■ Threads always on the same CPU (cache locality) ■ Threads always on different CPUs (no interference) ■ Virtual machine Adam Lackorzynski Mobile Use Cases 12 OUTLINE
■ ICT!eMuCo project ■ Multi!Cores and Load Balancing ■ Virtualization
Adam Lackorzynski Mobile Use Cases 13 VIRTUALIZATION ■ Application side ■ Standard OS → Standard applications ■ Integration in the system
■ Resource and Protocol device usage Stack
■ Isolation of the L4Re Runtime Environment virtual machine Fiasco.OC Microkernel ■ No disturbance of Hardware Platform other programs
Adam Lackorzynski Mobile Use Cases 14 EMUCO PLATFORM
■ ARM architecture ■ Available CPU features: MMU ■ Paravirtualization – L4Linux ■ TECOM!FP7: TrustZone for Virtualization
Adam Lackorzynski Mobile Use Cases 15 L4LINUX ■ Adapted Linux kernel ■ runs on Fiasco.OC & L4Re ■ „Normal“ program, runs Linux kernel code in user mode, including device drivers ■ Binary compatible for applications ■ Address space for kernel and each program ■ Programs isolated from each other ■ Kernel isolated from programs ■ CPU, memory, devices Adam Lackorzynski Mobile Use Cases 16 L4LINUX
L4Linux CPU Virtualization
Linux Linux ■ Native execution Program Program ■ Exceptions reflected by microkernel ■ System calls Linux Kernel ■ Page faults Fault handling ■ Other exceptions Microkernel
Adam Lackorzynski Mobile Use Cases 17 SMP
Multi!Processor Virtualization ■ Linux has multiple virtual CPUs (vCPU) ■ Shared memory between cores ■ Migration of application threads is done by the Linux kernel ■ Inter (v)CPU communication done with microkernel primitives
Adam Lackorzynski Mobile Use Cases 18 MEMORY
L4Linux Memory Virtualization ■ L4Re supplies Linux memory (virtual) ■ MMU managed by microkernel ■ Hooks in Linux page!table code use Fiasco memory!mapping functionality
Adam Lackorzynski Mobile Use Cases 19 DEVICES
■ Virtual PCI bus and/or platform devices ■ Pass!through devices according to configuration ■ Stub drivers for L4Re services: ■ Framebuffer driver for windowing system ■ Input driver for keyboard/mouse events ■ Serial driver for basic input/output ■ Network drivers for virtual switch
Adam Lackorzynski Mobile Use Cases 20 PLATFORM ■ Central IO service ■ Device discovery ■ Device enumeration ■ Per client device access ■ Virtual buses ■ Virtual interrupt controller ■ Device pass!through Adam Lackorzynski Mobile Use Cases 21 VIRTUALIZATION
Faithful Virtualization ■ Unmodified guest OS ■ AMD SVM, Intel VT ■ 1500 LoC in Fiasco for SVM and VT support ■ Off!the!shelf VMM (e.g. QEmu on L4Linux)
Adam Lackorzynski Mobile Use Cases 22 WHAT ELSE?
■ DDE, virtual network switch ■ Debugging: Valgrind ■ Checkpoint & Restart ■ ARM Virtualization ■ Run!time environment: libc, libstdc++, virtual!FS, pthread, communication framework, dynamic linking, scriptable startup with lua, ...
Adam Lackorzynski Mobile Use Cases 23 GO DOWNLOAD
http://L4Re.tudos.org
Adam Lackorzynski Mobile Use Cases 24