Construction

Introduction

Hermann Härtig SS2012 Torsten Frenzel

TU Dresden Operating Systems Group Microkernel Construction ClassClass GoalsGoals

Provide deeper understanding of OS mechanisms

Introduce L4 principles and concepts

Make you become enthusiastic L4 hackers

Propaganda for OS research at TU Dresden

Hermann Härtig Torsten Frenzel

TU Dresden Operating Systems Group 2 Microkernel Construction AdministrationAdministration

 Thursday, 4th DS, 2 SWS  Slides: http://www.tudos.org → Teaching → Microkernel Construction  Subscribe to our mailing list: http://www.tudos.org/mailman/listinfo/mkc2012  In winter term: – Construction of Microkernel-based Systems (2 SWS) – Various Labs

Hermann Härtig Torsten Frenzel

TU Dresden Operating Systems Group 3 Microkernel Construction „„Monolithic“Monolithic“ KernelKernel SystemSystem DesignDesign

Application ApplicationApplication Application Unprivileged Mode

Privileged File Network Mode Systems Subsystem

Memory Drivers Management Management

Monolithic Kernel

Hermann Härtig Torsten Hardware Frenzel

TU Dresden Operating Systems Group 4 Microkernel Construction MonolithicMonolithic KernelKernel OSOS (Propaganda)(Propaganda)

 System components run in privileged mode ➔ No protection between system components – Faulty driver can crash the whole system – More than 2/3 of today's OS code are drivers ➔ No need for good system design – Direct access to data structures – Undocumented and frequently changing interfaces ➔ Big and inflexible – Difficult to replace system components Why something different? Hermann  Härtig More and more difficult to manage increasing OS Torsten Frenzel complexity

TU Dresden Operating Systems Group 5 Microkernel Construction MicrokernelMicrokernel SystemSystem DesignDesign

Application Application Application

File Network Systems Stacks

Memory Process Management Management Drivers Unprivileged System Services Mode

Privileged Tasks Threads IPC Mode

Hermann Härtig Microkernel Torsten Frenzel Hardware TU Dresden Operating Systems Group 6 Microkernel Construction ExampleExample –– IBMIBM WorkplaceWorkplace OSOS // MachMach

OS/2 DOS OS/400 AIX Windows Application Application Application Application Application

OS/2 DOS OS/400 Windows AIX Personality Personality Personality Personality Personality

File Network Service Security Power Management

Default Pager Device Support Bootstrap Name Service

Hermann Microkernel Härtig Torsten Frenzel ARM PowerPC IA32 MIPS Alpha

TU Dresden Operating Systems Group 7 Microkernel Construction ExampleExample –– QNXQNX // NeutrinoNeutrino

. Embedded systems  system (IPC)  Network transparency

Application Application Application

Filesystem Device Process Network Manager Manager Manager Manager Unprivileged Mode

Privileged Network IPC Scheduler Mode Redirector Driver Hermann Härtig Neutrino - Microkernel Torsten Frenzel Hardware TU Dresden Operating Systems Group 8 Microkernel Construction MoreMore InterestingInteresting

Hermann Härtig Torsten Frenzel

TU Dresden Operating Systems Group 9 Microkernel Construction Example:Example: VPNVPN

Hermann Härtig Torsten Frenzel

TU Dresden Operating Systems Group 10 Microkernel Construction

Hermann Härtig Torsten Frenzel

TU Dresden Operating Systems Group 11 Microkernel Construction

Hermann Härtig Torsten Frenzel

TU Dresden Operating Systems Group 12 Microkernel Construction

Hermann Härtig Torsten Frenzel

TU Dresden Operating Systems Group 13 Microkernel Construction

Hermann Härtig Torsten Frenzel

TU Dresden Operating Systems Group 14 Microkernel Construction MoreMore OnOn CriticalCritical ApplicationsApplications

Hermann Härtig Torsten Frenzel

TU Dresden Operating Systems Group 15 Microkernel Construction

Hermann Härtig Torsten Frenzel

TU Dresden Operating Systems Group 16 Microkernel Construction

Hermann Härtig Torsten Frenzel

TU Dresden Operating Systems Group 17 Microkernel Construction VirtualVirtual PrivatePrivate FileFile SystemSystem

Hermann Härtig Torsten Frenzel

TU Dresden Operating Systems Group 18 Microkernel Construction

Hermann Härtig Torsten Frenzel

TU Dresden Operating Systems Group 19 Microkernel Construction ShoppingShopping

Hermann Härtig Torsten Frenzel

TU Dresden Operating Systems Group 20 Microkernel Construction

Hermann Härtig Torsten Frenzel

TU Dresden Operating Systems Group 21 Microkernel Construction MicrokernelMicrokernel OSOS -- TheThe VisionVision (1)(1)

 System components run as user-level servers  Protection and isolation between system components – More secure / safe systems – Less error prone – Small  „Enforces“ clear system design – Well defined interfaces to system services – No dependencies between system services other than explicitly specified through service interfaces  Small and flexible – Small OS kernel Hermann Härtig – Easier to replace system components Torsten Frenzel

TU Dresden Operating Systems Group 22 Microkernel Construction VisionsVisions vs.vs. RealityReality

 Flexibility and Customizable – Monolithic kernels are modular  Maintainability and complexity – have layered architecture ✔ Robustness – are superior due to isolated system components – Trusted code size () • Fiasco kernel: about 15.000 loc • kernel: about 300.000 loc (without drivers) Performance Hermann ✗ Härtig Torsten – Application performance degraded Frenzel – Communication overhead (see next slides)

TU Dresden Operating Systems Group 23 Microkernel Construction RobustnessRobustness vs.vs. PerformancePerformance (1)(1)

 System calls – Monolithic kernel: 2 kernel entries/exits – Microkernel: 4 kernel entries/exits + 2 context switches

Application Application

1 4 Driver Driver

2 3

Microkernel Monolithic kernel Hermann Härtig Torsten Frenzel HardwareHardware Hardware

TU Dresden Operating Systems Group 24 Microkernel Construction RobustnessRobustness vs.vs. PerformancePerformance (2)(2)

 Calls between system services – Monolithic kernel: 1 function call – Microkernel: 4 kernel entries/exits + 2 context switches

Network Network Driver Driver Subsystem Subsystem 1 4 2 3

Microkernel Monolithic kernel

Hermann Härtig Torsten Hardware Hardware Frenzel

TU Dresden Operating Systems Group 25 Microkernel Construction ChallengesChallenges

 Build functional powerful and fast microkernels – Provide abstractions and mechanisms – Fast communication primitive (IPC) – Fast context switches and kernel entries/exits ➔ Subject of this lecture

 Build efficient OS services – – Synchronization – Device Drivers – File Systems Hermann – Communication Interfaces Härtig Torsten ➔ Frenzel Subject of lecture “Construction of Microkernel-based systems” (in winter term) TU Dresden Operating Systems Group 26 Microkernel Construction L4L4 MicrokernelMicrokernel FamilyFamily

 Originally developed by (GMD / IBM Research)  Current development: – Uni Karlsruhe: Pistachio – UNSW/NICTA/OKLABS: OKL4, SEL4, L4Verified – TU Dresden: Fiasco.OC, Nova  Support for hardware architectures: – , ARM, ...

Hermann Härtig Torsten Frenzel

TU Dresden Operating Systems Group 27 Microkernel Construction MoreMore MicrokernelsMicrokernels (Incomplete(Incomplete list)list)

 Commercial kernels – Singularity @ Microsoft Research – @ IBM Research – velOSity/INTEGRITY @ Green Hills – Chorus/ChorusOS @ Sun Microsystems – PikeOS @ SYSGO AG – OKL4  Research kernels – EROS/CoyotOS @ John Hopkins University – @ FU Amsterdam – Amoeba @ FU Amsterdam Hermann Härtig – Pebble @ Bell Labs Torsten Frenzel – Grasshopper @ University of Sterling – Flux/Fluke @ University of Utah TU Dresden Operating Systems Group 28 Microkernel Construction L4L4 -- ConceptsConcepts

 Jochen Liedtke: “A microkernel does no real work” – Kernel provides only inevitable mechanisms – No policies implemented in the kernel  Abstractions – Tasks with address spaces – Threads executing programs/code  Mechanisms – Resource access control – Scheduling – Communication (IPC)

Hermann Härtig Torsten Frenzel

TU Dresden Operating Systems Group 29 Microkernel Construction ThreadsThreads andand TasksTasks

Task A Task B

User User Code Code Thread2

User User User Stack Stack Stack

Kernel Code Thread1 Thread3

Hermann Kernel Kernel Kernel Härtig Torsten Stack Stack Stack Frenzel Microkernel TU Dresden Operating Systems Group 30 Microkernel Construction ThreadsThreads (1)(1)

 Represent unit of execution – Execute user code (application) – Execute kernel code (system calls, page faults, , exceptions)  Subject to scheduling – Quasi-parallel execution on one CPU – Parallel execution on multiple CPUs – Voluntarily switch to another possible – Preemptive scheduling by the kernel according to certain parameters  Associated with an

Hermann – Executes code in one task at one point in time Härtig Torsten • Migration allows threads move to another task Frenzel – Several threads can execute in one task TU Dresden Operating Systems Group 31 Microkernel Construction TasksTasks (1)(1)

 Represent domain of protection and isolation  Container for code, data and resources  Address space: capabilities + memory pages  Three management operations: – Map: share page with other address space – Grant: give page to other address space – Unmap: revoke previously mapped page

X X X

grant unmap Hermann map Härtig Torsten Frenzel X X X TU Dresden Operating Systems Group 32 Microkernel Construction RecursiveRecursive AddressAddress SpacesSpaces

Application 1 Application 2

Pager 3

Pager 1 Pager 2

Initial Pager Hermann Härtig Torsten Frenzel Physical Memory TU Dresden Operating Systems Group 33 Microkernel Construction Messages:Messages: CopyCopy DataData

 Direct and indirect data copy  UTCB message (special area)  Special case: register-only message  Pagefaults during user-level memory access possible Task A Task B

receive(msg, …) send(msg,…)

copy data area

data area

Hermann data word 1 data word 1 Härtig msg data word 2 msg data word 2 Torsten send string receive string Frenzel

TU Dresden Operating Systems Group 34 Microkernel Construction Message:Message: MapMap ReferencesReferences

 Used to transfer memory pages and capabilities  Kernel manipulates page tables  Used to implement the map/grant operations

Task A Task B

send(msg,…) receive(msg, …)

flexpage map

flexpage

Hermann send flexpage received flexpage Härtig msg msg Torsten receive window Frenzel

TU Dresden memory page Operating Systems Group 35 Microkernel Construction CommunicationCommunication andand ResourceResource ControlControl

 Need to control who can send data to whom – Security and isolation – Access to resources  Approaches – IPC-redirection/introspection – Central vs. Distributed policy and mechanism – ACL-based vs. capability-based

Task A Task B

Thread IPC? Thread

Hermann Härtig Resource Access? Torsten Frenzel Hardware TU Dresden Resources Operating Systems Group 36 Microkernel Construction CapabilitiesCapabilities Task A Task B

Capability Handles Capability Handles 3 1 2 1 2 2

Capability Table Capability Table C1 C3 C5C5 C1 C2 C4

Hermann Kernel Kernel Kernel Kernel Kernel Härtig Torsten Object1 Object2 Object3 Object4 Object5 Frenzel

TU Dresden Operating Systems Group 37 Microkernel Construction CapabilitiesCapabilities -- DetailsDetails  Kernel objects represent resources and communication channels  Capability – Reference to kernel object – Associated with access rights – Can be mapped from task to another task  Capability table is task-local data structure inside the kernel – Similar to – Valid entries contain capabilities  Capability handle is index number to reference entry

Hermann into capability table Härtig Torsten – Similar to file handle (in POSIX) Frenzel  Mapping capabilities establishes a new valid entry into TU Dresden Operating Systems Group the capability table 38 Microkernel Construction PagePage FaultsFaults andand PagersPagers

 Page Faults are mapped to IPC – Pager is special thread that receives page faults – Page fault IPC cannot trigger another page fault  Kernel receives the flexpage from pager and inserts mapping into page table of application  Other faults normally terminate threads

map Application X X Pager

Unnprivileged Mode 1.Page Fault 4.Resume 2.receive 3.send(X) Hermann Härtig Privileged Torsten Mode Frenzel L4 Microkernel TU Dresden Operating Systems Group 39 Microkernel Construction DeviceDevice DriversDrivers

 Hardware interrupts: mapped to IPC  I/O memory & I/O ports: mapped via flexpages

IO-Memory Driver

2.receive(irq-id, …)

L4 Microkernel

map 1. Interrupt

Hermann Härtig IO-Memory Torsten Frenzel

TU Dresden Operating Systems Group 40 Microkernel Construction L4L4 ApplicationsApplications -- LL4LinuxLinux

 Paravirtualized and native Linux applications run as user-level L4 tasks  System calls / page faults are mapped to L4 IPC

L4Linux Linux Linux Server Application Application

L4 Interface

Hermann Unprivileged System Services Härtig Mode Torsten Frenzel Privileged Mode L4 Microkernel TU Dresden Operating Systems Group 41 Microkernel Construction LectureLecture OutlineOutline

 Introduction  Address spaces, threads, thread switching  Kernel entry and exit  Thread synchronization  IPC  Address space management  Scheduling  Portability  Platform optimizations

Hermann  Virtualization Härtig Torsten Frenzel

TU Dresden Operating Systems Group 42