Copyright Notice These slides are distributed under the Creative Commons Attribution 3.0 License COMP9242 Advanced OS • You are free: S2/2016 W01: Introduction to seL4 – to share—to copy, distribute and transmit the work @GernotHeiser – to remix—to adapt the work • under the following conditions: – Attribution: You must attribute the work (but not in any way that suggests that the author endorses you or your use of the work) as follows: “Courtesy of Gernot Heiser, UNSW Australia” The complete license text can be found at http://creativecommons.org/licenses/by/3.0/legalcode 2 COMP9242 S2/2016 W01 © 2016 Gernot Heiser. Distributed under CC Attribution License Monolithic Kernels vs Microkernels Microkernel Evolution • Idea of microkernel: First generation Second generation Third generation – Flexible, minimal platform • Eg Mach [’87] • Eg L4 [’95] • seL4 [’09] – Mechanisms, not policies – Goes back to Nucleus [Brinch Hansen, CACM’70] Composite kernel Memory Objects does user-mode Low-level FS, scheduling Memory- Swapping mangmt Devices library Kernel memory Kernel memory Application Syscall Scheduling Scheduling Scheduling User IPC, MMU abstr. IPC, MMU abstr. IPC, MMU abstr. VFS Mode Unix File Server Device Server IPC, file system Application Driver • 180 syscalls • ~7 syscalls • ~3 syscalls Scheduler, virtual memory Kernel • 100 kLOC • ~10 kLOC • 9 kLOC Mode Device drivers, dispatcher IPC, virtual memory IPC • 100 µs IPC • ~ 1 µs IPC • 0.1 µs IPC Hardware Hardware • capabilities • design for isolation 3 COMP9242 S2/2016 W01 © 2016 Gernot Heiser. Distributed under CC Attribution License 4 COMP9242 S2/2016 W01 © 2016 Gernot Heiser. Distributed under CC Attribution License 2nd-Generation Microkernels L4: A Family of High-Performance Microkernels First L4 • 1st-generation kernels (Mach, Chorus) were a failure iOS security kernel with co-processor – Complex, inflexible, slow capabilities • L4 was first 2G microkernel [Liedtke, SOSP’93, SOSP’95] API Inheritance L4-embed. seL4 – Radical simplification & manual micro-optimisation Code Inheritance – “A concept is tolerated inside the microkernel only if moving it outside OKL4 Microvisor the kernel, i.e. permitting competing implementations, would prevent the implementation of the system’s required functionality.” L4/MIPS OKL4 µKernel – High IPC performance L4/Alpha Codezero Qualcomm • Family of L4 kernels: modem chips L3 → L4 “X” Hazelnut Pistachio – Original Liedtke (GMD) assembler kernel (‘95) – Family of kernels developed by Dresden, UNSW/NICTA, Karlsruhe Fiasco Fiasco.OC UNSW/NICTA – Commercial clones (PikeOS, P4, CodeZero, …) GMD/IBM/Karlsruhe NOVA – Influenced commercial QNX (‘82), Green Hills Integrity (‘90s) Dresden OK Labs P4 → PikeOS – Generated NICTA startup Open Kernel Labs (OK Labs) Commercial Clone o large-scale commercial deployment (multiple billions shipped) 93 94 95 96 97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13 5 COMP9242 S2/2016 W01 © 2016 Gernot Heiser. Distributed under CC Attribution License 6 COMP9242 S2/2016 W01 © 2016 Gernot Heiser. Distributed under CC Attribution License Issues of 2G Microkernels seL4 Principles • L4 solved performance issue [Härtig et al, SOSP’97] • Single protection mechanism: capabilities • Left a number of security issues unsolved – Proper time management to be finished this year • Problem: ad-hoc approach to protection and resource management • All resource-management policy at user level – Global thread name space covert channels [Shapiro’03] – Painful to use – Threads as IPC targets insufficient encapsulation – Need to provide standard memory-management library – Single kernel memory pool DoS attacks o Results in L4-like programming model – Insufficient delegation of authority limited flexibility, • Suitable for formal verification (proof of implementation correctness) performance – Attempted since ‘70s – Unprinciple management of time – Finally achieved by L4.verified project at NICTA [Klein et al, SOSP’09] • Addressed by seL4 – Designed to support safety- and security-critical systems – Principled time management not yet mainline (RT branch) 7 COMP9242 S2/2016 W01 © 2016 Gernot Heiser. Distributed under CC Attribution License 8 COMP9242 S2/2016 W01 © 2016 Gernot Heiser. Distributed under CC Attribution License seL4 Concepts Note: differences What are (Object) Capabilities? between AOS and mainline kernels! • Capabilities (Caps) Cap = Access Token: Prima-facie evidence – mediate access Eg. thread, • Kernel objects: of privilege file, … – Threads (thread-control blocks: TCBs) – Address spaces (page table objects: PDs, PTs) – Endpoints (IPC EPs, Notification AEPs) Obj reference – Capability spaces (CNodes) Access rights Object – Frames Eg. read, – Interrupt objects write, send, Cap typically in kernel to execute… – Untyped memory • OO API: protect from forgery • System calls err = method( cap, args ); Ø user references cap – Send, Wait (and variants) • Used in some earlier microkernels: through handle – Yield – KeyKOS [‘85], Mach [‘87], EROS [‘99] 9 COMP9242 S2/2016 W01 © 2016 Gernot Heiser. Distributed under CC Attribution License 10 COMP9242 S2/2016 W01 © 2016 Gernot Heiser. Distributed under CC Attribution License seL4 Capabilities Inter-Process Communication (IPC) • Stored in cap space (CSpace) • Fundamental microkernel operation – Kernel object made up of CNodes – Kernel provides no services, only mechanisms – each an array of cap “slots” – OS services provided by (protected) user-level server processes • Inaccessible to userland – invoked by IPC – But referred to by pointers into CSpace (slot addresses) – These CSpace addresses are called CPTRs Client Server • Caps convey specific privilege (access rights) IPC – Read, Write, Grant (cap transfer) Mainline has seL4 • Main operations on caps: Execute too – Invoke: perform operation on object referred to by cap • seL4 IPC uses a handshake through endpoints: o Possible operations depend on object type – Transfer points without storage capacity send receive – Copy/Mint/Grant: create copy of cap with same/lesser privilege – Message must be transferred instantly – Move/Mutate: transfer to different address with same/lesser privilege o Single-copy user user by kernel – Delete: invalidate slot (cleans up object if this is the only cap to it) – Revoke: delete any derived (eg. copied or minted) caps 11 COMP9242 S2/2016 W01 © 2016 Gernot Heiser. Distributed under CC Attribution License 12 COMP9242 S2/2016 W01 © 2016 Gernot Heiser. Distributed under CC Attribution License IPC: (Synchronous) Endpoints IPC Endpoints are Message Queues Thread1 Thread2 Server Running Blocked Blocked Running Client1 ….. Wait (ep1_cap, …) Client2 Send (ep1_cap, …) ….... Wait (ep2_cap, …) ….... ….... Send (ep2_cap, …) Kernel TCB TCB EP • Threads must rendez-vous for message transfer 1 2 – One side blocks until the other is ready – Implicit synchronisation • EP has no sense of direction • Message copied from sender’s to receiver’s message registers First invocation queues caller • May queue senders or receivers – Message is combination of caps and data words Further callers of – never both at the same time! o Presently max 121 words (484B, incl message “tag”) same direction queue behind o Should never use anywhere near that much! • Communication needs 2 EPs! 13 COMP9242 S2/2016 W01 © 2016 Gernot Heiser. Distributed under CC Attribution License 14 COMP9242 S2/2016 W01 © 2016 Gernot Heiser. Distributed under CC Attribution License Client-Server Communication Call RPC Semantics • Asymmetric relationship: Client Server – Server widely accessible, clients not Server Client1 Client2 – How can server reply back to client (distinguish between them)? Client Kernel Server • Client can pass (session) reply cap in first request Wait(ep,&rep) – server needs to maintain session state Call(ep,…) – forces stateful server design mint rep deliver to server • seL4 solution: Kernel provides single-use reply cap process – only for Call operation (Send+Wait) Send(rep,…) – allows server to reply to client deliver to client – cannot be copied/minted/re-used but can be moved destroy rep – one-shot (automatically destroyed after first use) process process 15 COMP9242 S2/2016 W01 © 2016 Gernot Heiser. Distributed under CC Attribution License 16 COMP9242 S2/2016 W01 © 2016 Gernot Heiser. Distributed under CC Attribution License Identifying Clients IPC Mechanics: Virtual Registers Stateful server serving multiple clients • Like physical registers, virtual registers are thread state • Must respond to correct client – context-switched by kernel Server – Ensured by reply cap Client1 – implemented as physical registers or thread-local memory Client 1 • Message registers • Must associate request state Client2 – contain message transferred in IPC with correct state Client2 state – architecture-dependent subset mapped to physical registers • Could use separate EP per client o 5 on ARM, 3 on x86 – endpoints are lightweight (16 B) – library interface hides details – but requires mechanism to wait on a set of EPs (like select) o 1st transferred word is special, contains message tag • Instead, seL4 allows to individually mark (“badge”) caps to same EP – API MR[0] refers to next word (not the tag!) – server provides individually badged caps to clients • Reply cap – server tags client state with badge (through Mint()) – overwritten by next receive! – kernel delivers badge to receiver on invocation of badged caps – can move to CSpace with cspace_save_reply_cap() 17 COMP9242 S2/2016 W01 © 2016 Gernot Heiser. Distributed under CC Attribution