General-purpose computing with VirtualBox on Genode/NOVA Norman Feske <[email protected]> Outline 1. VirtualBox 2. NOVA microhypervisor and Genode 3. Transplantation of VirtualBox to NOVA 4. Demo 5. War stories 6. Project Turmvilla 7. The Book “Genode Foundations” General-purpose computing with VirtualBox on Genode/NOVA2 Outline 1. VirtualBox 2. NOVA microhypervisor and Genode 3. Transplantation of VirtualBox to NOVA 4. Demo 5. War stories 6. Project Turmvilla 7. The Book “Genode Foundations” General-purpose computing with VirtualBox on Genode/NOVA3 Architecture overview config, status SVC VM xpcom VM process xpcom process IPCD xpcom xpcom VBoxManage VirtualBox Application /dev/vboxdrv /dev/vboxdrv General-purpose computing with VirtualBox on Genode/NOVA4 Starting up a VM process VM process open /dev/vboxdrv kernel vboxdrv.ko General-purpose computing with VirtualBox on Genode/NOVA5 VM process running root mode non-root mode VM process load VMMR0 /dev/vboxdrv kernel vboxdrv.ko VMMR0 / Hypervisor General-purpose computing with VirtualBox on Genode/NOVA6 Entering the Guest OS root mode non-root mode VM process ioctrl VM RUN /dev/vboxdrv Guest OS kernel vboxdrv.ko world switch General-purpose computing with VirtualBox on Genode/NOVA7 Flow of a virtualization event root mode non-root mode VM process VM RUN returns /dev/vboxdrv Guest OS kernel vboxdrv.ko no yes VMMR0 ? world switch General-purpose computing with VirtualBox on Genode/NOVA8 root mode non-root mode VM process /dev/vboxdrv Guest OS kernel highly complex vboxdrv.ko VMMR0 / Hypervisor Risks for desktop virtualization General-purpose computing with VirtualBox on Genode/NOVA9 root mode non-root mode VM process /dev/vboxdrv Guest OS kernel highly complex vboxdrv.ko VMMR0 / Hypervisor Risks for desktop virtualization General-purpose computing with VirtualBox on Genode/NOVA9 Risks for desktop virtualization root mode non-root mode VM process access control? /dev/vboxdrv Guest OS kernel highly complex vboxdrv.ko VMMR0 / Hypervisor General-purpose computing with VirtualBox on Genode/NOVA 10 Risks for desktop virtualization root mode non-root mode authorized to change the kernel VM process highly complex access control? /dev/vboxdrv Guest OS kernel highly complex vboxdrv.ko VMMR0 / Hypervisor General-purpose computing with VirtualBox on Genode/NOVA 11 Outline 1. VirtualBox 2. NOVA microhypervisor and Genode 3. Transplantation of VirtualBox to NOVA 4. Demo 5. War stories 6. Project Turmvilla 7. The Book “Genode Foundations” General-purpose computing with VirtualBox on Genode/NOVA 12 NOVA architecture Guest OS Guest OS Guest OS non-root mode root mode VMM VMM VMM Resource management Apps Drivers 9,000 SLOC kernel NOVA Microhypervisor General-purpose computing with VirtualBox on Genode/NOVA 13 Flow of a virtualization event User-level VMM Guest OS UTCB VMCS UTCB copy NOVA world switch General-purpose computing with VirtualBox on Genode/NOVA 14 Genode OS architecture → Application-specific TCB General-purpose computing with VirtualBox on Genode/NOVA 15 Genode OS framework General-purpose computing with VirtualBox on Genode/NOVA 16 Genode combined with virtualization General-purpose computing with VirtualBox on Genode/NOVA 17 Seoul VMM on top of Genode/NOVA Unmodified Guest OS Kernel virtual virtual virtual CPU RAM device Resource Device VMM Multiplexer Driver Init Core User Mode NOVA Hypervisor Privileged Mode General-purpose computing with VirtualBox on Genode/NOVA 18 Idea Device models and features of VirtualBox + Security of the Genode/NOVA architecture General-purpose computing with VirtualBox on Genode/NOVA 19 Outline 1. VirtualBox 2. NOVA microhypervisor and Genode 3. Transplantation of VirtualBox to NOVA 4. Demo 5. War stories 6. Project Turmvilla 7. The Book “Genode Foundations” General-purpose computing with VirtualBox on Genode/NOVA 20 Identify the interesting parts Entire VirtualBox code base > 4 million lines of code (sloccount) Narrowed to the interesting parts > 2 million lines of code src/VBox/VMM src/recompiler src/VBox/Main src/libs/liblzf-3.4 src/VBox/Runtime src/libs/liblzf-3.4/cs src/VBox/Devices src/libs/libxml2-2.6.31 src/VBox/Storage src/libs/zlib-1.2.6 src/VBox/GuestHost include/VBox src/VBox/Disassembler include/iprt src/VBox/HostServices General-purpose computing with VirtualBox on Genode/NOVA 21 → Most parts of the POSIX runtime could be reused Porting the VirtualBox Runtime to Genode Facilitate Genode’s existing infrastructure I 3rd-party software management tools I FreeBSD libc I Standard C++ library I POSIX threads General-purpose computing with VirtualBox on Genode/NOVA 22 Porting the VirtualBox Runtime to Genode Facilitate Genode’s existing infrastructure I 3rd-party software management tools I FreeBSD libc I Standard C++ library I POSIX threads → Most parts of the POSIX runtime could be reused General-purpose computing with VirtualBox on Genode/NOVA 22 Guest memory (accessed by recompiler and device models) RAM, MMIO I/O-port handling PGM, HWACCM, TM Device models, PDM, BIOS Host drivers I Using the “Basic front end” I Reimplement SDLConsole interface VM process initialization Enable subsystems one by one General-purpose computing with VirtualBox on Genode/NOVA 23 I/O-port handling PGM, HWACCM, TM Device models, PDM, BIOS Host drivers I Using the “Basic front end” I Reimplement SDLConsole interface VM process initialization Enable subsystems one by one Guest memory (accessed by recompiler and device models) RAM, MMIO General-purpose computing with VirtualBox on Genode/NOVA 23 PGM, HWACCM, TM Device models, PDM, BIOS Host drivers I Using the “Basic front end” I Reimplement SDLConsole interface VM process initialization Enable subsystems one by one Guest memory (accessed by recompiler and device models) RAM, MMIO I/O-port handling General-purpose computing with VirtualBox on Genode/NOVA 23 Device models, PDM, BIOS Host drivers I Using the “Basic front end” I Reimplement SDLConsole interface VM process initialization Enable subsystems one by one Guest memory (accessed by recompiler and device models) RAM, MMIO I/O-port handling PGM, HWACCM, TM General-purpose computing with VirtualBox on Genode/NOVA 23 Host drivers I Using the “Basic front end” I Reimplement SDLConsole interface VM process initialization Enable subsystems one by one Guest memory (accessed by recompiler and device models) RAM, MMIO I/O-port handling PGM, HWACCM, TM Device models, PDM, BIOS General-purpose computing with VirtualBox on Genode/NOVA 23 VM process initialization Enable subsystems one by one Guest memory (accessed by recompiler and device models) RAM, MMIO I/O-port handling PGM, HWACCM, TM Device models, PDM, BIOS Host drivers I Using the “Basic front end” I Reimplement SDLConsole interface General-purpose computing with VirtualBox on Genode/NOVA 23 A look inside a VM process Recompiler Hardware Acceleration Execution Manager VM VM Instruction Exit Enter Emulator General-purpose computing with VirtualBox on Genode/NOVA 24 Start with executing the recompiler only Recompiler Hardware Acceleration Execution Manager Instruction Emulator General-purpose computing with VirtualBox on Genode/NOVA 25 Simple test scenario FB SDL VirtualBox Framebuffer Input ISO image Init ROM Core kernel Linux General-purpose computing with VirtualBox on Genode/NOVA 26 2. Small Linux-based images (Tinycore, GRML) 3. Windows XP Increasing guest complexity 1. Custom-made Genode OS scenarios General-purpose computing with VirtualBox on Genode/NOVA 27 3. Windows XP Increasing guest complexity 1. Custom-made Genode OS scenarios 2. Small Linux-based images (Tinycore, GRML) General-purpose computing with VirtualBox on Genode/NOVA 27 Increasing guest complexity 1. Custom-made Genode OS scenarios 2. Small Linux-based images (Tinycore, GRML) 3. Windows XP General-purpose computing with VirtualBox on Genode/NOVA 27 Windows XP as a guest FB SDL LX Proxy FS VirtualBox Framebuffer Input File system VDI image Init Core kernel Linux General-purpose computing with VirtualBox on Genode/NOVA 28 PS/2 driver VESA driver Rump FS AHCI driver VirtualBox Input Framebuffer File system Block VDI image Init Core kernel NOVA Move scenario to NOVA General-purpose computing with VirtualBox on Genode/NOVA 29 PS/2 driver VESA driver Rump FS AHCI driver VirtualBox Input Framebuffer File system Block VDI image Init Core kernel NOVA Move scenario to NOVA General-purpose computing with VirtualBox on Genode/NOVA 29 Entering non-root mode Recompiler Hardware Acceleration Execution Manager IRQs VM VM Instruction Exit Enter Emulator General-purpose computing with VirtualBox on Genode/NOVA 30 Virtualization of guest memory (EPT faults) Enter VT-x conservatively (if protected mode and paging enabled) Inject IRQs into recompiler Later: IRQ injection via NOVA into VT-X Entering non-root mode VBox VM state ↔ NOVA UTCB state General-purpose computing with VirtualBox on Genode/NOVA 31 Enter VT-x conservatively (if protected mode and paging enabled) Inject IRQs into recompiler Later: IRQ injection via NOVA into VT-X Entering non-root mode VBox VM state ↔ NOVA UTCB state Virtualization of guest memory (EPT faults) General-purpose computing with VirtualBox on Genode/NOVA 31 Inject IRQs into recompiler Later: IRQ injection via NOVA into VT-X Entering non-root mode VBox VM state ↔ NOVA UTCB state Virtualization of guest memory (EPT faults) Enter VT-x conservatively (if protected mode and paging enabled) General-purpose computing with VirtualBox on Genode/NOVA 31 Later: IRQ injection via NOVA into VT-X Entering non-root mode VBox VM state ↔ NOVA