MINIX3: A Reliable and Secure Operating System Andrew S. Tanenbaum and a team of students and programmers who actually did all the work Vrije Universiteit Amsterdam, The Netherlands 1 GOAL OF OUR WORK: BUILD A RELIABLE OS Tanenbaum’s definition of a reliable OS: “An operating system is said to be reliable when a typical user has never experienced even a single failure in his or her lifetime and does not know anybody who has ever experienced a failure.” In engineering terms, this is probably mean time to failure > 50 years I don’t think we are there yet 2 THE TELEVISION MODEL 1. You buy the television 2. You plug it in 3. It works perfectly for the next 10 years 3 THE COMPUTER MODEL (WINDOWS EDITION) 1. You buy the computer 2. You plug it in 3. You install service packs 1 through 9f 4. You install 18 new emergency security patches 5. You find and install 7 new device drivers 6. You install antivirus software 7. You install antispyware software 8. You install antihacker software (firewall) 9. You install antispam software 10. You reboot the computer 4 THE COMPUTER MODEL (2) 11. It doesn’t work 12. You call the helpdesk 13. You wait on hold for 30 minutes 14. They tell you to reinstall Windows 5 TYPICAL USER REACTION The New York Times recently reported that 25% of computer users have gotten so angry at their computer that they physically hit it. 6 IS RELIABILITY SO IMPORTANT? • Annoying • Lost work • But also think about – Industrial control systems in factories – Power grids – Hospital operating rooms – Banking and e-commerce servers – Emergency phone centers – Control software in cars, airplanes, etc. 7 IS THIS FEASIBLE? • We won’t find out if we don’t try • Dutch Royal Academy gave me €2 million to try • European Union gave me €2.5 million to give it a shot • So, we’re trying 8 IS RELIABILITY ACHIEVABLE AT ALL? • Systems can survive hardware failures! – RAIDs can survive failed disks – ECC memory can survive parity errors in memory – TCP/IP can survive lost packets – CD-ROM drives can correct many simultaneous errors • We need to be able to survive software failures, too 9 A NEED TO RETHINK OPERATING SYSTEMS • Operating systems research need to be refocused – We have nearly infinite hardware on PC-class machines – Plenty of CPU cycles, RAM, bandwidth – Current software has tons of (useless) features – Consequently, the software is slow, bloated, and buggy • To achieve the TV model, future OSes, must be – Small – Simple – Modular – Reliable – Secure – Self-healing 10 BRIEF HISTORY OF OUR WORK • (1976) John Lions wrote a book on UNIX V6 • (1979) AT&T released V7 and forbade books on it L • (1985) I started to write a UNIX-like OS from scratch • (1987) MINIX 1 + book for teaching OS classes released • (1997) MINIX 2 (POSIX) & 2nd edition of book released • (2000) MINIX 2 license changed to BSD • (2004) MINIX 3: start of work making a reliable OS • (2006) 3rd edition of book • (2008) European grant • (2010) Focus moved towards embedded systems • (2013) MINIX 3.3.0 moves to NetBSD “compatibility” 11 THREE EDITIONS OF THE BOOK 1 2 3 12 INTELLIGENT DESIGN AS APPLIED TO OPERATING SYSTEMS • Microkernel (15,000 LoC vs. > 15 million for Linux) – Bugs per 1000 LoC: Most S/W (1-10) – MINIX 3 at least 15 kernel bugs; Linux has > 15,000 – Drivers have 3-7x more bugs than rest of kernel – About 70% of the code is drivers • Highly modular • OS runs as multiple user-mode server processes 13 STEP 1: ISOLATE COMPONENTS • Move all loadable modules out of the kernel – includes all device drivers and file systems • Run each module as a separate process with POLA (Principle Of Least Authority) 14 STEP 2: ISOLATE I/O • Isolate I/O devices • Limit access to I/O ports • Constrain DMA (needs hardware assistance) 15 STEP 3: ISOLATE COMMUNICATION • Limit interprocess communication • Restrict kernel calls on a per component basis • Restrict IPC on a ‘need-to-communicate’ basis • Make sure faulty receiver cannot hang sender 16 ARCHITECTURE OF MINIX 3 Process Shell Make ... User User mode FS 1 FS 2 Proc. ... Other Servers Disk TTY Net Print ... Other Drivers Kernel Microkernel handles interrupts, mode processes, scheduling, IPC 17 USER-MODE DEVICE DRIVERS • Each driver runs as a user-mode process • No superuser privileges • Protected by the MMU • Do not have access to I/O ports, privileged instrs 18 USER-MODE SERVERS • Each server runs as a separate process • Some key servers – Virtual file server – Actual file servers – Process manager – Memory manager – Network server – Reincarnation server 19 A SIMPLIFIED EXAMPLE: DOING A READ User Users 1 User 4 mode FS Servers Disk 2 3 Drivers Kernel File access when the block is in the FS cache 20 FILE SERVER (2) User Users 1 User 9 mode FS Servers 2 6 7,8 5 Disk Drivers 3 4 Notification Kernel File access when the block is NOT in the FS cache 21 REINCARNATION SERVER • Parent of all the drivers and servers • When a driver or server dies, RS collects it • RS checks a table for action to take e.g., restart it • RS also pings drivers and servers frequently 22 DISK DRIVER RECOVERY RS 1 User Users User 5 4 FS Servers mode 2 New Disk driver driverX 3. Crash! Drivers Kernel System is self healing—this is how we hope to make it reliable 23 KERNEL RELIABILITY/SECURITY • Fewer LoC means fewer kernel bugs • Small kernel (15,000 LoC) means reduced TCB • NO foreign code (e.g., drivers) in the kernel • Static data structures (no malloc in kernel) • Moving bugs to user space reduces their power 24 IPC RELIABILITY/SECURITY • Fixed-length messages (no buffer overruns) • Rendezvous system was simple – No lost messages – No buffer management – We had to add asynchronous messages • Interrupts and messages are unified 25 DRIVER RELIABILITY/SECURITY • Untrusted code: heavily isolated • Bugs, viruses cannot spread to other modules • Cannot touch kernel data structures • Bad pointers crash only one driver; recoverable • Infinite loops detected and driver restarted • Restricted power to do damage (not superuser) 26 OTHER ADVANTAGES OF USER DRIVERS • Short development cycle • Normal programming model • No down time for crash and reboot • Easy debugging • Good flexibility 27 FAULT INJECTION EXPERIMENT • We injected 800,000 faults into each of 3 drivers • Done on the binary drivers • Examples, change src addr, dest addr, loop condition • 100 faults were injected on each experiment • Waited 1 sec to see if the driver crashed • If no crash, inject another 100 faults and repeat • The driver crashed in 18,038 trials • The operating system NEVER crashed 28 PORT OF MINIX 3 TO ARM • Restructured source tree for multiple architectures • Changed booting to support uboot for ARM • Rewrote the low-level code dealing with hardware • Changed code for context switching, paging, etc. • Removed x86 segmentation code • Imported NetBSD ARM headers and libraries • Ported build.sh for cross-toolchain support • Wrote drivers for SD card and other Beagle devices 29 EMBEDDED SYSTEMS BeagleBone Black 5 cm 9 cm 30 CHARACTERISTICS Item Beaglebone Black 31 CHARACTERISTICS Item Beaglebone Black CPU ARM v7 32 CHARACTERISTICS Item Beaglebone Black CPU ARM v7 Clock 1 GHz 33 CHARACTERISTICS Item Beaglebone Black CPU ARM v7 Clock 1 GHz RAM 512 MB 34 CHARACTERISTICS Item Beaglebone Black CPU ARM v7 Clock 1 GHz RAM 512 MB Flash 4 GB 35 CHARACTERISTICS Item Beaglebone Black CPU ARM v7 Clock 1 GHz RAM 512 MB Flash 4 GB Video HDMI/1080p 36 CHARACTERISTICS Item Beaglebone Black CPU ARM v7 Clock 1 GHz RAM 512 MB Flash 4 GB Video HDMI/1080p GPIO pins 92 37 CHARACTERISTICS Item Beaglebone Black CPU ARM v7 Clock 1 GHz RAM 512 MB Flash 4 GB Video HDMI/1080p GPIO pins 92 Ethernet 10/100 Mbps 38 CHARACTERISTICS Item Beaglebone Black CPU ARM v7 Clock 1 GHz RAM 512 MB Flash 4 GB Video HDMI/1080p GPIO pins 92 Ethernet 10/100 Mbps USB 1 39 CHARACTERISTICS Item Beaglebone Black CPU ARM v7 Clock 1 GHz RAM 512 MB Flash 4 GB Video HDMI/1080p GPIO pins 92 Ethernet 10/100 Mbps USB 1 Open source Yes 40 CHARACTERISTICS Item Beaglebone Black CPU ARM v7 Clock 1 GHz RAM 512 MB Flash 4 GB Video HDMI/1080p GPIO pins 92 Ethernet 10/100 Mbps USB 1 Open source Yes Price (quantity 1) $45 41 CHARACTERISTICS Item Beaglebone Black Raspberry Pi B+ CPU ARM v7 Clock 1 GHz RAM 512 MB Flash 4 GB Video HDMI/1080p GPIO pins 92 Ethernet 10/100 Mbps USB 1 Open source Yes Price (quantity 1) $45 42 CHARACTERISTICS Item Beaglebone Black Raspberry Pi B+ CPU ARM v7 ARM v6 Clock 1 GHz RAM 512 MB Flash 4 GB Video HDMI/1080p GPIO pins 92 Ethernet 10/100 Mbps USB 1 Open source Yes Price (quantity 1) $45 43 CHARACTERISTICS Item Beaglebone Black Raspberry Pi B+ CPU ARM v7 ARM v6 Clock 1 GHz 700 MHz RAM 512 MB Flash 4 GB Video HDMI/1080p GPIO pins 92 Ethernet 10/100 Mbps USB 1 Open source Yes Price (quantity 1) $45 44 CHARACTERISTICS Item Beaglebone Black Raspberry Pi B+ CPU ARM v7 ARM v6 Clock 1 GHz 700 MHz RAM 512 MB 512 MB Flash 4 GB Video HDMI/1080p GPIO pins 92 Ethernet 10/100 Mbps USB 1 Open source Yes Price (quantity 1) $45 45 CHARACTERISTICS Item Beaglebone Black Raspberry Pi B+ CPU ARM v7 ARM v6 Clock 1 GHz 700 MHz RAM 512 MB 512 MB Flash 4 GB None Video HDMI/1080p GPIO pins 92 Ethernet 10/100 Mbps USB 1 Open source Yes Price (quantity 1) $45 46 CHARACTERISTICS Item Beaglebone Black Raspberry Pi B+ CPU ARM v7 ARM v6 Clock 1 GHz 700 MHz RAM 512 MB 512 MB Flash 4 GB None Video HDMI/1080p HDMI/1080p GPIO pins 92 Ethernet 10/100 Mbps USB 1 Open source Yes Price (quantity 1) $45 47 CHARACTERISTICS Item Beaglebone Black Raspberry Pi B+ CPU ARM v7 ARM v6 Clock 1 GHz 700 MHz RAM 512 MB 512 MB Flash 4 GB None Video HDMI/1080p HDMI/1080p GPIO