Evolving Mach 3.0 to a Migrating Thread Model
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Extensible Distributed Operating System for Reliable Control Systems
194 Extensible Distributed Operating System for Reliable Control Systems Katsumi Maruyama, Kazuya Kodama, Soichiro Hidaka, Hiromichi Hashizume National Institute of Informatics, 2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo, Japan Email:{maruyama,kazuya,hidaka,has}@nii.ac.jp Abstract small monitor-like OSs are used. However, these monitor- like OSs lack program protection mechanisms, and program Since most control systems software is hardware-related, development is difficult. real-time-oriented and complex, adaptable OSs which help Therefore, an extensible/adaptable OS for control sys- program productivity and maintainability improvement are tems is required . We are developing a new OS character- in strong demand. ized by: We are developing an adaptable and extensible OS based on micro-kernel and multi-server scheme: each server runs • Use of an efficient and flexible micro-kernel (L4-ka). • Multi-server based modular OS. (Each OS service is in a protected mode interacting only via messages, and implemented as individual user-level process.) could be added/extended/deleted easily. Since this OS is • Robustness. Only the micro-kernel runs in kernel highly modularized, inter-process messaging overhead is a mode and in kernel space. Other modules run in a pro- concern. Our implementation proved good efficiency and tected user space and mode. maintainability. • Hardware driver programs in user-level process. • Flexible distributed processing by global message passing. 1. Introduction This OS structure proved to enhance OS modularity and ease of programming. However, inter-process messaging Most systems, from large scale public telephone switch- overhead should be considered . We measured the overhead, ing systems to home electronics, are controlled by software, and the overhead was proved to be small enough. -
Moving FLASK to BSD Systems
MovingMoving FLASKFLASK toto BSDBSD SSystemsystems SELinux Symposium 2006 Chris Vance Information Systems Security Operation, SPARTA, Inc. Vance_20060301_01 Overview • Security Frameworks • A Brief History • SELinux Inspired… – Security Enhanced BSD – Security Enhanced Darwin Vance_20060301_02 Security Frameworks • Traditional UNIX security isn’t enough – OS hardening – Mandatory protection – Flexible, manageable, scalable protection • Support required in the operating system for new security services – Costs of locally maintaining security extensions are high – Framework offers extensibility so that policies may be enhanced without changing the base operating system • There does not appear to be one perfect security model or policy – Sites may have different security/performance trade-offs – Sites may have special local requirements – Vendors unlikely to adopt a single policy • Bottom Line: Frameworks for Linux, FreeBSD, Darwin Vance_20060301_03 How We Got Here… Vance_20060301_04 Focus and Reuse • Don’t “reinvent” security – Learn from the past – The research is often decades old – A good design is durable, doesn’t require constant change – FLASK hasn’t changed much recently, what’s that mean? • Leverage existing technology, focus on “new” issues – Focus on improving operating system security – Spend the time to get Frameworks correct – Work with vendor for acceptance – Develop rule sets that work – Develop effective tools to manage policy • Do innovate – Don’t stop thinking! – Don’t squash new ideas – Re-factor old ideas Vance_20060301_05 -
Operating System Structure
Operating System Structure Joey Echeverria [email protected] modified by: Matthew Brewer [email protected] Nov 15, 2006 Carnegie Mellon University: 15-410 Fall 2006 Overview • Motivations • Kernel Structures – Monolithic Kernels ∗ Kernel Extensions – Open Systems – Microkernels – Exokernels – More Microkernels • Final Thoughts Carnegie Mellon University: 15-410 Fall 2006 1 Motivations • Operating systems have a hard job. • Operating systems are: – Hardware Multiplexers – Abstraction layers – Protection boundaries – Complicated Carnegie Mellon University: 15-410 Fall 2006 2 Motivations • Hardware Multiplexer – Each process sees a “computer” as if it were alone – Requires allocation and multiplexing of: ∗ Memory ∗ Disk ∗ CPU ∗ IO in general (network, graphics, keyboard etc.) • If OS is multiplexing it must also allocate – Priorities, Classes? - HARD problems!!! Carnegie Mellon University: 15-410 Fall 2006 3 Motivations • Abstraction Layer – Presents “simple”, “uniform” interface to hardware – Applications see a well defined interface (system calls) ∗ Block Device (hard drive, flash card, network mount, USB drive) ∗ CD drive (SCSI, IDE) ∗ tty (teletype, serial terminal, virtual terminal) ∗ filesystem (ext2-4, reiserfs, UFS, FFS, NFS, AFS, JFFS2, CRAMFS) ∗ network stack (TCP/IP abstraction) Carnegie Mellon University: 15-410 Fall 2006 4 Motivations • Protection Boundaries – Protect processes from each other – Protect crucial services (like the kernel) from process – Note: Everyone trusts the kernel • Complicated – See Project 3 :) – Full -
Microkernel Construction Introduction
Microkernel Construction Introduction Nils Asmussen 04/06/2017 1 / 28 Outline Introduction Goals Administration Monolithic vs. Microkernel Overview About L4/NOVA 2 / 28 Goals 1 Provide deeper understanding of OS mechanisms 2 Look at the implementation details of microkernels 3 Make you become enthusiastic microkernel hackers 4 Propaganda for OS research at TU Dresden 3 / 28 Administration Thursday, 4th DS, 2 SWS Slides: www.tudos.org ! Teaching ! Microkernel Construction Subscribe to our mailing list: www.tudos.org/mailman/listinfo/mkc2017 In winter term: Microkernel-based operating systems (MOS) Various labs 4 / 28 Outline Introduction Monolithic vs. Microkernel Kernel design comparison Examples for microkernel-based systems Vision vs. Reality Challenges Overview About L4/NOVA 5 / 28 Monolithic Kernel System Design u s Application Application Application e r k Kernel e r File Network n e Systems Stacks l m Memory Process o Drivers Management Management d e Hardware 6 / 28 Monolithic Kernel OS (Propaganda) System components run in privileged mode No protection between system components Faulty driver can crash the whole system Malicious app could exploit bug in faulty driver More than 2=3 of today's OS code are drivers No need for good system design Direct access to data structures Undocumented and frequently changing interfaces Big and inflexible Difficult to replace system components Difficult to understand and maintain Why something different? ! Increasingly difficult to manage growing OS complexity 7 / 28 Microkernel System Design Application -
Chapter 2 Operating System Structures
Operating Systems Associate Prof. Yongkun Li 中科大-计算机学院 副教授 http://staff.ustc.edu.cn/~ykli Chapter 2 Operating System Structures 1 Objectives • Operating System Services – User Operating System Interface – System Calls • Operating System Structure • Operating System Design and Implementation • MISC: Debugging, Generation & System Boot 2 Operating System Services Services Overview, User Interface 3 Operating System Services • Operating systems provide – an environment for execution of programs and – services to programs and users • Services may differ from one OS to another • What are the common classes? – Convenience of the user – Efficiency of the system 4 Overview of Operating System Services 5 OS Services for Helping Users • User interface - Almost all operating systems have a user interface (UI). – Three forms • Command-Line (CLI) – Shell command • Batch – Shell script • Graphics User Interface (GUI) – Windows system 6 OS Services for Helping Users • Program execution – Load a program into memory – Run the program – End execution • either normally or • abnormally (indicating error) 7 OS Services for Helping Users • I/O operations - A running program may require I/O, which may involve a file or an I/O device – Common I/Os: read, write, etc. – Special functions: recording CD/DVD • Notes: Users usually cannot control I/O devices directly, so OS provides a mean to do I/O – Mainly for efficiency and protection 8 OS Services for Helping Users • File-system manipulation - The file system is of particular interest – OS provides a variety of file systems • Major services – read and write files and directories – create and delete files and directories – search for a given file – list file Information – permission management: allow/deny access 9 OS Services for Helping Users • Communications: information exchange between processes – Processes on the same computer – Processes between computers over a network • Implementations – Shared memory • Two or more processes read/write to a shared section of mem. -
Darwin: Mac OS X's Core OS
ADC April 2001 3/8/01 1:57 PM Page 1 Apple Developer Connection Direct Darwin: Mac OS X’s Core OS eneath Mac OS X’s user-friendly and attractive user interface, Most of the reference documents can be Aqua, and the application frameworks (Classic, Carbon and found in the /Developer/Documentation/ BCocoa) is Darwin: Mac OS X’s core OS. Unseen by users, Kernel directory on any Mac OS X system Darwin provides a strong yet flexible foundation with features like with the Mac OS X Developer Tools package preemptive multitasking, protected memory and real-time support installed. that make Mac OS X a truly modern operating system. The focus of this article is to provide a brief overview of Components of Darwin Darwin and its components as well as give an introduction to Just like in the old Reese’s Peanut Butter developing kernel extensions—modules that extend Darwin’s Cups commercials (“You’ve got chocolate in functionality. For more in-depth information, you should read my peanut butter… No, you’ve got peanut butter on my choco- Inside Mac OS X: Kernel Environment which is available, along late!”), Darwin blends a mixture of mature industry standard com- with other documents referred to in this article, on the Apple ponents such as Mach and BSD with Apple-engineered components Developer Connection (ADC) web site in the Mac OS X to provide Mac OS X with a stable, reliable and extensible founda- Documentation section: tion. Darwin consists of five main components: Mach, I/O Kit, File http://developer.apple.com/techpubs/macosx/macosx.html System, Networking and BSD. -
L4 – Virtualization and Beyond
L4 – Virtualization and Beyond Hermann Härtig!," Michael Roitzsch! Adam Lackorzynski" Björn Döbel" Alexander Böttcher! #!Department of Computer Science# "GWT-TUD GmbH # Technische Universität Dresden# 01187 Dresden, Germany # 01062 Dresden, Germany {haertig,mroi,adam,doebel,boettcher}@os.inf.tu-dresden.de Abstract Mac OS X environment, using full hardware After being introduced by IBM in the 1960s, acceleration by the GPU. Virtual machines are virtualization has experienced a renaissance in used to test potentially malicious software recent years. It has become a major industry trend without risking the host environment and they in the server context and is also popular on help developers in debugging crashes with back- consumer desktops. In addition to the well-known in-time execution. benefits of server consolidation and legacy In the server world, virtual machines are used to preservation, virtualization is now considered in consolidate multiple services previously located embedded systems. In this paper, we want to look on dedicated machines. Running them within beyond the term to evaluate advantages and virtual machines on one physical server eases downsides of various virtualization approaches. management and helps saving power by We show how virtualization can be increasing utilization. In server farms, migration complemented or even superseded by modern of virtual machines between servers is used to operating system paradigms. Using L4 as the balance load with the potential of shutting down basis for virtualization and as an advanced completely unloaded servers or adding more to microkernel provides a best-of-both-worlds increase capacity. Lastly, virtual machines also combination. Microkernels can contribute proven isolate different customers who can purchase real-time capabilities and small trusted computing virtual shares of a physical server in a data center. -
L4 Microkernel
L4 Microkernel See final slide for sources Presented by Michael LeMay L4 Advantages ● Recursive construction of memory spaces – Allows construction of memory managers in userspace that just use memory mapping – Kernel provides map, grant, and flush ● Blazing fast IPC – Passes short messages in registers – Avoids copying large messages and thus avoids TLB flushes and cache misses – Lazy scheduling of thread queues improves small message IPC around 25% L4 Performance OS Microseconds Instructions Mach 115 1150 L4 5 50 Exokernel 1.4 30 SPIN 102 1100 (Exokernel runs on MIPS R2000, which operates more efficiently than the x86 for L4, and has a tagged TLB. It also does not provide a portable API.) L4 Conceptual Contributions ● The author of the L4 paper, Jochen Liedtke, showed that microkernels do not inherently: – Exhibit slow IPC – Impose significant processor overhead ● Instead, he showed that particular implementations are to blame: – Mach was causing lots of cache misses because it's fat, making it look like microkernels have high overhead Microkernel Non-portability ● Liedtke argues that microkernels should be non- portable, but present a uniform API: – Abstract microkernels require additional hardware abstraction layer – Can't take advantage of specific performance-enhancing hardware features – Can't protect against peculiar hardware performance pitfalls L4 Subprojects http://www.dakotacountyswcd.org/treetype.htm ● L3 – Predecessor to L4 ● FIASCO – C++ implementation of L4, poor performer in comparison to “Pip” series ● L4Ka – Most active -
The L4 Ecosystem
Faculty of Computer Science, Operating Systems Group The L4 ecosystem Berlin, Dec 2004 System Issues ● Convential systems are ● Complex ● Linux kernel at least 500,000 LoC ● Prone to errors ● Drivers ● All system components run in privileged mode ● Inflexible ● Global policies ● Large Trusted Computing Base (TCB) 27.12.04 Adam Lackorzynski, Michael Peter Folie 2 Insights Observation: Most kernel functionality does not need CPU privileges, like: – Filesystems – Driver functionality – User management 27.12.04 Adam Lackorzynski, Michael Peter Folie 3 What is really needed Jochen Liedtke: “A microkernel does no real work” Kernel provides only inevitable mechanisms No policies enforced by the kernel What is inevitable? Abstractions Mechanisms . Threads . Communication . Scheduling . Address Spaces . Safe construction This should be sufficient for everything 27.12.04 Adam Lackorzynski, Michael Peter Folie 4 The Marred Perception of Microkernels • Supposed to be slow – Not true any more • No obvious benefit – Infamous dispute Torvalds vs. Tannenbaum – How much worth is manageability of complexity? • GNU Hurd – Late – Slow – Constantly lagging behind other OS in functionality 27.12.04 Adam Lackorzynski, Michael Peter Folie 5 The Case for Microkernels • Complexity needs to be handled – Structure beyond monolithic kernels are needed – Several candidates • Virtualisation • Paravirtualisation • Microkernel • Implementation of some functionality is even simplified – Real time • DROPS • RTLinux – Security • Substantially smaller trusted computing -
Rust and IPC on L4re”
Rust and Inter-Process Communication (IPC) on L4Re Implementing a Safe and Efficient IPC Abstraction Name Sebastian Humenda Supervisor Dr. Carsten Weinhold Supervising Professor Prof. Dr. rer. nat. Hermann Härtig Published at Technische Universität Dresden, Germany Date 11th of May 2019 Contents 1 Introduction 5 2 Fundamentals 8 2.1 L4Re . 8 2.2 Rust....................................... 11 2.2.1 Language Basics . 11 2.2.2 Ecosystem and Build Tools . 14 2.2.3 Macros . 15 3 Related Work 17 3.1 Rust RPC Libraries . 17 3.2 Rust on Other Microkernels . 18 3.3 L4Re IPC in Other Languages . 20 3.4 Discussion . 21 3.4.1 Remote Procedure Call Libraries . 21 3.4.2 IDL-based interface definition . 22 3.4.3 L4Re IPC Interfaces In Other Programming Languages . 23 4 L4Re and Rust Infrastructure Adaptation 24 4.1 Build System Adaptation . 25 4.1.1 Libraries . 26 4.1.2 Applications . 27 4.2 L4rust Libraries . 29 4.2.1 L4 Library Split . 29 4.2.2 Inlining vs. Reimplementing . 30 4.3 Rust Abstractions . 31 4.3.1 Error Handling . 31 4.3.2 Capabilities . 32 5 IPC Framework Implementation 35 5.1 A Brief Overview of the C++ Framework . 36 5.2 Rust Interface Definition . 37 5.2.1 Channel-based Communication . 37 5.2.2 Macro-based Interface Definition . 39 5.3 Data Serialisation . 43 2 5.4 Server Loop . 46 5.4.1 Vector-based Service Registration . 46 5.4.2 Pointer-based Service Registration . 48 5.5 Interface Export . 52 5.6 Implementation Tests . -
Mach-O Internals
Mach-O Internals William Woodru February 10, 2016 1 / 31 General Agenda 1. Who are you? 2. What is Mach-O? 3. An Extremely Brief History of Mach and Mach-O 4. Structure of a Mach-O File 5. Quirks Encountered 6. Concluding Notes 2 / 31 Who are you? My name is William Woodru. I'm a Computer Science major and Philosophy minor at the University of Maryland, College Park. Outside of school, I'm a member of the Homebrew project and a regular contributor to several open source groups. My work for Homebrew is largely concerned with the underlying system interface and reconciling OS X's intricacies/irregularities with the package manager. 3 / 31 What is Mach-O? Mach-O is the Mach Object binary format. Mach-O is used primarily by Apple in OS X and iOS. Apps on both platforms are really just directory trees containing Mach-O binaries and resources (fonts, icons, congurations). Metadata is stored in a number of places, but mainly within bundled plists (XML) and the binaries themselves. Like its cousins on Linux (ELF) and Windows (PE), Mach-O supports multiple object types: I Executable I Core dump I Shared library/object I Prelinked object le I etc. 4 / 31 . and multiple architectures: I m68k/m88k (yes, it's still supported!*) I x86 I AMD64 I POWER I ARMv6/7/8 Unlike ELF or PE, Mach-O has been extended to allow multi-architecture fat binaries. This has resulted in some interesting properties not shared by the other two. More on that later. -
KOS - Principles, Design, and Implementation
KOS - Principles, Design, and Implementation Martin Karsten, University of Waterloo Work In Progress - September 29, 2015 Abstract KOS is an experimental operating system kernel that is designed to be simple and accessible to serve as a platform for research, experimen- tation, and teaching. The overall focus of this project is on system-level infrastructure software, in particular runtime systems. 1 Introduction KOS (pronounce "Chaos") is an experimental operating system kernel that is designed to be simple and accessible to serve as a platform for research, ex- perimentation, and teaching. It is focused on building a nucleus kernel that provides the basic set of operating system functionality, primarily for resource mediation, that must realistically be implemented for execution in privileged mode. While being simple KOS is not simplistic and avoids taking design shortcuts that would prohibit adding more sophisticated resource management strategies later on { inside the kernel or at higher software layers. The nu- cleus kernel is augmented with several prototype subsystems typically found in an operating system to eventually support running realistic applications. The entire code base is written in C++, except for small assembler parts. The C++ code makes use of advanced language features, such as code reuse with efficient strong type safety using templates, the C++ library for data struc- ture reuse, as well as (limited) polymorphism. Existing open-source software is reused for non-nucleus subsystems as much as possible. The project is hosted at https://git.uwaterloo.ca/mkarsten/KOS 2 Motivation In principle, an operating system has two basic functions. First, it consolidates low-level hardware interfaces and provides higher-level software abstractions to facilitate and alleviate application programming.