DOCSLIB.ORG

1 Preliminaries 2 K42 Overview

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
1 Preliminaries 2 K42 Overview K42 CS380L: Mike Dahlin October 13, 2003 1 Preliminaries 1.1 Review 1.2 Outline • Overview • Scalability – Clustered objects – Existence locks – IPC • Discussion and open issues 1.3 Preview 2 K42 overview Takes lots of well-received research ideas from last 10+ years and builds an OS with them... • Exokernel/scheduler activations – User-level libraries approach • Mach-like virtual memory • Hydra, mach, L4 microkernel • Emulate old OS with library + server (Hydra, mach, exokernel, ...) • OS-kit device drivers (internal API) • Device driver virtualization (Disco?) • Local RPC (LRPC) for interprocess communication – LRPC + Clustered objects similar to RPC + {default sub, caching, ...} • ... 1 • Key ideas in design – Structure (from intro) ∗ Modular, object oriented ∗ Avoid centralized code paths, global data structures, global locks ∗ move system functionality from kernel to server processes and application libraries – Claimed benefits ∗ Easy prototyping of new OS technologies ∗ Scale up (to large multiprocessor and large applications), Scale down (to small multiprocessors), support small scale applications ∗ Backwards compatible with Linux but easy to add specialized components ∗ Customizable: Allow applications to determine how OS manages their resources ∗ Flexible: support wide range of applications and problem domains, support wide range of hardware, scale down to embedded HW and up to future enterprise server • These ideas (largely) taken from past systems (and adapted) – Kernel: memory management, process management, IPC, base scheduling – User-level libraries for abstractions (Linux API/ABI) (a la exokernel) ∗ Customizable ∗ Overhead is reduced – avoid crossing address space boundaries (e.g., setting timers that will be cleared before they go off) ∗ Space and time are consumed by calling application not by the kernel or servers – Implement key functionality as user-level servers (a la microkernel) ∗ Why is “Linux file server” a separate process rather than a library? – Memory management v. mach ∗ Compare K42 memory management to mach 2 3 Admin Next week: • Monday 10/20: Guest lecture – Emmett Witchel: ACES 2.402 3:30-4:30 • Wednesday 10/22: Midterm – in class – Bring one 8.5x11 inch page of notes (font ≥ 8pt) 4 K42 Scalability 4.1 Clustered objects • Multiprocessor is like a distributed system – Use RPC for communication ∗ LRPC – local remote procedure call (anderson) – RPC a good match for interprocess communication (fewer problems than remote RPC – performance, reliability, ...) – Optimize RPC for scalability (when needed) ∗ Default stub (usually) ∗ Cache/replicate (when needed) (see NFS, Globe) – Mechanism: ∗ Object reference – pointer to pointer to object representative ∗ Object representative – interface (global, shared, local) to global object (cache, replicate, or not) · Global pointer to same virtual address but (possibly) different physical address on different processors · Fill local table on demand with trap handler · e.g., to create a new clustered object, bump a pointer in per-process table and use next entry as Object reference; set it to NULL on each processor. Then, when a processor first references the object, go to global trap handler, see that it is a Clustered object reference, look at global table to see which clustered object and call its handler. It can then decide whether to instantiate a pointer to single global object representative, to object representative shared by 4 processors, or to create new object representative 4.2 Existence locks • Hairy problem in OO programming + concurrency – how to cleanly shut an object down – Need to wait until no one is using the object – Can’t put lock inside object (b/c deallocation is a race between deallocators and threads contending for that lock; e.g., what to do about waiting threads on that lock...) – Putting “existence lock” for object outside of object is complex ∗ Somewhat non-modular ∗ Need existence lock for existence lock, so on to global lock 3 ∗ “Tends to encourage holding of locks for a long time” • Solution: semi-automatic garbage collection – 3 phases 1. Remove all persistent references (e.g., references in global data structures) ∗ “just as you normally would when removing object” ∗ Now, what about threads that have references on their stacks? 2. Uniprocessor: Wait until zero threads active in kernel ∗ Then, no stack can have reference to object → garbage collect it ∗ Not provably live, but intuition is: “system server calls tend to be short...we do not believe this to be a problem in practice” ∗ Future work: keep several generations of “active counts” 3. Multiprocessor: Hand token to each processor and repeat step 2 for all processors 4.3 IPC • Interprocess communication: Must be fast (servers in different processes) • Optimize common case: same processor – process trap, kernel reti to callee – Callee receives caller’s registers, protected authenticator, (optional) shared memory page – Cheaper than context switch: no save registers, no scheduling – Maintains locality 5 Discussion and open issues What were contributions of • Multics • THE • Unix • Disco • Exokernel • Mesa/Threads/Monitors • Scheduling/Scheduler activations • RPC • Active messages • Virtual memory • K42 What are the solved problems in OS? What are the open problems (or at least not addressed so far) in OS? 4.
Load more

Recommended publications
  • A Microkernel API for Fine-Grained Decomposition
    A Microkernel API for Fine-Grained Decomposition Sebastian Reichelt Jan Stoess Frank Bellosa System Architecture Group, University of Karlsruhe, Germany freichelt,stoess,[email protected] ABSTRACT from the microkernel APIs in existence. The need, for in- Microkernel-based operating systems typically require spe- stance, to explicitly pass messages between servers, or the cial attention to issues that otherwise arise only in dis- need to set up threads and address spaces in every server for tributed systems. The resulting extra code degrades per- parallelism or protection require OS developers to adopt the formance and increases development effort, severely limiting mindset of a distributed-system programmer rather than to decomposition granularity. take advantage of their knowledge on traditional OS design. We present a new microkernel design that enables OS devel- Distributed-system paradigms, though well-understood and opers to decompose systems into very fine-grained servers. suited for physically (and, thus, coarsely) partitioned sys- We avoid the typical obstacles by defining servers as light- tems, present obstacles to the fine-grained decomposition weight, passive objects. We replace complex IPC mecha- required to exploit the benefits of microkernels: First, a nisms by a simple function-call approach, and our passive, lot of development effort must be spent into matching the module-like server model obviates the need to create threads OS structure to the architecture of the selected microkernel, in every server. Server code is compiled into small self- which also hinders porting existing code from monolithic sys- contained files, which can be loaded into the same address tems. Second, the more servers exist | a desired property space (for speed) or different address spaces (for safety).
    [Show full text]
  • Run-Time Reconfigurable RTOS for Reconfigurable Systems-On-Chip
    Run-time Reconfigurable RTOS for Reconfigurable Systems-on-Chip Dissertation A thesis submited to the Faculty of Computer Science, Electrical Engineering and Mathematics of the University of Paderborn and to the Graduate Program in Electrical Engineering (PPGEE) of the Federal University of Rio Grande do Sul in partial fulfillment of the requirements for the degree of Dr. rer. nat. and Dr. in Electrical Engineering Marcelo G¨otz November, 2007 Supervisors: Prof. Dr. rer. nat. Franz J. Rammig, University of Paderborn, Germany Prof. Dr.-Ing. Carlos E. Pereira, Federal University of Rio Grande do Sul, Brazil Public examination in Paderborn, Germany Additional members of examination committee: Prof. Dr. Marco Platzner Prof. Dr. Urlich R¨uckert Dr. Mario Porrmann Date: April 23, 2007 Public examination in Porto Alegre, Brazil Additional members of examination committee: Prof. Dr. Fl´avioRech Wagner Prof. Dr. Luigi Carro Prof. Dr. Altamiro Amadeu Susin Prof. Dr. Leandro Buss Becker Prof. Dr. Fernando Gehm Moraes Date: October 11, 2007 Acknowledgements This PhD thesis was carried out, in his great majority, at the Department of Computer Science, Electrical Engineering and Mathematics of the University of Paderborn, during my time as a fellow of the working group of Prof. Dr. Franz J. Rammig. Due to a formal agreement between Federal University of Rio Grande do Sul (UFRGS) and University of Paderborn, I had the opportunity to receive a bi-national doctoral degree. Therefore, I had to accomplished, in addition, the PhD Graduate Program in Electrical Engineering of UFRGS. So, I hereby acknowledge, without mention everyone personally, all persons involved in making this agreement possible.
    [Show full text]
  • Introduction
    1 INTRODUCTION A modern computer consists of one or more processors, some main memory, disks, printers, a keyboard, a mouse, a display, network interfaces, and various other input/output devices. All in all, a complex system.oo If every application pro- grammer had to understand how all these things work in detail, no code would ever get written. Furthermore, managing all these components and using them optimally is an exceedingly challenging job. For this reason, computers are equipped with a layer of software called the operating system, whose job is to provide user pro- grams with a better, simpler, cleaner, model of the computer and to handle manag- ing all the resources just mentioned. Operating systems are the subject of this book. Most readers will have had some experience with an operating system such as Windows, Linux, FreeBSD, or OS X, but appearances can be deceiving. The pro- gram that users interact with, usually called the shell when it is text based and the GUI (Graphical User Interface)—which is pronounced ‘‘gooey’’—when it uses icons, is actually not part of the operating system, although it uses the operating system to get its work done. A simple overview of the main components under discussion here is given in Fig. 1-1. Here we see the hardware at the bottom. The hardware consists of chips, boards, disks, a keyboard, a monitor, and similar physical objects. On top of the hardware is the software. Most computers have two modes of operation: kernel mode and user mode. The operating system, the most fundamental piece of soft- ware, runs in kernel mode (also called supervisor mode).
    [Show full text]
  • Microkernel Construction Introduction
    Microkernel Construction Introduction Nils Asmussen 04/06/2017 1 / 28 Outline Introduction Goals Administration Monolithic vs. Microkernel Overview About L4/NOVA 2 / 28 Goals 1 Provide deeper understanding of OS mechanisms 2 Look at the implementation details of microkernels 3 Make you become enthusiastic microkernel hackers 4 Propaganda for OS research at TU Dresden 3 / 28 Administration Thursday, 4th DS, 2 SWS Slides: www.tudos.org ! Teaching ! Microkernel Construction Subscribe to our mailing list: www.tudos.org/mailman/listinfo/mkc2017 In winter term: Microkernel-based operating systems (MOS) Various labs 4 / 28 Outline Introduction Monolithic vs. Microkernel Kernel design comparison Examples for microkernel-based systems Vision vs. Reality Challenges Overview About L4/NOVA 5 / 28 Monolithic Kernel System Design u s Application Application Application e r k Kernel e r File Network n e Systems Stacks l m Memory Process o Drivers Management Management d e Hardware 6 / 28 Monolithic Kernel OS (Propaganda) System components run in privileged mode No protection between system components Faulty driver can crash the whole system Malicious app could exploit bug in faulty driver More than 2=3 of today's OS code are drivers No need for good system design Direct access to data structures Undocumented and frequently changing interfaces Big and inflexible Difficult to replace system components Difficult to understand and maintain Why something different? ! Increasingly difficult to manage growing OS complexity 7 / 28 Microkernel System Design Application
    [Show full text]
  • Analysis of Existing Dynamic Software Updating Techniques for Safe and Secure Industrial Control Systems
    I. Mugarza, et al., Int. J. of Safety and Security Eng., Vol. 8, No. 1 (2018) 121–131 ANALYSIS OF EXISTING DYNAMIC SOFTWARE UPDATING TECHNIQUES FOR SAFE AND SECURE INDUSTRIAL CONTROL SYSTEMS IMANOL MUGARZA1, JORGE PARRA1 & EDUARDO JACOB2 1 IK4-Ikerlan Technology Research Centre, Dependable Embedded Systems Area. 2 Faculty of Engineering, University of the Basque Country UPV/EHU. ABSTRACT Higher interconnectivity among devices, machines, the cloud and humans is envisioned in the actual trend of automation, also known as Industrial Internet of Things (IIoT). These industrial control systems, which may require high availability and/or safety related capabilities, are no longer isolated from the corporate environment or Internet. Software updates will be needed during the product life cycle, due to the long service life, the increasing number of security related vulnerabilities discovered on these industrial control systems and the high interconnectivity desired in IIoT. These updates aim at fixing all these security weaknesses, bugs and vulnerabilities that could appear, while the required safety integrity levels are ensured. Security-related concerns have just been addressed by the safety engineering community, because of the increasing number of cyber-attacks against safety-critical systems, such as Stuxnet. Moreover, system shut-downs caused by software updates could not be plausible when high availability is required. Typically, in order to perform the software update, the whole industrial process or the production is halted, so that the software upgrade is safely applied. However, this scenario might not be applied in critical infrastructures, such as nuclear or hydro- electrical power plants, where these production and service interruptions are not acceptable from the business and service point of view.
    [Show full text]
  • Scalability of Microkernel-Based Systems
    Scalability of Microkernel-Based Systems Zur Erlangung des akademischen Grades eines DOKTORS DER INGENIERWISSENSCHAFTEN von der Fakultat¨ fur¨ Informatik der Universitat¨ Fridericiana zu Karlsruhe (TH) genehmigte DISSERTATION von Volkmar Uhlig aus Dresden Tag der mundlichen¨ Prufung:¨ 30.05.2005 Hauptreferent: Prof. Dr. rer. nat. Gerhard Goos Universitat¨ Fridericiana zu Karlsruhe (TH) Korreferent: Prof. Dr. sc. tech. (ETH) Gernot Heiser University of New South Wales, Sydney, Australia Karlsruhe: 15.06.2005 i Abstract Microkernel-based systems divide the operating system functionality into individ- ual and isolated components. The system components are subject to application- class protection and isolation. This structuring method has a number of benefits, such as fault isolation between system components, safe extensibility, co-existence of different policies, and isolation between mutually distrusting components. How- ever, such strict isolation limits the information flow between subsystems including information that is essential for performance and scalability in multiprocessor sys- tems. Semantically richer kernel abstractions scale at the cost of generality and mini- mality–two desired properties of a microkernel. I propose an architecture that al- lows for dynamic adjustment of scalability-relevant parameters in a general, flex- ible, and safe manner. I introduce isolation boundaries for microkernel resources and the system processors. The boundaries are controlled at user-level. Operating system components and applications can transform their semantic information into three basic parameters relevant for scalability: the involved processors (depending on their relation and interconnect), degree of concurrency, and groups of resources. I developed a set of mechanisms that allow a kernel to: 1. efficiently track processors on a per-resource basis with support for very large number of processors, 2.
    [Show full text]
  • Symbian Os 925
    CONTENTS PREFACE xxiv 1 INTRODUCTION 1 1.1 WHAT IS AN OPERATING SYSTEM? 3 1.1.1 The Operating System as an Extended Machine 4 1.1.2 The Operating System as a Resource Manager 6 1.2 HISTORY OF OPERATING SYSTEMS 7 1.2.1 The First Generation (1945-55) Vacuum Tubes 7 1.2.2 The Second Generation (1955-65) Transistors and Batch Systems 8 1.2.3 The Third Generation (1965-1980) ICs and Multiprogramming 10 1.2.4 The Fourth Generation (1980—Present) Personal Computers 13 1.3 COMPUTER HARDWARE REVIEW 17 1.3.1 Processors 17 1.3.2 Memory 21 1.3.3 Disks 24 1.3.4 Tapes 25 1.3.5 1/0 Devices 25 1.3.6 Buses 28 1.3.7 Booting the Computer 31 vii Viii CONTENTS 1.4 THE OPERATING SYSTEM ZOO 31 1.4.1 Mainframe Operating Systems 32 1.4.2 Server Operating Systems 32 1.4.3 Multiprocessor Operating Systems 32 1.4.4 Personal Computer Operating Systems 33 1.4.5 Handheld Computer Operating Systems 33 1.4.6 Embedded Operating Systems. 33 1.4.7 Sensor Node Operating Systems 34 1.4.8 Real-Time Operating Systems 34 1.4.9 Smart Card Operating Systems 35 1.5 OPERATING SYSTEM CONCEPTS 35 1.5.1 Processes 36 1.5.2 Address Spaces 38 1.5.3 Files 38 1.5.4 Input/Output 41 1.5.5 Protection 42 1.5.6 The Shell 42 1.5.7 Ontogeny Recapitulates Phylogeny 44 1.6 SYSTEM CALLS 47 1.6.1 System Calls for Process Management 50 1.6.2 System Calls for File Management 54 1.6.3 System Calls for Directory Management 55 1.6.4 Miscellaneous System Calls 56 1.6.5 The Windows Win32 API 57 1.7 OPERATING SYSTEM STRUCTURE 60 1.7.1 Monolithic Systems 60 1.7.2 Layered Systems 61 1.7.3 Microkernels 62 1.7.4
    [Show full text]
  • Microkernels: Mach and L4
    Microkernels: Mach and L4 Presented by Jason Wu With content borrowed from Dan Williams (2009) and Hakim Weatherspoon (2008) Outline • Introduction to Kernels • 1st Generation Microkernels – Mach • 2nd Generation Microkernels – L4 • Conclusions Introduction to Kernels • Different Types of Kernel Designs – Monolithic kernel – Microkernel – Hybrid Kernel – Exokernel – Virtual Machines? Monolithic Kernels • All OS services operate in kernel space • Good performance • Disadvantages – Dependencies between system component – Complex & huge (millions(!) of lines of code) – Larger size makes it hard to maintain • E.g. Multics, Unix, BSD, Linux Microkernels • Minimalist approach – IPC, virtual memory, thread scheduling • Put the rest into user space – Device drivers, networking, file system, user interface • More stable with less services in kernel space • Disadvantages – Lots of system calls and context switches • E.g. Mach, L4, AmigaOS, Minix, K42 Monolithic Kernels VS Microkernels Hybrid Kernels • Combine the best of both worlds – Speed and simple design of a monolithic kernel – Modularity and stability of a microkernel • Still similar to a monolithic kernel – Disadvantages still apply here • E.g. Windows NT, NetWare, BeOS Exokernels • Follows end-to-end principle – Extremely minimal – Fewest hardware abstractions as possible – Just allocates physical resources to apps • Disadvantages – More work for application developers • E.g. Nemesis, ExOS • Next Thursday! The Microkernel Debate • How big should it be? • Big debate during the 1980’s Summary:
    [Show full text]
  • Airetama Um Arcabouço Baseado Em Sistemas Multiagentes Para a Implantação De Comunidades Virtuais De Prática Na Web
    UNIVERSIDADE FEDERAL DE GOIÁS INSTITUTO DE INFORMÁTICA JAIR ABÚ BECHIR LÁSCAR ALARCÓN Airetama Um Arcabouço Baseado em Sistemas Multiagentes para a Implantação de Comunidades Virtuais de Prática na Web Goiânia <2010> UNIVERSIDADE FEDERAL DE GOIÁS INSTITUTO DE INFORMÁTICA AUTORIZAÇÃO PARA PUBLICAÇÃO DE DISSERTAÇÃO EM FORMATO ELETRÔNICO Na qualidade de titular dos direitos de autor, AUTORIZO o Instituto de Infor- mática da Universidade Federal de Goiás – UFG a reproduzir, inclusive em outro formato ou mídia e através de armazenamento permanente ou temporário, bem como a publicar na rede mundial de computadores (Internet) e na biblioteca virtual da UFG, entendendo-se os termos “reproduzir” e “publicar” conforme definições dos incisos VI e I, respectiva- mente, do artigo 5o da Lei no 9610/98 de 10/02/1998, a obra abaixo especificada, sem que me seja devido pagamento a título de direitos autorais, desde que a reprodução e/ou publi- cação tenham a finalidade exclusiva de uso por quem a consulta, e a título de divulgação da produção acadêmica gerada pela Universidade, a partir desta data. Título: Airetama – Um Arcabouço Baseado em Sistemas Multiagentes para a Implanta- ção de Comunidades Virtuais de Prática na Web Autor(a): Jair Abú Bechir Láscar Alarcón Goiânia, <04 > de de <2010>. Jair Abú Bechir Láscar Alarcón – Autor Cedric Luiz de Carvalho – Orientador JAIR ABÚ BECHIR LÁSCAR ALARCÓN Airetama Um Arcabouço Baseado em Sistemas Multiagentes para a Implantação de Comunidades Virtuais de Prática na Web Dissertação apresentada ao Programa de Pós–Graduação do Instituto de Informática da Universidade Federal de Goiás, como requisito parcial para obtenção do título de Mestre em Computação.
    [Show full text]
  • Microkernel Construction Introduction
    Microkernel Construction Introduction Nils Asmussen 04/12/2018 1 / 28 Outline Introduction Goals Administration Monolithic vs. Microkernel Overview About L4/NOVA 2 / 28 Goals 1 Provide deeper understanding of OS mechanisms 2 Look at the implementation details of microkernels 3 Make you become enthusiastic microkernel hackers 4 Propaganda for OS research at TU Dresden 3 / 28 Administration Thursday, 4th DS, 2 SWS Slides: www.tudos.org ! Teaching ! Microkernel Construction Subscribe to our mailing list: www.tudos.org/mailman/listinfo/mkc2018 In winter term: Microkernel-based operating systems (MOS) Various labs 4 / 28 Outline Introduction Monolithic vs. Microkernel Kernel design comparison Examples for microkernel-based systems Vision vs. Reality Challenges Overview About L4/NOVA 5 / 28 Monolithic Kernel System Design u s Application Application Application e r k Kernel e r File Network n e Systems Stacks l m Memory Process o Drivers Management Management d e Hardware 6 / 28 Monolithic Kernel OS (Propaganda) System components run in privileged mode No protection between system components Faulty driver can crash the whole system Malicious app could exploit bug in faulty driver More than 2=3 of today's OS code are drivers No need for good system design Direct access to data structures Undocumented and frequently changing interfaces Big and inflexible Difficult to replace system components Difficult to understand and maintain Why something different? ! Increasingly difficult to manage growing OS complexity 7 / 28 Microkernel System Design Application
    [Show full text]
  • Otherworld - Giving Applications a Chance to Survive OS Kernel Crashes
    Otherworld - Giving Applications a Chance to Survive OS Kernel Crashes by Alexandre Depoutovitch, B.Sc., M.Sc. A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy Graduate Department of Computer Science University of Toronto Copyright c 2011 by Alexandre Depoutovitch, B.Sc., M.Sc. Abstract Otherworld - Giving Applications a Chance to Survive OS Kernel Crashes Alexandre Depoutovitch, B.Sc., M.Sc. Doctor of Philosophy Graduate Department of Computer Science University of Toronto 2011 The default behavior of all commodity operating systems today is to restart the system when a critical error is encountered in the kernel. This terminates all running applications with an attendant loss of ”work in progress” that is non-persistent. Our thesis is that an operating system kernel is simply a component of a larger software system, which is logically well isolated from other components, such as applications, and therefore it should be possible to reboot the kernel without terminating everything else running on the same system. In order to prove this thesis, we designed and implemented a new mechanism, called Otherworld, that microreboots the operating system kernel when a critical error is en- countered in the kernel, and it does so without clobbering the state of the running appli- cations. After the kernel microreboot, Otherworld attempts to resurrect the applications that were running at the time of failure. It does so by restoring the application memory spaces, open files and other resources. In the default case it then continues executing the processes from the point at which they were interrupted by the failure.
    [Show full text]
  • Baumann: the Multikernel: a New OS Architecture for Scalable Multicore
    The Multikernel: A new OS architecture for scalable multicore systems Andrew Baumann,∗ Paul Barham,y Pierre-Evariste Dagand,z Tim Harris,y Rebecca Isaacs,y Simon Peter,∗ Timothy Roscoe,∗ Adrian Schüpbach,∗ and Akhilesh Singhania∗ ∗Systems Group, ETH Zurich yMicrosoft Research, Cambridge zENS Cachan Bretagne Abstract App App App App Commodity computer systems contain more and more OS node OS node OS node OS node processor cores and exhibit increasingly diverse archi- Agreement algorithms State State State Async messages State tectural tradeoffs, including memory hierarchies, inter- replica replica replica replica connects, instruction sets and variants, and IO configu- Arch-specific code rations. Previous high-performance computing systems have scaled in specific cases, but the dynamic nature of Heterogeneous x86 x64 ARM GPU modern client and server workloads, coupled with the cores impossibility of statically optimizing an OS for all work- Interconnect loads and hardware variants pose serious challenges for operating system structures. Figure 1: The multikernel model. We argue that the challenge of future multicore hard- ware is best met by embracing the networked nature of the machine, rethinking OS architecture using ideas from Such hardware, while in some regards similar to ear- distributed systems. We investigate a new OS structure, lier parallel systems, is new in the general-purpose com- the multikernel, that treats the machine as a network of puting domain. We increasingly find multicore systems independent cores, assumes no inter-core sharing at the in a variety of environments ranging from personal com- lowest level, and moves traditional OS functionality to puting platforms to data centers, with workloads that are a distributed system of processes that communicate via less predictable, and often more OS-intensive, than tradi- message-passing.
    [Show full text]