DOCSLIB.ORG

Real-Time Audio Servers on BSD Unix Derivatives

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Real-Time Audio Servers on BSD Unix Derivatives Juha Erkkilä Real-Time Audio Servers on BSD Unix Derivatives Master's Thesis in Information Technology June 17, 2005 University of Jyväskylä Department of Mathematical Information Technology Jyväskylä Author: Juha Erkkilä Contact information: [email protected].fi Title: Real-Time Audio Servers on BSD Unix Derivatives Työn nimi: Reaaliaikaiset äänipalvelinsovellukset BSD Unix -johdannaisjärjestelmissä Project: Master's Thesis in Information Technology Page count: 146 Abstract: This paper covers real-time and interprocess communication features of 4.4BSD Unix derived operating systems, and especially their applicability for real- time audio servers. The research ground of bringing real-time properties to tradi- tional Unix operating systems (such as 4.4BSD) is covered. Included are some design ideas used in BSD-variants, such as using multithreaded kernels, and schedulers that can provide real-time guarantees to processes. Factors affecting the design of real- time audio servers are considered, especially the suitability of various interprocess communication facilities as mechanisms to pass audio data between applications. To test these mechanisms on a real operating system, an audio server and a client utilizing these techniques is written and tested on an OpenBSD operating system. The performance of the audio server and OpenBSD is analyzed, with attempts to identify some bottlenecks of real-time operation in the OpenBSD system. Suomenkielinen tiivistelmä: Tämä tutkielma kattaa reaaliaikaisuus- ja prosessien väliset kommunikaatio-ominaisuudet, keskittyen 4.4BSD Unix -johdannaisiin käyt- töjärjestelmiin, ja erityisesti siihen kuinka hyvin nämä soveltuvat reaaliaikaisille äänipalvelinsovelluksille. Tutkimusalueeseen sisältyy reaaliaikaisuusominaisuuk- sien tuominen perinteisiin Unix-käyttöjärjestelmiin (kuten 4.4BSD:hen). Mukana on suunnitteluideoita, joita on käytetty joissakin BSD-varianteissa, kuten säikeis- tetyt kernelit, ja skedulerit, jotka voivat tarjota reaaliaikaisuustakeita prosesseille. Huomioon otettavina seikkoina ovat myös reaaliaikaisten äänipalvelimien suun- nitteluun vaikuttavat tekijät, erityisesti erilaisten prosessien välisten kommunikaa- tiomekanismien soveltuvuus sovellusten väliseen äänidatan välitykseen. Näiden mekanismien testaamista varten tutkielmaa varten on kirjoitettu yksi palvelinsovel- lus sekä asiakassovellus, joita testataan OpenBSD-käyttöjärjestelmän päällä. Ana- lyysin kohteena ovat testipalvelinsovelluksen ja OpenBSD:n suorituskyky, tarkoituk- sena on erityisesti löytää reaaliaikaisen suorituskyvyn pullonkauloja OpenBSD- käyttöjärjestelmästä. Keywords: real-time distributed BSD unix kernel audio scheduling Avainsanat: reaaliaikainen hajautettu BSD unix kernel ääni skedulointi No copyright, this work is in public domain. Preface You might already know this, but in case you're reading some treeware edition, you might want to learn that this work can also be found on a server that speaks that famous “http”-protocol too. At address http://www.cc.jyu.fi/∼erkkila/thesis/ you should be able to find the LATEX sources, the test software code that has been typeset into the appendices, test scripts, and some other related things. If you ever find them useful, that's great, as that's more than I could ever expect ;-) As is pretty much obvious, this work is heavily inspired by the work that the whole BSD community is doing. While the main reasons for that are actually quite political, it is also because of the way that the community handles politics: it's not in what they say, but in what they do. Thus, if this work inspires anyone to hack on something, anything, it would be interesting to know about that, as that would be the best reward I can ever get. i Glossary API Application Programming Interface. Whatever rules a programmer must abide in order to use some other software component from another software compo- nent Asynchronous I/O With asynchronous I/O, I/O operations proceed independently from the execution of the process that initiated them BSD Unix Berkeley Software Distribution Unix, the branch of Unix that was devel- oped in University of Berkeley, California. OpenBSD is a derivative of BSD Unix, like other systems such as NetBSD, FreeBSD, and Macintosh OS X Exokernel A kernel of an operating system model, where the kernel is reduced to merely controlling access to hardware resources. All higher level functional- ity (such as device drivers and the abstractions they offer) are provided by application libraries Fork The act of a software project splitting off from another project. Also the project that does the forking. The term comes from the Unix fork() system call that creates a new process from the current one GUI Graphical User Interface IPC Interprocess communication. Means exchange of data between processes. Fa- cilities for IPC in practically all Unix system include signals, (anonymous and named) pipes, files, sockets, and shared memory IPL Interrupt Priority Level. Hardware and software interrupts are associated with a level, and kernel can delay interrupt handling (“mask” interrupts) until it is ready to deal with interrupts that have happened on some specific level. This way access to critical data in the kernel can be protected Kernel The core of an operating system. A kernel handles accessing hardware, and provides a uniform interface to hardware for application software Kqueue A generic kernel event notification mechanism, common in BSD variants. It provides an extensible framework for various kernel events, and a notification interface for applications interested in those events ii JACK Jack Audio Connection Kit. A low-latency audio server project for Unix systems Preemptive scheduling Preemptive scheduling means a kind of multitasking, where scheduler may interrupt the execution of a process in order to switch to another process. Preemptive kernel means that a kernel can allow this to happen also when the process is executing in kernel mode QNX A fault-tolerant, real-time, distributed operating system that is based on a microkernel architecture Scheduler The part of the operating system that apportions CPU time to processes SMP Symmetric Multi-Processing. Means that operating system and its processes run on more than one CPU, where CPUs have symmetrical relation to each other, that is, no CPU is treated special SMT Simultaneous multithreading. A performance improvement technique in hardware, which means that a CPU can have several threads executing in parallel. Does not usually provide any benefit unless the operating system knows how to utilize this feature. The system could see deal with it as a variation of SMP SUSv3 Single Unix Specification version 3, the most prevalent standard for the interfaces of Unix systems. Current version combines SUSv2 and the POSIX standard (the older Unix standard published by IEEE), and also in many ways supersedes POSIX System V The version of Unix created by AT&T, first released in 1983. Historically the two major flavors of Unix were considered to be System V and BSD, from which most of the Unix flavors were derived from Thread A thread is a kind of process that shares some execution context with other threads. A process can be viewed as a thread that has its execution context totally isolated from other processes. What this means in practice may be partially system-dependent Unix Originally a system developed at Bell Labs at AT&T. In this paper the term is used to mean a family of operating systems. Unix is a trademarked term, so its use is controlled and for correctness, all occurrences of it in this paper should possibly be replaced with “Unix-like”. This, however, is ridiculous iii X Window System The mostly commonly implemented graphics system in most Unix operating systems. It has client-server architecture, where server accesses the underlying graphics hardware and clients send requests to the server to draw things to the screen. Often referred to simply as “X” iv Contents Preface i Glossary ii 1 Introduction 1 1.1 Real-Time audio servers . 2 1.2 4.4BSD Unix and its derivatives . 5 2 System design in BSD Unix derivatives 8 2.1 Processes . 8 2.2 Interrupts . 10 2.3 Scheduler . 11 2.4 I/O model and IPC . 12 2.5 Audio subsystems in BSD kernels . 13 2.5.1 NetBSD/OpenBSD kernel audio internals . 15 3 Real-Time audio servers 20 3.1 Real-Time systems . 20 3.2 Are audio servers necessary? . 21 3.3 Server requirements . 22 3.4 Acceptable latencies . 24 3.5 The context switching problem . 25 3.5.1 Working around the context switches . 29 3.6 Audio libraries . 30 3.6.1 Audio API emulation . 31 3.7 Working around real-time requirements . 33 3.8 Audio protocols . 35 3.9 Threaded programming . 37 4 Real-Time Unix design 39 4.1 Traditional Unix kernel design . 40 4.2 Semi-preemptibility . 41 4.3 Preemption in kernels with fine-grained locking . 43 v 4.4 Message passing kernels with LWKTs . 48 4.5 Monolithic kernels on microkernels . 52 4.6 Real-Time features of operating systems . 54 4.6.1 Memory locking . 54 4.6.2 Clocks and timers . 55 5 Real-Time scheduling 56 5.1 Traditional Unix scheduling and real-time applications . 56 5.2 SUS scheduling policies . 59 5.3 Scheduler implementations . 61 5.3.1 SVR4 scheduler . 61 5.3.2 Schedulers in BSD systems . 63 5.3.3 Some problems in scheduler implementations . 64 5.3.4 SMP and SMT . 64 5.3.5 Scheduling of client-server systems . 65 5.3.6 Security and scheduler partitioning . 68 5.4 Research on scheduling algorithms . 69 5.4.1 Fair-Share scheduling algorithms . 71 5.4.2 Approaches based on partitioning . 73 6 Interprocess communication 76 6.1 Signals . 76 6.1.1 Real-Time signals . 78 6.2 Pipes and sockets . 79 6.2.1 Anonymous and named pipes . 79 6.2.2 Sockets . 80 6.2.3 Synchronous and asynchronous I/O . 82 6.2.4 Kernel event queues . 84 6.3 Shared memory . 86 6.3.1 Techniques in Mach IPC . 90 6.3.2 Solaris doors . 91 6.4 Data on a network . 92 6.4.1 Stream and datagram sockets .
Load more

Recommended publications
  • The Linux Kernel Module Programming Guide
    The Linux Kernel Module Programming Guide Peter Jay Salzman Michael Burian Ori Pomerantz Copyright © 2001 Peter Jay Salzman 2007−05−18 ver 2.6.4 The Linux Kernel Module Programming Guide is a free book; you may reproduce and/or modify it under the terms of the Open Software License, version 1.1. You can obtain a copy of this license at http://opensource.org/licenses/osl.php. This book is distributed in the hope it will be useful, but without any warranty, without even the implied warranty of merchantability or fitness for a particular purpose. The author encourages wide distribution of this book for personal or commercial use, provided the above copyright notice remains intact and the method adheres to the provisions of the Open Software License. In summary, you may copy and distribute this book free of charge or for a profit. No explicit permission is required from the author for reproduction of this book in any medium, physical or electronic. Derivative works and translations of this document must be placed under the Open Software License, and the original copyright notice must remain intact. If you have contributed new material to this book, you must make the material and source code available for your revisions. Please make revisions and updates available directly to the document maintainer, Peter Jay Salzman <[email protected]>. This will allow for the merging of updates and provide consistent revisions to the Linux community. If you publish or distribute this book commercially, donations, royalties, and/or printed copies are greatly appreciated by the author and the Linux Documentation Project (LDP).
    [Show full text]
  • THINC: a Virtual and Remote Display Architecture for Desktop Computing and Mobile Devices
    THINC: A Virtual and Remote Display Architecture for Desktop Computing and Mobile Devices Ricardo A. Baratto Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Graduate School of Arts and Sciences COLUMBIA UNIVERSITY 2011 c 2011 Ricardo A. Baratto This work may be used in accordance with Creative Commons, Attribution-NonCommercial-NoDerivs License. For more information about that license, see http://creativecommons.org/licenses/by-nc-nd/3.0/. For other uses, please contact the author. ABSTRACT THINC: A Virtual and Remote Display Architecture for Desktop Computing and Mobile Devices Ricardo A. Baratto THINC is a new virtual and remote display architecture for desktop computing. It has been designed to address the limitations and performance shortcomings of existing remote display technology, and to provide a building block around which novel desktop architectures can be built. THINC is architected around the notion of a virtual display device driver, a software-only component that behaves like a traditional device driver, but instead of managing specific hardware, enables desktop input and output to be intercepted, manipulated, and redirected at will. On top of this architecture, THINC introduces a simple, low-level, device-independent representation of display changes, and a number of novel optimizations and techniques to perform efficient interception and redirection of display output. This dissertation presents the design and implementation of THINC. It also intro- duces a number of novel systems which build upon THINC's architecture to provide new and improved desktop computing services. The contributions of this dissertation are as follows: • A high performance remote display system for LAN and WAN environments.
    [Show full text]
  • Procedures to Build Crypto Libraries in Minix
    Created by Jinkai Gao (Syracuse University) Seed Document How to talk to inet server Note: this docment is fully tested only on Minix3.1.2a. In this document, we introduce a method to let user level program to talk to inet server. I. Problem with system call Recall the process of system call, refering to http://www.cis.syr.edu/~wedu/seed/Labs/Documentation/Minix3/System_call_sequence.pd f We can see the real system call happens in this function call: _syscall(FS, CHMOD, &m) This function executes ‘INT 80’ to trap into kernel. Look at the parameter it passs to kernel. ‘CHMOD’ is a macro which is merely the system call number. ‘FS’ is a macro which indicates the server which handles the chmod system call, in this case ‘FS’ is 1, which is the pid of file system server process. Now your might ask ‘why can we hard code the pid of the process? Won’t it change?’ Yes, normally the pid of process is unpredictable each time the system boots up. But for fs, pm, rs, init and other processes which is loaded from system image at the very beginning of booting time, it is not true. In minix, you can dump the content of system image by pressing ‘F3’, then dump the current running process table by pressing ‘F1’. What do you find? The first 12 entries in current process table is exactly the ones in system image with the same order. So the pids of these 12 processes will not change. Inet is different. It is not in the system image, so it is not loaded into memory in the very first time.
    [Show full text]
  • UNIX Systems Programming II Systems Unixprogramming II Short Course Notes
    Systems UNIXProgramming II Systems UNIXProgramming II Systems UNIXProgramming II UNIX Systems Programming II Systems UNIXProgramming II Short Course Notes Alan Dix © 1996 Systems Programming II http://www.hcibook.com/alan/ UNIX Systems Course UNIXProgramming II Outline Alan Dix http://www.hcibook.com/alan/ Session 1 files and devices inodes, stat, /dev files, ioctl, reading directories, file descriptor sharing and dup2, locking and network caching Session 2 process handling UNIX processes, fork, exec, process death: SIGCHLD and wait, kill and I/O issues for fork Session 3 inter-process pipes: at the shell , in C code and communication use with exec, pseudo-terminals, sockets and deadlock avoidance Session 4 non-blocking I/O and UNIX events: signals, times and select I/O; setting timers, polling, select, interaction with signals and an example Internet server Systems UNIXProgrammingII Short Course Notes Alan Dix © 1996 II/i Systems Reading UNIXProgramming II ¥ The Unix V Environment, Stephen R. Bourne, Wiley, 1987, ISBN 0 201 18484 2 The author of the Borne Shell! A 'classic' which deals with system calls, the shell and other aspects of UNIX. ¥ Unix For Programmers and Users, Graham Glass, Prentice-Hall, 1993, ISBN 0 13 061771 7 Slightly more recent book also covering shell and C programming. Ì BEWARE Ð UNIX systems differ in details, check on-line documentation ¥ UNIX manual pages: man creat etc. Most of the system calls and functions are in section 2 and 3 of the manual. The pages are useful once you get used to reading them! ¥ The include files themselves /usr/include/time.h etc.
    [Show full text]
  • Active-Active Firewall Cluster Support in Openbsd
    Active-Active Firewall Cluster Support in OpenBSD David Gwynne School of Information Technology and Electrical Engineering, University of Queensland Submitted for the degree of Bachelor of Information Technology COMP4000 Special Topics Industry Project February 2009 to leese, who puts up with this stuff ii Acknowledgements I would like to thank Peter Sutton for allowing me the opportunity to do this work as part of my studies at the University of Queensland. A huge thanks must go to Ryan McBride for answering all my questions about pf and pfsync in general, and for the many hours working with me on this problem and helping me test and debug the code. Thanks also go to Theo de Raadt, Claudio Jeker, Henning Brauer, and everyone else at the OpenBSD network hackathons who helped me through this. iii Abstract The OpenBSD UNIX-like operating system has developed several technologies that make it useful in the role of an IP router and packet filtering firewall. These technologies include support for several standard routing protocols such as BGP and OSPF, a high performance stateful IP packet filter called pf, shared IP address and fail-over support with CARP (Common Address Redundancy Protocol), and a protocol called pfsync for synchronisation of the firewalls state with firewalls over a network link. These technologies together allow the deployment of two or more computers to provide redundant and highly available routers on a network. However, when performing stateful filtering of the TCP protocol with pf, the routers must be configured in an active-passive configuration due to the current semantics of pfsync.
    [Show full text]
  • Microkernel Mechanisms for Improving the Trustworthiness of Commodity Hardware
    Microkernel Mechanisms for Improving the Trustworthiness of Commodity Hardware Yanyan Shen Submitted in fulfilment of the requirements for the degree of Doctor of Philosophy School of Computer Science and Engineering Faculty of Engineering March 2019 Thesis/Dissertation Sheet Surname/Family Name : Shen Given Name/s : Yanyan Abbreviation for degree as give in the University calendar : PhD Faculty : Faculty of Engineering School : School of Computer Science and Engineering Microkernel Mechanisms for Improving the Trustworthiness of Commodity Thesis Title : Hardware Abstract 350 words maximum: (PLEASE TYPE) The thesis presents microkernel-based software-implemented mechanisms for improving the trustworthiness of computer systems based on commercial off-the-shelf (COTS) hardware that can malfunction when the hardware is impacted by transient hardware faults. The hardware anomalies, if undetected, can cause data corruptions, system crashes, and security vulnerabilities, significantly undermining system dependability. Specifically, we adopt the single event upset (SEU) fault model and address transient CPU or memory faults. We take advantage of the functional correctness and isolation guarantee provided by the formally verified seL4 microkernel and hardware redundancy provided by multicore processors, design the redundant co-execution (RCoE) architecture that replicates a whole software system (including the microkernel) onto different CPU cores, and implement two variants, loosely-coupled redundant co-execution (LC-RCoE) and closely-coupled redundant co-execution (CC-RCoE), for the ARM and x86 architectures. RCoE treats each replica of the software system as a state machine and ensures that the replicas start from the same initial state, observe consistent inputs, perform equivalent state transitions, and thus produce consistent outputs during error-free executions.
    [Show full text]
  • The Design and Implementation of Multiprocessor Support for an Industrial Operating System Kernel
    Blekinge Institute of Technology Research Report 2005:06 The Design and Implementation of Multiprocessor Support for an Industrial Operating System Kernel Simon Kågström Håkan Grahn Lars Lundberg Department of Systems and Software School of Engineering Blekinge Institute of Technology The Design and Implementation of Multiprocessor Support for an Industrial Operating System Kernel Simon Kågström, Håkan Grahn, and Lars Lundberg Department of Systems and Software Engineering School of Engineering Blekinge Institute of Technology P.O. Box 520, SE-372 25 Ronneby, Sweden {ska, hgr, llu}@bth.se Abstract The ongoing transition from uniprocessor to multiprocessor computers requires support from the op- erating system kernel. Although many general-purpose multiprocessor operating systems exist, there is a large number of specialized operating systems which require porting in order to work on multipro- cessors. In this paper we describe the multiprocessor port of a cluster operating system kernel from a producer of industrial systems. Our initial implementation uses a giant locking scheme that serializes kernel execution. We also employed a method in which CPU-local variables are placed in a special sec- tion mapped to per-CPU physical memory pages. The giant lock and CPU-local section allowed us to implement an initial working version with only minor changes to the original code, although the giant lock and kernel-bound applications limit the performance of our multiprocessor port. Finally, we also discuss experiences from the implementation. 1 Introduction A current trend in the computer industry is the transition from uniprocessors to various kinds of multipro- cessors, also for desktop and embeddedsystems. Apart from traditional SMP systems, many manufacturers are now presenting chip multiprocessors or simultaneous multithreaded CPUs [9, 15, 16] which allow more efficient use of chip area.
    [Show full text]
  • Accessing Windows Applications from Unix and Vice Versa
    50-20-42 DATA COMMUNICATIONS MANAGEMENT ACCESSING WINDOWS APPLICATIONS FROM UNIX AND VICE VERSA Raj Rajagopal INSIDE Accessing Windows Applications from an X-Station, Coexistence Options, Windows in an X-Station, Accessing Windows Applications, Accessing UNIX Applications from Windows Desktops, Emulators Migrating from one environment to another takes planning, resources and, most importantly, time (except in very trivial cases). This implies that even if eventually migrating to another environment, one still has to deal with coexistence among environments in the interim. In many com- panies it would make good business sense not to migrate legacy systems at all. Instead, it may be better to develop new systems in the desired en- vironment and phase out the legacy applications. The data created by the legacy applications is important and one must ensure that data can be ac- cessed from a new environment. Coexistence considerations are very im- portant in this case. Coexistence between Windows PAYOFF IDEA NT, UNIX, and NetWare deals with a Some users want applications they develop in number of related issues. One may one environment to execute in other environ- need to access Windows applications ments with very little change. With this approach, they can continue to develop applications with from a UNIX machine or need to ac- the confidence that they will execute in another cess UNIX applications from Win- environment even if the environments change in dows desktops. One may prefer to the future. In applications that can run in both have the same type of desktop (Òan Windows NT and UNIX, this can be accomplished enterprise desktopÓ) for all users and in several ways: be able to access different environ- •use APIs — there are three flavors of this ap- ments.
    [Show full text]
  • Advanced Operating Systems #1
    <Slides download> https://www.pf.is.s.u-tokyo.ac.jp/classes Advanced Operating Systems #2 Hiroyuki Chishiro Project Lecturer Department of Computer Science Graduate School of Information Science and Technology The University of Tokyo Room 007 Introduction of Myself • Name: Hiroyuki Chishiro • Project Lecturer at Kato Laboratory in December 2017 - Present. • Short Bibliography • Ph.D. at Keio University on March 2012 (Yamasaki Laboratory: Same as Prof. Kato). • JSPS Research Fellow (PD) in April 2012 – March 2014. • Research Associate at Keio University in April 2014 – March 2016. • Assistant Professor at Advanced Institute of Industrial Technology in April 2016 – November 2017. • Research Interests • Real-Time Systems • Operating Systems • Middleware • Trading Systems Course Plan • Multi-core Resource Management • Many-core Resource Management • GPU Resource Management • Virtual Machines • Distributed File Systems • High-performance Networking • Memory Management • Network on a Chip • Embedded Real-time OS • Device Drivers • Linux Kernel Schedule 1. 2018.9.28 Introduction + Linux Kernel (Kato) 2. 2018.10.5 Linux Kernel (Chishiro) 3. 2018.10.12 Linux Kernel (Kato) 4. 2018.10.19 Linux Kernel (Kato) 5. 2018.10.26 Linux Kernel (Kato) 6. 2018.11.2 Advanced Research (Chishiro) 7. 2018.11.9 Advanced Research (Chishiro) 8. 2018.11.16 (No Class) 9. 2018.11.23 (Holiday) 10. 2018.11.30 Advanced Research (Kato) 11. 2018.12.7 Advanced Research (Kato) 12. 2018.12.14 Advanced Research (Chishiro) 13. 2018.12.21 Linux Kernel 14. 2019.1.11 Linux Kernel 15. 2019.1.18 (No Class) 16. 2019.1.25 (No Class) Linux Kernel Introducing Synchronization /* The cases for Linux */ Acknowledgement: Prof.
    [Show full text]
  • Application Performance and Flexibility on Exokernel Systems
    Application Performance and Flexibility On Exokernel Systems –M. F. Kaashoek, D.R. Engler, G.R. Ganger et. al. Presented by Henry Monti Exokernel ● What is an Exokernel? ● What are its advantages? ● What are its disadvantages? ● Performance ● Conclusions 2 The Problem ● Traditional operating systems provide both protection and resource management ● Will protect processes address space, and manages the systems virtual memory ● Provides abstractions such as processes, files, pipes, sockets,etc ● Problems with this? 3 Exokernel's Solution ● The problem is that OS designers must try and accommodate every possible use of abstractions, and choose general solutions for resource management ● This causes the performance of applications to suffer, since they are restricted to general abstractions, interfaces, and management ● Research has shown that more application control of resources yields better performance 4 (Separation vs. Protection) ● An exokernel solves these problems by separating protection from management ● Creates idea of LibOS, which is like a virtual machine with out isolation ● Moves abstractions (processes, files, virtual memory, filesystems), to the user space inside of LibOSes ● Provides a low level interface as close to the hardware as possible 5 General Design Principles ● Exposes as much as possible of the hardware, most of the time using hardware names, allowing libOSes access to things like individual data blocks ● Provides methods to securely bind to resources, revoke resources, and an abort protocol that the kernel itself
    [Show full text]
  • CSC 405 Computer Security Linux Security
    CSC 405 Computer Security Linux Security Alexandros Kapravelos [email protected] Unix / Linux • Started in 1969 at AT&T / Bell Labs • Split into a number of popular branches – BSD, System V (commercial, AT&T), Solaris, HP-UX, AIX • Inspired a number of Unix-like systems – Linux, Minix • Standardization attempts – POSIX, Single Unix Specification (SUS), Filesystem Hierarchy Standard (FHS), Linux Standard Base (LSB), ELF OS Security • Kernel vulnerability – usually leads to complete system compromise – attacks performed via system calls Kernel vulnerabilities Kernel vulnerabilities Kernel exploitation research is active Unleashing Use-Before-Initialization Vulnerabilities in the Linux Kernel Using Targeted Stack Spraying • reliably exploiting uninitialized uses on the kernel stack has been considered infeasible • code executed prior to triggering the vulnerability must leave an attacker-controlled pattern on the stack • a fully automated targeted stackspraying approach for the Linux kernel that reliably facilitates the exploitation of uninitialized uses • published in NDSS 2017 source: https://www.cc.gatech.edu/~klu38/publications/ubi-ndss17.pdf Unix • Code running in user mode is always linked to a certain identity – security checks and access control decisions are based on user identity • Unix is user-centric – no roles • User – identified by username (UID), group name (GID) – typically authenticated by password (stored encrypted) • User root – superuser, system administrator – special privileges (access resources, modify OS) – cannot
    [Show full text]
  • Filesystem Performance on Freebsd
    Filesystem Performance on FreeBSD Kris Kennaway [email protected] BSDCan 2006, Ottawa, May 12 Introduction ● Filesystem performance has many aspects ● No single metric for quantifying it ● I will focus on aspects that are relevant for my workloads (concurrent package building) ● The main relevant filesystem workloads seem to be – Concurrent tarball extraction – Recursive filesystem traversals ● Aim: determine relative performance of FreeBSD 4.x, 5.x and 6.x on these workloads – Overall performance and SMP scaling – Evaluate results of multi-year kernel locking strategy as it relates to these workloads Outline ● SMP architectural differences between 4/5/6.x ● Test methodology ● Hardware used ● Parallel tarball extraction test – Disk array and memory disk ● Scaling beyond 4 CPUs ● Recursive filesystem traversal test ● Conclusions & future work SMP Architectural Overview ● FreeBSD 4.x; rudimentary SMP support – Giant kernel lock restricts kernel access to one process at a time – SPL model; interrupts may still be processed in parallel ● FreeBSD 5.x; aim towards greater scalability – Giant-locked to begin with; then finer-grained locking pushdown ● FreeBSD 5.3; VM Giant-free ● FreeBSD 5.4; network stack Giant-free (mostly) ● Many other subsystems/drivers also locked – Interrupts as kernel threads; compete for common locks (if any) with everything else ● FreeBSD 6.x; – Consolidation; further pushdown; payoff! – VFS subsystem, UFS filesystem Giant-free FreeBSD versions ● FreeBSD 4.11-STABLE (11/2005) – Needed for amr driver fixes after 4.11-RELEASE ● FreeBSD 5.4-STABLE (11/05) – No patches needed ● FreeBSD 6.0-STABLE (11/05) – patches: ● Locking reworked in amr driver by Scott Long for better performance ● All relevant changes merged into FreeBSD 6.1 – A kernel panic was encountered at very high I/O loads ● Also fixed in 6.1 Test aims and Methodology ● Want to measure – overall performance difference between FreeBSD branches under varying (concurrent process I/O) loads – scaling to multiple CPUs ● Avoid saturating hardware resources (e.g.
    [Show full text]