An Overview of Scheduling in the Freebsd Kernel
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Proceedings of the Bsdcon 2002 Conference
USENIX Association Proceedings of the BSDCon 2002 Conference San Francisco, California, USA February 11-14, 2002 THE ADVANCED COMPUTING SYSTEMS ASSOCIATION © 2002 by The USENIX Association All Rights Reserved For more information about the USENIX Association: Phone: 1 510 528 8649 FAX: 1 510 548 5738 Email: [email protected] WWW: http://www.usenix.org Rights to individual papers remain with the author or the author's employer. Permission is granted for noncommercial reproduction of the work for educational or research purposes. This copyright notice must be included in the reproduced paper. USENIX acknowledges all trademarks herein. Rethinking /devand devices in the UNIX kernel Poul-Henning Kamp <[email protected]> The FreeBSD Project Abstract An outstanding novelty in UNIX at its introduction was the notion of ‘‘a file is a file is a file and evenadevice is a file.’’ Going from ‘‘hardware only changes when the DEC Field engineer is here’’to‘‘my toaster has USB’’has put serious strain on the rather crude implementation of the ‘‘devices as files’’concept, an implementation which has survivedpractically unchanged for 30 years in most UNIX variants. Starting from a high-levelviewofdevices and the semantics that have grown around them overthe years, this paper takes the audience on a grand tour of the redesigned FreeBSD device-I/O system, to convey anoverviewofhow itall fits together,and to explain whythings ended up as theydid, howtouse the newfeatures and in particular hownot to. 1. Introduction tax and meaning, so that a program expecting a file name as a parameter can be passed a device name; There are really only twofundamental ways to concep- finally,special files are subject to the same protec- tualise I/O devices in an operating system: The usual tion mechanism as regular files. -
Kernel-Scheduled Entities for Freebsd
Kernel-Scheduled Entities for FreeBSD Jason Evans [email protected] November 7, 2000 Abstract FreeBSD has historically had less than ideal support for multi-threaded application programming. At present, there are two threading libraries available. libc r is entirely invisible to the kernel, and multiplexes threads within a single process. The linuxthreads port, which creates a separate process for each thread via rfork(), plus one thread to handle thread synchronization, relies on the kernel scheduler to multiplex ”threads” onto the available processors. Both approaches have scaling problems. libc r does not take advantage of multiple processors and cannot avoid blocking when doing I/O on ”fast” devices. The linuxthreads port requires one process per thread, and thus puts strain on the kernel scheduler, as well as requiring significant kernel resources. In addition, thread switching can only be as fast as the kernel's process switching. This paper summarizes various methods of implementing threads for application programming, then goes on to describe a new threading architecture for FreeBSD, based on what are called kernel- scheduled entities, or scheduler activations. 1 Background FreeBSD has been slow to implement application threading facilities, perhaps because threading is difficult to implement correctly, and the FreeBSD's general reluctance to build substandard solutions to problems. Instead, two technologies originally developed by others have been integrated. libc r is based on Chris Provenzano's userland pthreads implementation, and was significantly re- worked by John Birrell and integrated into the FreeBSD source tree. Since then, numerous other people have improved and expanded libc r's capabilities. At this time, libc r is a high-quality userland implementation of POSIX threads. -
Jitk: a Trustworthy In-Kernel Interpreter Infrastructure
Jitk: A trustworthy in-kernel interpreter infrastructure Xi Wang, David Lazar, Nickolai Zeldovich, Adam Chlipala, Zachary Tatlock MIT and University of Washington Modern OSes run untrusted user code in kernel In-kernel interpreters - Seccomp: sandboxing (Linux) - BPF: packet filtering - INET_DIAG: socket monitoring - Dtrace: instrumentation Critical to overall system security - Any interpreter bugs are serious! 2/30 Many bugs have been found in interpreters Kernel space bugs - Control flow errors: incorrect jump offset, ... - Arithmetic errors: incorrect result, ... - Memory errors: buffer overflow, ... - Information leak: uninitialized read Kernel-user interface bugs - Incorrect encoding/decoding User space bugs - Incorrect input generated by tools/libraries Some have security consequences: CVE-2014-2889, ... See our paper for a case study of bugs 3/30 How to get rid of all these bugs at once? Theorem proving can help kill all these bugs seL4: provably correct microkernel [SOSP'09] CompCert: provably correct C compiler [CACM'09] This talk: Jitk - Provably correct interpreter for running untrusted user code - Drop-in replacement for Linux's seccomp - Built using Coq proof assistant + CompCert 5/30 Theorem proving: overview specification proof implementation Proof is machine-checkable: Coq proof assistant Proof: correct specification correct implementation Specification should be much simpler than implementation 6/30 Challenges What is the specification? How to translate systems properties into proofs? How to extract a running -
Chapter 1. Origins of Mac OS X
1 Chapter 1. Origins of Mac OS X "Most ideas come from previous ideas." Alan Curtis Kay The Mac OS X operating system represents a rather successful coming together of paradigms, ideologies, and technologies that have often resisted each other in the past. A good example is the cordial relationship that exists between the command-line and graphical interfaces in Mac OS X. The system is a result of the trials and tribulations of Apple and NeXT, as well as their user and developer communities. Mac OS X exemplifies how a capable system can result from the direct or indirect efforts of corporations, academic and research communities, the Open Source and Free Software movements, and, of course, individuals. Apple has been around since 1976, and many accounts of its history have been told. If the story of Apple as a company is fascinating, so is the technical history of Apple's operating systems. In this chapter,[1] we will trace the history of Mac OS X, discussing several technologies whose confluence eventually led to the modern-day Apple operating system. [1] This book's accompanying web site (www.osxbook.com) provides a more detailed technical history of all of Apple's operating systems. 1 2 2 1 1.1. Apple's Quest for the[2] Operating System [2] Whereas the word "the" is used here to designate prominence and desirability, it is an interesting coincidence that "THE" was the name of a multiprogramming system described by Edsger W. Dijkstra in a 1968 paper. It was March 1988. The Macintosh had been around for four years. -
On Intelligent Mitigation of Process Starvation in Multilevel Feedback Queue Scheduling Joseph E
Kennesaw State University DigitalCommons@Kennesaw State University Master of Science in Computer Science Theses Department of Computer Science Spring 4-20-2017 On Intelligent Mitigation of Process Starvation In Multilevel Feedback Queue Scheduling Joseph E. Brown Kennesaw State University Follow this and additional works at: http://digitalcommons.kennesaw.edu/cs_etd Part of the Computational Engineering Commons Recommended Citation Brown, Joseph E., "On Intelligent Mitigation of Process Starvation In Multilevel Feedback Queue Scheduling" (2017). Master of Science in Computer Science Theses. 8. http://digitalcommons.kennesaw.edu/cs_etd/8 This Thesis is brought to you for free and open access by the Department of Computer Science at DigitalCommons@Kennesaw State University. It has been accepted for inclusion in Master of Science in Computer Science Theses by an authorized administrator of DigitalCommons@Kennesaw State University. For more information, please contact [email protected]. On Intelligent Mitigation of Process Starvation In Multilevel Feedback Queue Scheduling Master of Science in Computer Science Thesis By Joseph E Brown MSCS Student Department of Computer Science College of Computing and Software Engineering Kennesaw State University, USA Submitted in partial fulfillment of the Requirements for the degree of Master of Science in Computer Science November 2016 On Intelligent Mitigation of Process Starvation In Multilevel Feedback Queue Scheduling This thesis approved for recommendation to the Graduate Council. 2 Kennesaw State University College of Computing and Software Engineering Thesis Title: On Intelligent Mitigation of Process Starvation In Multilevel Feedback Queue Schedul- ing . Author: Joseph E Brown. Department: Computer Science. Approved for Thesis Requirements of the Master of Science Degree Thesis Advisor: Ken Hoganson Date Thesis Reader: Dr. -
Packet Capture Procedures on Cisco Firepower Device
Packet Capture Procedures on Cisco Firepower Device Contents Introduction Prerequisites Requirements Components Used Steps to Capture Packets Copy a Pcap File Introduction This document describes how to use the tcpdump command in order to capture packets that are seen by a network interface of your Firepower device. It uses Berkeley Packet Filter (BPF) syntax. Prerequisites Requirements Cisco recommends that you have knowledge of the Cisco Firepower device and the virtual device models. Components Used This document is not restricted to specific software and hardware versions. The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command. Warning: If you run tcpdump command on a production system, it can impact network performance. Steps to Capture Packets Log in to the CLI of your Firepower device. In versions 6.1 and later, enter capture-traffic. For example, > capture-traffic Please choose domain to capture traffic from: 0 - eth0 1 - Default Inline Set (Interfaces s2p1, s2p2) In versions 6.0.x.x and earlier, enter system support capture-traffic. For example, > system support capture-traffic Please choose domain to capture traffic from: 0 - eth0 1 - Default Inline Set (Interfaces s2p1, s2p2) After you make a selection, you will be prompted for options: Please specify tcpdump options desired. (or enter '?' for a list of supported options) Options: In order to capture sufficient data from the packets, it is necessary to use the -s option in order to set the snaplength correctly. -
The Complete Freebsd
The Complete FreeBSD® If you find errors in this book, please report them to Greg Lehey <grog@Free- BSD.org> for inclusion in the errata list. The Complete FreeBSD® Fourth Edition Tenth anniversary version, 24 February 2006 Greg Lehey The Complete FreeBSD® by Greg Lehey <[email protected]> Copyright © 1996, 1997, 1999, 2002, 2003, 2006 by Greg Lehey. This book is licensed under the Creative Commons “Attribution-NonCommercial-ShareAlike 2.5” license. The full text is located at http://creativecommons.org/licenses/by-nc-sa/2.5/legalcode. You are free: • to copy, distribute, display, and perform the work • to make derivative works under the following conditions: • Attribution. You must attribute the work in the manner specified by the author or licensor. • Noncommercial. You may not use this work for commercial purposes. This clause is modified from the original by the provision: You may use this book for commercial purposes if you pay me the sum of USD 20 per copy printed (whether sold or not). You must also agree to allow inspection of printing records and other material necessary to confirm the royalty sums. The purpose of this clause is to make it attractive to negotiate sensible royalties before printing. • Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under a license identical to this one. • For any reuse or distribution, you must make clear to others the license terms of this work. • Any of these conditions can be waived if you get permission from the copyright holder. Your fair use and other rights are in no way affected by the above. -
Ebpf-Based Content and Computation-Aware Communication for Real-Time Edge Computing
eBPF-based Content and Computation-aware Communication for Real-time Edge Computing Sabur Baidya1, Yan Chen2 and Marco Levorato1 1Donald Bren School of Information and Computer Science, UC Irvine e-mail: fsbaidya, [email protected] 2America Software Laboratory, Huawei, e-mail: [email protected] Abstract—By placing computation resources within a one-hop interference constraints on IoT data streams and facilitate their wireless topology, the recent edge computing paradigm is a key coexistence. enabler of real-time Internet of Things (IoT) applications. In In this paper, we propose a computation-aware commu- the context of IoT scenarios where the same information from a sensor is used by multiple applications at different locations, the nication control framework for real-time IoT applications data stream needs to be replicated. However, the transportation generating high-volume data traffic processed at the network of parallel streams might not be feasible due to limitations in the edge. Driven by QoC requirements, the framework provides capacity of the network transporting the data. To address this real-time user-controlled packet replication and forwarding issue, a content and computation-aware communication control inside the in-kernel Virtual Machines (VM) using an extended framework is proposed based on the Software Defined Network (SDN) paradigm. The framework supports multi-streaming using Berkeley Packet Filter (eBPF) [9]. The implementation uses the extended Berkeley Packet Filter (eBPF), where the traffic flow the concepts of SDN and NFV to achieve highly program- and packet replication for each specific computation process is able and dynamic packet replication. Resource allocation is controlled by a program running inside an in-kernel Virtual Ma- semantic and content-aware, and, in the considered case, chine (VM). -
Thread Scheduling in Multi-Core Operating Systems Redha Gouicem
Thread Scheduling in Multi-core Operating Systems Redha Gouicem To cite this version: Redha Gouicem. Thread Scheduling in Multi-core Operating Systems. Computer Science [cs]. Sor- bonne Université, 2020. English. tel-02977242 HAL Id: tel-02977242 https://hal.archives-ouvertes.fr/tel-02977242 Submitted on 24 Oct 2020 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Ph.D thesis in Computer Science Thread Scheduling in Multi-core Operating Systems How to Understand, Improve and Fix your Scheduler Redha GOUICEM Sorbonne Université Laboratoire d’Informatique de Paris 6 Inria Whisper Team PH.D.DEFENSE: 23 October 2020, Paris, France JURYMEMBERS: Mr. Pascal Felber, Full Professor, Université de Neuchâtel Reviewer Mr. Vivien Quéma, Full Professor, Grenoble INP (ENSIMAG) Reviewer Mr. Rachid Guerraoui, Full Professor, École Polytechnique Fédérale de Lausanne Examiner Ms. Karine Heydemann, Associate Professor, Sorbonne Université Examiner Mr. Etienne Rivière, Full Professor, University of Louvain Examiner Mr. Gilles Muller, Senior Research Scientist, Inria Advisor Mr. Julien Sopena, Associate Professor, Sorbonne Université Advisor ABSTRACT In this thesis, we address the problem of schedulers for multi-core architectures from several perspectives: design (simplicity and correct- ness), performance improvement and the development of application- specific schedulers. -
Kratka Povijest Unixa Od Unicsa Do Freebsda I Linuxa
Kratka povijest UNIXa Od UNICSa do FreeBSDa i Linuxa 1 Autor: Hrvoje Horvat Naslov: Kratka povijest UNIXa - Od UNICSa do FreeBSDa i Linuxa Licenca i prava korištenja: Svi imaju pravo koristiti, mijenjati, kopirati i štampati (printati) knjigu, prema pravilima GNU GPL licence. Mjesto i godina izdavanja: Osijek, 2017 ISBN: 978-953-59438-0-8 (PDF-online) URL publikacije (PDF): https://www.opensource-osijek.org/knjige/Kratka povijest UNIXa - Od UNICSa do FreeBSDa i Linuxa.pdf ISBN: 978-953- 59438-1- 5 (HTML-online) DokuWiki URL (HTML): https://www.opensource-osijek.org/dokuwiki/wiki:knjige:kratka-povijest- unixa Verzija publikacije : 1.0 Nakalada : Vlastita naklada Uz pravo svakoga na vlastito štampanje (printanje), prema pravilima GNU GPL licence. Ova knjiga je napisana unutar inicijative Open Source Osijek: https://www.opensource-osijek.org Inicijativa Open Source Osijek je član udruge Osijek Software City: http://softwarecity.hr/ UNIX je registrirano i zaštićeno ime od strane tvrtke X/Open (Open Group). FreeBSD i FreeBSD logo su registrirani i zaštićeni od strane FreeBSD Foundation. Imena i logo : Apple, Mac, Macintosh, iOS i Mac OS su registrirani i zaštićeni od strane tvrtke Apple Computer. Ime i logo IBM i AIX su registrirani i zaštićeni od strane tvrtke International Business Machines Corporation. IEEE, POSIX i 802 registrirani i zaštićeni od strane instituta Institute of Electrical and Electronics Engineers. Ime Linux je registrirano i zaštićeno od strane Linusa Torvaldsa u Sjedinjenim Američkim Državama. Ime i logo : Sun, Sun Microsystems, SunOS, Solaris i Java su registrirani i zaštićeni od strane tvrtke Sun Microsystems, sada u vlasništvu tvrtke Oracle. Ime i logo Oracle su u vlasništvu tvrtke Oracle. -
2. Virtualizing CPU(2)
Operating Systems 2. Virtualizing CPU Pablo Prieto Torralbo DEPARTMENT OF COMPUTER ENGINEERING AND ELECTRONICS This material is published under: Creative Commons BY-NC-SA 4.0 2.1 Virtualizing the CPU -Process What is a process? } A running program. ◦ Program: Static code and static data sitting on the disk. ◦ Process: Dynamic instance of a program. ◦ You can have multiple instances (processes) of the same program (or none). ◦ Users usually run more than one program at a time. Web browser, mail program, music player, a game… } The process is the OS’s abstraction for execution ◦ often called a job, task... ◦ Is the unit of scheduling Program } A program consists of: ◦ Code: machine instructions. ◦ Data: variables stored and manipulated in memory. Initialized variables (global) Dynamically allocated (malloc, new) Stack variables (function arguments, C automatic variables) } What is added to a program to become a process? ◦ DLLs: Libraries not compiled or linked with the program (probably shared with other programs). ◦ OS resources: open files… Program } Preparing a program: Task Editor Source Code Source Code A B Compiler/Assembler Object Code Object Code Other A B Objects Linker Executable Dynamic Program File Libraries Loader Executable Process in Memory Process Creation Process Address space Code Static Data OS res/DLLs Heap CPU Memory Stack Loading: Code OS Reads on disk program Static Data and places it into the address space of the process Program Disk Eagerly/Lazily Process State } Each process has an execution state, which indicates what it is currently doing ps -l, top ◦ Running (R): executing on the CPU Is the process that currently controls the CPU How many processes can be running simultaneously? ◦ Ready/Runnable (R): Ready to run and waiting to be assigned by the OS Could run, but another process has the CPU Same state (TASK_RUNNING) in Linux. -
Processor Affinity Or Bound Multiprocessing?
Processor Affinity or Bound Multiprocessing? Easing the Migration to Embedded Multicore Processing Shiv Nagarajan, Ph.D. QNX Software Systems [email protected] Processor Affinity or Bound Multiprocessing? QNX Software Systems Abstract Thanks to higher computing power and system density at lower clock speeds, multicore processing has gained widespread acceptance in embedded systems. Designing systems that make full use of multicore processors remains a challenge, however, as does migrating systems designed for single-core processors. Bound multiprocessing (BMP) can help with these designs and these migrations. It adds a subtle but critical improvement to symmetric multiprocessing (SMP) processor affinity: it implements runmask inheritance, a feature that allows developers to bind all threads in a process or even a subsystem to a specific processor without code changes. QNX Neutrino RTOS Introduction The QNX® Neutrino® RTOS has been Effective use of multicore processors can multiprocessor capable since 1997, and profoundly improve the performance of embedded is deployed in hundreds of systems and applications ranging from industrial control systems thousands of multicore processors in to multimedia platforms. Designing systems that embedded environments. It supports make effective use of multiprocessors remains a AMP, SMP — and BMP. challenge, however. The problem is not just one of making full use of the new multicore environments to optimize performance. Developers must not only guarantee that the systems they themselves write will run correctly on multicore processors, but they must also ensure that applications written for single-core processors will not fail or cause other applications to fail when they are migrated to a multicore environment. To complicate matters further, these new and legacy applications can contain third-party code, which developers may not be able to change.