DOCSLIB.ORG

The Status of the Preempt-RT Patch Embedded Linux Conference Europe 2017

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
The Status of the Preempt-RT Patch Embedded Linux Conference Europe 2017 The Status of the Preempt-RT Patch Embedded Linux Conference Europe 2017 Sebastian A. Siewior Linutronix GmbH October 24, 2017 Sebastian A. Siewior Linutronix GmbH 1/35 1 Overview 2 The patchqueue 3 Summary Sebastian A. Siewior Linutronix GmbH 2/35 1 Overview 2 The patchqueue 3 Summary Sebastian A. Siewior Linutronix GmbH 3/35 What is PREEMPT-RT A patchset provided for certain kernels Provides realtime functionality Preempts softirq, spinlocks, IRQ-off regions “Is super deterministic, not super fast” Sebastian A. Siewior Linutronix GmbH 4/35 PREEMPT-RT Project was without funds for a few years LF announced RTL Collaborative Project in October 2015 Documentation https://rt.wiki.kernel.org https://wiki.linuxfoundation.org/realtime/start Testing https://ci-rt.linutronix.de Sebastian A. Siewior Linutronix GmbH 5/35 1 Overview 2 The patchqueue 3 Summary Sebastian A. Siewior Linutronix GmbH 6/35 v3.10 450 400 350 300 250 200 150 100 50 0 v3.10.4-rt1 v3.10.6-rt2 v3.10.6-rt3 v3.10.9-rt4 v3.10.9-rt5 v3.10.10-rt6 v3.10.10-rt7 v3.10.14-rt8 v3.10.14-rt9 v3.10.15-rt10 v3.10.15-rt11 v3.10.17-rt12 v3.10.18-rt13 v3.10.18-rt14 all upstream RT only Sebastian A. Siewior Linutronix GmbH 7/35 8/35 Linutronix GmbH v3.12 450 400 350 300 250 200 150 100 50 0 v3.12.0-rt1 v3.12.0-rt2 v3.12.1-rt3 v3.12.1-rt4 v3.12.5-rt5 v3.12.5-rt6 v3.12.5-rt7 v3.12.6-rt8 v3.12.6-rt9 v3.12.8-rt10 v3.12.8-rt11 v3.12.9-rt12 v3.12.9-rt13 v3.12.10-rt14 v3.12.10-rt15 v3.12.11-rt16 v3.12.11-rt17 v3.12.12-rt18 v3.12.12-rt19 v3.12.13-rt20 v3.12.13-rt21 v3.12.14-rt22 v3.12.14-rt23 v3.12.15-rt24 v3.12.15-rt25 all upstream RT only Sebastian A. Siewior v3.14 450 400 350 300 250 200 150 100 50 0 v3.14.0-rt1 v3.14.2-rt2 v3.14.2-rt3 v3.14.3-rt4 v3.14.3-rt5 all upstream RT only Sebastian A. Siewior Linutronix GmbH 9/35 v3.18 450 400 350 300 250 200 150 100 50 0 v3.18.7-rt1 v3.18.7-rt2 v3.18.9-rt3 v3.18.9-rt4 v3.18.9-rt5 v3.18.11-rt6 v3.18.11-rt7 v3.18.13-rt8 v3.18.13-rt9 v3.18.13-rt10 all upstream RT only Sebastian A. Siewior Linutronix GmbH 10/35 v4.0 450 400 350 300 250 200 150 100 50 0 v4.0.4-rt1 v4.0.5-rt2 v4.0.5-rt3 v4.0.5-rt4 v4.0.8-rt5 v4.0.8-rt6 all upstream RT only Sebastian A. Siewior Linutronix GmbH 11/35 v4.1 450 400 350 300 250 200 150 100 50 0 v4.1.2-rt1 v4.1.3-rt2 v4.1.3-rt3 v4.1.5-rt4 v4.1.5-rt5 v4.1.6-rt6 v4.1.7-rt7 v4.1.7-rt8 v4.1.10-rt10 v4.1.10-rt11 v4.1.12-rt12 v4.1.12-rt13 v4.1.13-rt14 v4.1.13-rt15 v4.1.15-rt16 v4.1.15-rt17 all upstream RT only Sebastian A. Siewior Linutronix GmbH 12/35 v4.1 450 400 350 300 250 200 150 100 TI GPIO rework 50 0 v4.1.2-rt1 v4.1.3-rt2 v4.1.3-rt3 v4.1.5-rt4 v4.1.5-rt5 v4.1.6-rt6 v4.1.7-rt7 v4.1.7-rt8 v4.1.10-rt10 v4.1.10-rt11 v4.1.12-rt12 v4.1.12-rt13 v4.1.13-rt14 v4.1.13-rt15 v4.1.15-rt16 v4.1.15-rt17 all upstream RT only Sebastian A. Siewior Linutronix GmbH 13/35 14/35 Linutronix GmbH v4.4 450 400 350 300 250 200 150 100 50 0 v4.4-rc6-rt1 v4.4-rt2 v4.4-rt3 v4.4.1-rt4 v4.4.1-rt5 v4.4.1-rt6 v4.4.2-rt7 v4.4.3-rt8 v4.4.3-rt9 v4.4.4-rt10 v4.4.4-rt11 v4.4.6-rt12 v4.4.6-rt13 v4.4.6-rt14 v4.4.7-rt15 v4.4.7-rt16 v4.4.9-rt17 v4.4.12-rt18 v4.4.12-rt19 all upstream RT only Sebastian A. Siewior 15/35 Linutronix GmbH v4.4 450 400 350 300 250 200 150 100 AT91, SWAIT in TIP 50 0 v4.4-rc6-rt1 v4.4-rt2 v4.4-rt3 v4.4.1-rt4 v4.4.1-rt5 v4.4.1-rt6 v4.4.2-rt7 v4.4.3-rt8 v4.4.3-rt9 v4.4.4-rt10 v4.4.4-rt11 v4.4.6-rt12 v4.4.6-rt13 v4.4.6-rt14 v4.4.7-rt15 v4.4.7-rt16 v4.4.9-rt17 v4.4.12-rt18 v4.4.12-rt19 all upstream RT only Sebastian A. Siewior v4.6 450 400 350 300 250 200 150 100 50 0 v4.6-rc7-rt1 v4.6.1-rt2 v4.6.1-rt3 v4.6.2-rt4 v4.6.2-rt5 v4.6.4-rt6 v4.6.4-rt7 v4.6.4-rt8 v4.6.5-rt9 v4.6.5-rt10 v4.6.7-rt11 v4.6.7-rt12 v4.6.7-rt13 v4.6.7-rt14 all upstream RT only Sebastian A. Siewior Linutronix GmbH 16/35 v4.6 450 400 350 300 250 200 150 100 Timer wheel rework TIP 50 0 v4.6-rc7-rt1 v4.6.1-rt2 v4.6.1-rt3 v4.6.2-rt4 v4.6.2-rt5 v4.6.4-rt6 v4.6.4-rt7 v4.6.4-rt8 v4.6.5-rt9 v4.6.5-rt10 v4.6.7-rt11 v4.6.7-rt12 v4.6.7-rt13 v4.6.7-rt14 all upstream RT only Sebastian A. Siewior Linutronix GmbH 17/35 v4.8 450 400 350 300 250 200 150 100 50 0 v4.8-rt1 v4.8.2-rt2 v4.8.2-rt3 v4.8.6-rt4 v4.8.6-rt5 v4.8.11-rt6 v4.8.11-rt7 v4.8.14-rt8 v4.8.14-rt9 v4.8.15-rt10 all upstream RT only Sebastian A. Siewior Linutronix GmbH 18/35 19/35 Linutronix GmbH v4.9 450 400 350 300 250 200 150 100 50 0 v4.9-rt1 v4.9.4-rt2 v4.9.6-rt3 v4.9.6-rt4 v4.9.9-rt5 v4.9.9-rt6 v4.9.10-rt7 v4.9.11-rt8 v4.9.11-rt9 v4.9.13-rt10 v4.9.13-rt11 v4.9.13-rt12 v4.9.18-rt13 v4.9.18-rt14 v4.9.20-rt15 v4.9.20-rt16 v4.9.27-rt17 v4.9.27-rt18 v4.9.30-rt19 v4.9.30-rt20 v4.9.30-rt21 v4.9.33-rt22 v4.9.33-rt23 all upstream RT only Sebastian A. Siewior 20/35 Linutronix GmbH v4.9 450 400 350 300 250 200 150 100 FUTEX rework More FUTEX 50 0 v4.9-rt1 v4.9.4-rt2 v4.9.6-rt3 v4.9.6-rt4 v4.9.9-rt5 v4.9.9-rt6 v4.9.10-rt7 v4.9.11-rt8 v4.9.11-rt9 v4.9.13-rt10 v4.9.13-rt11 v4.9.13-rt12 v4.9.18-rt13 v4.9.18-rt14 v4.9.20-rt15 v4.9.20-rt16 v4.9.27-rt17 v4.9.27-rt18 v4.9.30-rt19 v4.9.30-rt20 v4.9.30-rt21 v4.9.33-rt22 v4.9.33-rt23 all upstream RT only Sebastian A. Siewior v4.11 450 400 350 300 250 200 150 100 50 0 v4.11.5-rt1 v4.11.7-rt2 v4.11.7-rt3 v4.11.8-rt4 v4.11.8-rt5 v4.11.9-rt6 v4.11.9-rt7 v4.11.12-rt8 v4.11.12-rt9 v4.11.12-rt10 v4.11.12-rt11 v4.11.12-rt12 v4.11.12-rt13 v4.11.12-rt14 v4.11.12-rt15 v4.11.12-rt16 all upstream RT only Sebastian A. Siewior Linutronix GmbH 21/35 v4.11 450 400 350 300 250 200 HR-Timer rework CPU-Hotplug rework 150 Inter-event tracing 100 50 0 v4.11.5-rt1 v4.11.7-rt2 v4.11.7-rt3 v4.11.8-rt4 v4.11.8-rt5 v4.11.9-rt6 v4.11.9-rt7 v4.11.12-rt8 v4.11.12-rt9 v4.11.12-rt10 v4.11.12-rt11 v4.11.12-rt12 v4.11.12-rt13 v4.11.12-rt14 v4.11.12-rt15 v4.11.12-rt16 all upstream RT only Sebastian A. Siewior Linutronix GmbH 22/35 v4.13 450 400 350 300 250 200 150 100 50 0 v4.13.7-rt1 v4.13.7-rt1 all upstream RT only Sebastian A. Siewior Linutronix GmbH 23/35 24/35 Linutronix GmbH Summary 450 400 350 300 250 200 150 100 50 0 v3.10.4-rt1 v3.10.18-rt14 v3.12.0-rt1 v3.12.15-rt25 v3.14.0-rt1 v3.14.3-rt5 v3.18.7-rt1 v3.18.13-rt10 v4.0.4-rt1 v4.0.8-rt6 v4.1.2-rt1 v4.1.15-rt17 v4.4-rc6-rt1 v4.4.12-rt19 v4.6-rc7-rt1 v4.6.7-rt14 v4.8-rt1 v4.8.15-rt10 v4.9-rt1 v4.9.33-rt23 v4.11.5-rt1 v4.11.12-rt16 v4.13.7-rt1 all upstream RT only Sebastian A.
Load more

Recommended publications
  • Demarinis Kent Williams-King Di Jin Rodrigo Fonseca Vasileios P
    sysfilter: Automated System Call Filtering for Commodity Software Nicholas DeMarinis Kent Williams-King Di Jin Rodrigo Fonseca Vasileios P. Kemerlis Department of Computer Science Brown University Abstract This constant stream of additional functionality integrated Modern OSes provide a rich set of services to applications, into modern applications, i.e., feature creep, not only has primarily accessible via the system call API, to support the dire effects in terms of security and protection [1, 71], but ever growing functionality of contemporary software. How- also necessitates a rich set of OS services: applications need ever, despite the fact that applications require access to part of to interact with the OS kernel—and, primarily, they do so the system call API (to function properly), OS kernels allow via the system call (syscall) API [52]—in order to perform full and unrestricted use of the entire system call set. This not useful tasks, such as acquiring or releasing memory, spawning only violates the principle of least privilege, but also enables and terminating additional processes and execution threads, attackers to utilize extra OS services, after seizing control communicating with other programs on the same or remote of vulnerable applications, or escalate privileges further via hosts, interacting with the filesystem, and performing I/O and exploiting vulnerabilities in less-stressed kernel interfaces. process introspection. To tackle this problem, we present sysfilter: a binary Indicatively, at the time of writing, the Linux
    [Show full text]
  • Thread Evolution Kit for Optimizing Thread Operations on CE/Iot Devices
    Thread Evolution Kit for Optimizing Thread Operations on CE/IoT Devices Geunsik Lim , Student Member, IEEE, Donghyun Kang , and Young Ik Eom Abstract—Most modern operating systems have adopted the the threads running on CE/IoT devices often unintentionally one-to-one thread model to support fast execution of threads spend a significant amount of time in taking the CPU resource in both multi-core and single-core systems. This thread model, and the frequency of context switch rapidly increases due to which maps the kernel-space and user-space threads in a one- to-one manner, supports quick thread creation and termination the limited system resources, degrading the performance of in high-performance server environments. However, the perfor- the system significantly. In addition, since CE/IoT devices mance of time-critical threads is degraded when multiple threads usually have limited memory space, they may suffer from the are being run in low-end CE devices with limited system re- segmentation fault [16] problem incurred by memory shortages sources. When a CE device runs many threads to support diverse as the number of threads increases and they remain running application functionalities, low-level hardware specifications often lead to significant resource contention among the threads trying for a long time. to obtain system resources. As a result, the operating system Some engineers have attempted to address the challenges encounters challenges, such as excessive thread context switching of IoT environments such as smart homes by using better overhead, execution delay of time-critical threads, and a lack of hardware specifications for CE/IoT devices [3], [17]–[21].
    [Show full text]
  • Red Hat Enterprise Linux for Real Time 7 Tuning Guide
    Red Hat Enterprise Linux for Real Time 7 Tuning Guide Advanced tuning procedures for Red Hat Enterprise Linux for Real Time Radek Bíba David Ryan Cheryn Tan Lana Brindley Alison Young Red Hat Enterprise Linux for Real Time 7 Tuning Guide Advanced tuning procedures for Red Hat Enterprise Linux for Real Time Radek Bíba Red Hat Customer Content Services [email protected] David Ryan Red Hat Customer Content Services [email protected] Cheryn Tan Red Hat Customer Content Services Lana Brindley Red Hat Customer Content Services Alison Young Red Hat Customer Content Services Legal Notice Copyright © 2015 Red Hat, Inc. This document is licensed by Red Hat under the Creative Commons Attribution-ShareAlike 3.0 Unported License. If you distribute this document, or a modified version of it, you must provide attribution to Red Hat, Inc. and provide a link to the original. If the document is modified, all Red Hat trademarks must be removed. Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law. Red Hat, Red Hat Enterprise Linux, the Shadowman logo, JBoss, MetaMatrix, Fedora, the Infinity Logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries. Linux ® is the registered trademark of Linus Torvalds in the United States and other countries. Java ® is a registered trademark of Oracle and/or its affiliates. XFS ® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
    [Show full text]
  • SUSE Linux Enterprise Server 12 SP4 System Analysis and Tuning Guide System Analysis and Tuning Guide SUSE Linux Enterprise Server 12 SP4
    SUSE Linux Enterprise Server 12 SP4 System Analysis and Tuning Guide System Analysis and Tuning Guide SUSE Linux Enterprise Server 12 SP4 An administrator's guide for problem detection, resolution and optimization. Find how to inspect and optimize your system by means of monitoring tools and how to eciently manage resources. Also contains an overview of common problems and solutions and of additional help and documentation resources. Publication Date: September 24, 2021 SUSE LLC 1800 South Novell Place Provo, UT 84606 USA https://documentation.suse.com Copyright © 2006– 2021 SUSE LLC and contributors. All rights reserved. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or (at your option) version 1.3; with the Invariant Section being this copyright notice and license. A copy of the license version 1.2 is included in the section entitled “GNU Free Documentation License”. For SUSE trademarks, see https://www.suse.com/company/legal/ . All other third-party trademarks are the property of their respective owners. Trademark symbols (®, ™ etc.) denote trademarks of SUSE and its aliates. Asterisks (*) denote third-party trademarks. All information found in this book has been compiled with utmost attention to detail. However, this does not guarantee complete accuracy. Neither SUSE LLC, its aliates, the authors nor the translators shall be held liable for possible errors or the consequences thereof. Contents About This Guide xii 1 Available Documentation xiii
    [Show full text]
  • Greg Kroah-Hartman [email protected] Github.Com/Gregkh/Presentation-Kdbus
    kdbus IPC for the modern world Greg Kroah-Hartman [email protected] github.com/gregkh/presentation-kdbus Interprocess Communication ● signal ● synchronization ● communication standard signals realtime The Linux Programming Interface, Michael Kerrisk, page 878 POSIX semaphore futex synchronization named eventfd unnamed semaphore System V semaphore “record” lock file lock file lock mutex threads condition variables barrier read/write lock The Linux Programming Interface, Michael Kerrisk, page 878 data transfer pipe communication FIFO stream socket pseudoterminal POSIX message queue message System V message queue memory mapping System V shared memory POSIX shared memory shared memory memory mapping Anonymous mapping mapped file The Linux Programming Interface, Michael Kerrisk, page 878 Android ● ashmem ● pmem ● binder ashmem ● POSIX shared memory for the lazy ● Uses virtual memory ● Can discard segments under pressure ● Unknown future pmem ● shares memory between kernel and user ● uses physically contigous memory ● GPUs ● Unknown future binder ● IPC bus for Android system ● Like D-Bus, but “different” ● Came from system without SysV types ● Works on object / message level ● Needs large userspace library ● NEVER use outside an Android system binder ● File descriptor passing ● Used for Intents and application separation ● Good for small messages ● Not for streams of data ● NEVER use outside an Android system QNX message passing ● Tight coupling to microkernel ● Send message and control, to another process ● Used to build complex messages
    [Show full text]
  • Futexes Are Tricky
    Futexes Are Tricky Ulrich Drepper Red Hat, Inc. [email protected] January 31, 2008 Abstract Starting with early version of the 2.5 series, the Linux kernel contains a light-weight method for process synchronization. It is used in the modern thread library implementation but is also useful when used directly. This article introduces the concept and user level code to use them. 1 Preface addr1. It has a size of 4 bytes on all platforms, 32-bit and 64-bit. The value of the variable is fully under the control 1 The base reference for futexes has been “Fuss, Futexes of the application. No value has a specific meaning. and Furwocks: Fast User Level Locking in Linux” writ- ten by Franke, Russell, and Kirkwood, released in the Any memory address in regular memory (excluding some- proceedings of the 2002 OLS [1]. This document is still thing like DMA areas etc) can be used for the futex. The mostly valid. But the kernel functionality got extended only requirement is that the variable is aligned at a mul- and generally improved. The biggest weakness, though, tiple of sizeof(int). is the lack of instruction on how to use futexes correctly. Rusty Russell distributes a package containing user level It is not obvious from the prototype, but the kernel han- code (ftp://ftp.kernel.org/pub/linux/kernel/people/rusty/) dles the actual physical addresses of the futexes. I.e., if but unfortunately this code is not very well documented two processes reference a futex in a memory region they and worse, as of this writing the code is actually incor- share, they will reference the same futex object.
    [Show full text]
  • Petalinux Tools Documentation: Reference Guide
    See all versions of this document PetaLinux Tools Documentation Reference Guide UG1144 (v2021.1) June 16, 2021 Revision History Revision History The following table shows the revision history for this document. Section Revision Summary 06/16/2021 Version 2021.1 Chapter 7: Customizing the Project Added a new section: Configuring UBIFS Boot. Chapter 5: Booting and Packaging Updated Steps to Boot a PetaLinux Image on Hardware with SD Card. Appendix A: Migration Added FPGA Manager Changes, Yocto Recipe Name Changes, Host GCC Version Upgrade. Chapter 10: Advanced Configurations Updated U-Boot Configuration and Image Packaging Configuration. UG1144 (v2021.1) June 16, 2021Send Feedback www.xilinx.com PetaLinux Tools Documentation Reference Guide 2 Table of Contents Revision History...............................................................................................................2 Chapter 1: Overview.................................................................................................... 8 Introduction................................................................................................................................. 8 Navigating Content by Design Process.................................................................................... 9 Chapter 2: Setting Up Your Environment...................................................... 11 Installation Steps.......................................................................................................................11 PetaLinux Working Environment Setup................................................................................
    [Show full text]
  • Systemtap Beginners Guide
    Red Hat Enterprise Linux 7 SystemTap Beginners Guide Introduction to SystemTap Last Updated: 2020-09-29 Red Hat Enterprise Linux 7 SystemTap Beginners Guide Introduction to SystemTap William Cohen Red Hat Software Engineering [email protected] Don Domingo Red Hat Customer Content Services Vladimír Slávik Red Hat Customer Content Services [email protected] Robert Kratky Red Hat Customer Content Services Jacquelynn East Red Hat Customer Content Services Legal Notice Copyright © 2019 Red Hat, Inc. This document is licensed by Red Hat under the Creative Commons Attribution-ShareAlike 3.0 Unported License. If you distribute this document, or a modified version of it, you must provide attribution to Red Hat, Inc. and provide a link to the original. If the document is modified, all Red Hat trademarks must be removed. Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law. Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries. Linux ® is the registered trademark of Linus Torvalds in the United States and other countries. Java ® is a registered trademark of Oracle and/or its affiliates. XFS ® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries. MySQL ® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
    [Show full text]
  • How to Run POSIX Apps in a Minimal Picoprocess Jon Howell, Bryan Parno, John R
    How to Run POSIX Apps in a Minimal Picoprocess Jon Howell, Bryan Parno, John R. Douceur Microsoft Research, Redmond, WA Abstract Libraries We envision a future where Web, mobile, and desktop Application Function # Examples applications are delivered as isolated, complete software Abiword word processor 63 Pango,Freetype stacks to a minimal, secure client host. This shift imbues Gimp raster graphics 55 Gtk,Gdk Gnucash personal finances 101 Gnome,Enchant app vendors with full autonomy to maintain their apps’ Gnumeric spreadsheet 54 Gtk,Gdk integrity. Achieving this goal requires shifting complex Hyperoid video game 6 svgalib behavior out of the client platform and into the vendors’ Inkscape vector drawing 96 Magick,Gnome isolated apps. We ported rich, interactive POSIX apps, Marble 3D globe 73 KDE, Qt such as Gimp and Inkscape, to a spartan host platform. Midori HTML/JS renderer 74 webkit We describe this effort in sufficient detail to support re- producibility. Table 1: A variety of rich, functional apps transplanted to run in a minimal native picoprocess. While these 1 Introduction apps are nearly fully functional, plugins that depend on fork() are not yet supported (§3.9). Numerous academic systems [5, 11, 13, 15, 19, 22, 25–28, 31] and deployed systems [1–3, 23] have started pushing towards a world in which Web, mobile, and multaneously [16]. It pushes the minimal client host in- desktop applications are strongly isolated by the client terface to an extreme, proposing a client host without kernel. A common theme in this work is that guarantee- TCP, a file system or even storage, and with a UI con- ing strong isolation requires simplifying the client, since strained to simple pixel blitting (i.e., copying pixel arrays complexity tends to breed vulnerability.
    [Show full text]
  • Thread Synchronization: Implementation
    Operating Systems Thread Synchronization: Implementation Thomas Ropars [email protected] 2020 1 References The content of these lectures is inspired by: • The lecture notes of Prof. Andr´eSchiper. • The lecture notes of Prof. David Mazi`eres. • Operating Systems: Three Easy Pieces by R. Arpaci-Dusseau and A. Arpaci-Dusseau Other references: • Modern Operating Systems by A. Tanenbaum • Operating System Concepts by A. Silberschatz et al. 2 Agenda Reminder Goals of the lecture Mutual exclusion: legacy solutions Atomic operations Spinlocks Sleeping locks About priorities 3 Agenda Reminder Goals of the lecture Mutual exclusion: legacy solutions Atomic operations Spinlocks Sleeping locks About priorities 4 Previous lecture Concurrent programming requires thread synchronization. The problem: Threads executing on a shared-memory (multi-)processor is an asynchronous system. • A thread can be preempted at any time. • Reading/writing a data in memory incurs unpredictable delays (data in L1 cache vs page fault). 5 Previous lecture Classical concurrent programming problems • Mutual exclusion • Producer-consumer Concepts related to concurrent programming • Critical section • Deadlock • Busy waiting Synchronization primitives • Locks • Condition variables • Semaphores 6 Agenda Reminder Goals of the lecture Mutual exclusion: legacy solutions Atomic operations Spinlocks Sleeping locks About priorities 7 High-level goals How to implement synchronization primitives? Answering this question is important to: • Better understand the semantic of the primitives • Learn about the interactions with the OS • Learn about the functioning of memory • Understand the trade-offs between different solutions 8 Content of the lecture Solutions to implement mutual exclusion • Peterson's algorithm • Spinlocks • Sleeping locks Basic mechanisms used for synchronization • Atomic operations (hardware) • Futex (OS) 9 Agenda Reminder Goals of the lecture Mutual exclusion: legacy solutions Atomic operations Spinlocks Sleeping locks About priorities 10 A shared counter (remember .
    [Show full text]
  • Leveraging Linux Kernel Tracing to Classify and Detail Application Bottlenecks
    MEng Individual Project Imperial College London Department of Computing Leveraging Linux kernel tracing to classify and detail application bottlenecks Supervisor: Author: Dr. Anthony Field Ashley J Davies-Lyons Second Marker: Dr. Giuliano Casale June 17, 2019 2 Abstract GAPP is a bottleneck identification tool that uses Linux kernel probes to identify periods of reduced parallelism in multithreaded programs. Although GAPP is effective at identifiying the lines of source code that lead to a bottleneck, it is unable to classify the type of bottleneck - for example whether it is due to lock contention or I/O. This project solves this problem by augmenting the stack traces generated by GAPP with classifications, and adds details of any files that were interacted with, or IP addresses that were interacted with. Additionally, by tracking kernel-level synchronisation (‘futex’) calls, we develop a lock analysis feature that assists with identifying particularly critical locks (and unlockers) in user applications. Further, we provide a summary of the most critical individual file and synchronisation actions. In the spirit of GAPP, we implement this without requiring instrumentation, and does not introduce any language or library dependencies. We find that our extended tool is able to reliably classify the different categories of bottleneck, and adds useful information to the GAPP output which is useful in diagnosing the root causes of a bottleneck. We verify this with two large open source projects - an image tracking benchmark, and a production game server. Finally, we find that the overhead we add is competitive with similar tools, and that our tool works correctly with alternative threading library - having evaluated with TBB and pthreads In addition to our main contributions, we additionally add a number of quality-of-life improvements to the tool, including a user interface to present the data, improved stack trace reporting, and easier methods of attaching to user applications.
    [Show full text]
  • Glibc and System Call Wrappers
    glibc and system call wrappers Florian Weimer, Red Hat Platform Tools Team Virtual Linux Plumbers, 2020-08-28 1 / 17 Outline I Why do we have system call wrappers? I How can we add them to glibc? I Do we actually want to do that? I What can the kernel do to make things easier? I Poll: What do you work on? I A: kernel, B: userspace toolchain (compiler, core libraries), C: other userspace, D: something else 2 / 17 What are system call wrappers? I off64_t lseek(int fd, off64_t off, int whence); lseek: movl $8, %eax /* syscall number */ syscall cmpq $-4096, %rax /* magic errno limit */ ja 1f /* handle error */ ret 1: movq __libc_errno@gottpoff(%rip), %rdx negl %eax movl %eax, %fs:(%rdx) /* update errno TLS */ movq $-1, %rax ret 3 / 17 Why bother? I Can we just use a generic wrapper? I syscall(__NR_lseek, fd, 0, SEEK_SET); 4 / 17 Why bother? Portability! I Need to use the correct types: syscall(__NR_lseek, fd, (off64_t) 0, SEEK_SET); I Need to use the correct system call: off64_t off; syscall(__NR__llseek, fd, 0L, 0L, &off, SEEK_SET); I This is more common than you would think (open ! openat, futex ! futex_time64). 5 / 17 glibc lseek (without symbol management) off64_t lseek (int fd, off64_t offset, int whence) { #ifdef __NR__llseek loff_t res; int rc = INLINE_SYSCALL_CALL (_llseek, fd, (long) (((uint64_t) (offset)) >> 32), (long) offset, &res, whence); return rc ? rc : res; #else return INLINE_SYSCALL_CALL (lseek, fd, offset, whence); #endif } 6 / 17 glibc implementation options I C with INLINE_SYSCALL_CALL: automatic errno handling I C with
    [Show full text]