Linux Kernel Series
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Development of a Verified Flash File System ⋆
Development of a Verified Flash File System ? Gerhard Schellhorn, Gidon Ernst, J¨orgPf¨ahler,Dominik Haneberg, and Wolfgang Reif Institute for Software & Systems Engineering University of Augsburg, Germany fschellhorn,ernst,joerg.pfaehler,haneberg,reifg @informatik.uni-augsburg.de Abstract. This paper gives an overview over the development of a for- mally verified file system for flash memory. We describe our approach that is based on Abstract State Machines and incremental modular re- finement. Some of the important intermediate levels and the features they introduce are given. We report on the verification challenges addressed so far, and point to open problems and future work. We furthermore draw preliminary conclusions on the methodology and the required tool support. 1 Introduction Flaws in the design and implementation of file systems already lead to serious problems in mission-critical systems. A prominent example is the Mars Explo- ration Rover Spirit [34] that got stuck in a reset cycle. In 2013, the Mars Rover Curiosity also had a bug in its file system implementation, that triggered an au- tomatic switch to safe mode. The first incident prompted a proposal to formally verify a file system for flash memory [24,18] as a pilot project for Hoare's Grand Challenge [22]. We are developing a verified flash file system (FFS). This paper reports on our progress and discusses some of the aspects of the project. We describe parts of the design, the formal models, and proofs, pointing out challenges and solutions. The main characteristic of flash memory that guides the design is that data cannot be overwritten in place, instead space can only be reused by erasing whole blocks. -
Vorlesung-Print.Pdf
1 Betriebssysteme Prof. Dipl.-Ing. Klaus Knopper (C) 2019 <[email protected]> Live GNU/Linux System Schwarz: Transparent,KNOPPIX CD−Hintergrundfarbe (silber) bei Zweifarbdruck, sonst schwarz. Vorlesung an der DHBW Karlsruhe im Sommersemester 2019 Organisatorisches + Vorlesung mit Ubungen¨ Betriebssysteme WWI17B2 jeweils Montags (einzelne Termine) in A369 + http://knopper.net/bs/ (spater¨ moodle) Folie 1 Kursziel µ Grundsatzlichen¨ Aufbau von Betriebssystemen in Theorie und Praxis kennen und verstehen, µ grundlegende Konzepte von Multitasking, Multiuser-Betrieb und Hardware-Unterstutzung¨ / Resource-Sharing erklaren¨ konnen,¨ µ Sicherheitsfragen und Risiken des Ubiquitous und Mobile Computing auf Betriebssystemebene analysieren, µ mit heterogenen Betriebssystemumgebungen und Virtua- lisierung arbeiten, Kompatibilitatsprobleme¨ erkennen und losen.¨ Folie 2 0 Themen (Top-Down) + Ubersicht¨ Betriebssysteme und Anwendungen, Unterschiede in Aufbau und Einsatz, Lizenzen, Distributionen, + GNU/Linux als OSS-Lernsystem fur¨ die Vorlesung, Tracing und Analyse des Bootvorgangs, + User Interface(s), + Dateisystem: VFS, reale Implementierungen, + Multitasking: Scheduler, Interrupts, Speicherverwaltung (VM), Prozessverwaltung (Timesharing), + Multiuser: Benutzerverwaltung, Rechtesystem, + Hardware-Unterstutzung:¨ Kernel und Module vs. Treiber“ - Kon- ” zept, + Kompatibilitat,¨ API-Emulation, Virtualisierung, Softwareentwick- lung. + Sicherheits-Aspekte von Betriebssystemen, Schadsoftware“ und ” forensische Analyse bei Kompromittierung oder Datenverlust. -
DASH: Database Shadowing for Mobile DBMS
DASH: Database Shadowing for Mobile DBMS Youjip Won1 Sundoo Kim2 Juseong Yun2 Dam Quang Tuan2 Jiwon Seo2 1KAIST, Daejeon, Korea 2Hanyang University, Seoul, Korea [email protected] [email protected] ABSTRACT 1. INTRODUCTION In this work, we propose Database Shadowing, or DASH, Crash recovery is a vital part of DBMS design. Algorithms which is a new crash recovery technique for SQLite DBMS. for crash recovery range from naive full-file shadowing [15] DASH is a hybrid mixture of classical shadow paging and to the sophisticated ARIES protocol [38]. Most enterprise logging. DASH addresses four major issues in the current DBMS's, e.g., IBM DB2, Informix, Micrsoft SQL and Oracle SQLite journal modes: the performance and write amplifi- 8, use ARIES or its variants for efficient concurrency control. cation issues of the rollback mode and the storage space re- SQLite is one of the most widely used DBMS's. It is quirement and tail latency issues of the WAL mode. DASH deployed on nearly all computing platform such as smart- exploits two unique characteristics of SQLite: the database phones (e.g, Android, Tizen, Firefox, and iPhone [52]), dis- files are small and the transactions are entirely serialized. tributed filesystems (e.g., Ceph [58] and Gluster filesys- DASH consists of three key ingredients Aggregate Update, tem [1]), wearable devices (e.g., smart watch [4, 21]), and Atomic Exchange and Version Reset. Aggregate Update elim- automobiles [19, 55]. As a library-based embedded DBMS, inates the redundant write overhead and the requirement to SQLite deliberately adopts a basic transaction management maintain multiple snapshots both of which are inherent in and crash recovery scheme. -
Redbooks Paper Linux on IBM Zseries and S/390
Redbooks Paper Simon Williams Linux on IBM zSeries and S/390: TCP/IP Broadcast on z/VM Guest LAN Preface This Redpaper provides information to help readers plan for and exploit Internet Protocol (IP) broadcast support that was made available to z/VM Guest LAN environments with the introduction of the z/VM 4.3 Operating System. Using IP broadcast support, Linux guests can for the first time use DHCP to lease an IP address dynamically from a DHCP server in a z/VM Guest LAN environment. This frees the administrator from the previous method of having to hardcode an IP address for every Linux guest in the system. This new feature enables easier deployment and administration of large-scale Linux environments. Objectives The objectives of this paper are to: Review the z/VM Guest LAN environment Explain IP broadcast Introduce the Dynamic Host Configuration Protocol (DHCP) Explain how DHCP works in a z/VM Guest LAN Describe how to implement DHCP in a z/VM Guest LAN environment © Copyright IBM Corp. 2003. All rights reserved. ibm.com/redbooks 1 z/VM Guest LAN Attention: While broadcast support for z/VM Guest LANs was announced with the base z/VM 4.3 operating system, the user must apply the PTF for APAR VM63172. This APAR resolves several issues which have been found to inhibit the use of DHCP by Linux-based applications running over the z/VM Guest LAN (in simulated QDIO mode). Introduction Prior to z/VM 4.2, virtual connectivity options for connecting one or more virtual machines (VM guests) was limited to virtual channel-to-channel adapters (CTCA) and the Inter-User Communications Vehicle (IUCV) facility. -
Improving the Reliability of Commodity Operating Systems
Improving the Reliability of Commodity Operating Systems MICHAEL M. SWIFT, BRIAN N. BERSHAD, and HENRY M. LEVY University of Washington Despite decades of research in extensible operating system technology, extensions such as device drivers remain a significant cause of system failures. In Windows XP, for example, drivers account for 85% of recently reported failures. This paper describes Nooks, a reliability subsystem that seeks to greatly enhance OS reliability by isolating the OS from driver failures. The Nooks approach is practical: rather than guaranteeing complete fault tolerance through a new (and incompatible) OS or driver architecture, our goal is to prevent the vast majority of driver-caused crashes with little or no change to existing driver and system code. Nooks isolates drivers within lightweight protection domains inside the kernel address space, where hardware and software prevent them from corrupting the kernel. Nooks also tracks a driver’s use of kernel resources to facilitate automatic clean-up during recovery. To prove the viability of our approach, we implemented Nooks in the Linux operating system and used it to fault-isolate several device drivers. Our results show that Nooks offers a substantial increase in the reliability of operating systems, catching and quickly recovering from many faults that would otherwise crash the system. Under a wide range and number of fault conditions, we show that Nooks recovers automatically from 99% of the faults that otherwise cause Linux to crash. While Nooks was designed for drivers, our techniques generalize to other kernel extensions. We demonstrate this by isolating a kernel-mode file system and an in-kernel Internet service. -
ZBD: Using Transparent Compression at the Block Level to Increase Storage Space Efficiency
ZBD: Using Transparent Compression at the Block Level to Increase Storage Space Efficiency Thanos Makatos∗†, Yannis Klonatos∗†, Manolis Marazakis∗, Michail D. Flouris∗, and Angelos Bilas∗† ∗ Foundation for Research and Technology - Hellas (FORTH) Institute of Computer Science (ICS) 100 N. Plastira Av., Vassilika Vouton, Heraklion, GR-70013, Greece † Department of Computer Science, University of Crete P.O. Box 2208, Heraklion, GR 71409, Greece {mcatos, klonatos, maraz, flouris, bilas}@ics.forth.gr Abstract—In this work we examine how transparent compres- • Logical to physical block mapping: Block-level compres- sion in the I/O path can improve space efficiency for online sion imposes a many-to-one mapping from logical to storage. We extend the block layer with the ability to compress physical blocks, as multiple compressed logical blocks and decompress data as they flow between the file-system and the disk. Achieving transparent compression requires extensive must be stored in the same physical block. This requires metadata management for dealing with variable block sizes, dy- using a translation mechanism that imposes low overhead namic block mapping, block allocation, explicit work scheduling in the common I/O path and scales with the capacity and I/O optimizations to mitigate the impact of additional I/Os of the underlying devices as well as a block alloca- and compression overheads. Preliminary results show that online tion/deallocation mechanism that affects data placement. transparent compression is a viable option for improving effective • storage capacity, it can improve I/O performance by reducing Increased number of I/Os: Using compression increases I/O traffic and seek distance, and has a negative impact on the number of I/Os required on the critical path during performance only when single-thread I/O latency is critical. -
Uebayasi's Simple Template
eXecute In Place support in NetBSD Masao “uebs” Uebayashi <[email protected]> BSDCan 2010 2010.5.13 Who am I NetBSD developer Japanese Living in Yokohama Self-employed Since Dec. 15 2008 Tombi Inc. Agenda Demonstration Introduction Program execution VM Virtual memory management Physical memory management Fault handler, pager : Design of XIP Demonstration XIP on NetBSD/arm (i.MX35) Introduction What is XIP? Execute programs directly from devices No memory copy Only about userland programs (Kernel XIP is another story) Introduction Who needs XIP? Embedded devices Memory saving for less power consumption Boot time Mainframes (Linux) Memory saving for virtualized instances : “Nothing in between” Introduction How to achieve XIP? Don't copy programs to memory when executing it “Execute” == mmap() : : : : : What does that *actually* mean? Goals No hacks Keep code cleanliness Keep abstraction Including device handling : : Performance Latency Memory efficiency Program execution execve(2) → sys_execve() Prepare Read program header using I/O Map sections Set program entry point Execute Page fault is triggered Load pages using VM Execution is resumed Program execution I/O part needs no changes If block device interface (d_strategy()) is provided VM part needs changes!!! Virtual memory management http://en.wikipedia.org/wiki/Virtual_memory Virtual memory is a computer system technique which gives an application program the impression that it has contiguous working memory (an address space), while in fact it may be physically fragmented and may even overflow on to disk storage. Developed for multitasking kernels, virtual memory provides two primary functions: Each process has its own address space, thereby not required to be relocated nor required to use relative addressing mode. -
Proceedings of the Linux Symposium
Proceedings of the Linux Symposium Volume One June 27th–30th, 2007 Ottawa, Ontario Canada Contents The Price of Safety: Evaluating IOMMU Performance 9 Ben-Yehuda, Xenidis, Mostrows, Rister, Bruemmer, Van Doorn Linux on Cell Broadband Engine status update 21 Arnd Bergmann Linux Kernel Debugging on Google-sized clusters 29 M. Bligh, M. Desnoyers, & R. Schultz Ltrace Internals 41 Rodrigo Rubira Branco Evaluating effects of cache memory compression on embedded systems 53 Anderson Briglia, Allan Bezerra, Leonid Moiseichuk, & Nitin Gupta ACPI in Linux – Myths vs. Reality 65 Len Brown Cool Hand Linux – Handheld Thermal Extensions 75 Len Brown Asynchronous System Calls 81 Zach Brown Frysk 1, Kernel 0? 87 Andrew Cagney Keeping Kernel Performance from Regressions 93 T. Chen, L. Ananiev, and A. Tikhonov Breaking the Chains—Using LinuxBIOS to Liberate Embedded x86 Processors 103 J. Crouse, M. Jones, & R. Minnich GANESHA, a multi-usage with large cache NFSv4 server 113 P. Deniel, T. Leibovici, & J.-C. Lafoucrière Why Virtualization Fragmentation Sucks 125 Justin M. Forbes A New Network File System is Born: Comparison of SMB2, CIFS, and NFS 131 Steven French Supporting the Allocation of Large Contiguous Regions of Memory 141 Mel Gorman Kernel Scalability—Expanding the Horizon Beyond Fine Grain Locks 153 Corey Gough, Suresh Siddha, & Ken Chen Kdump: Smarter, Easier, Trustier 167 Vivek Goyal Using KVM to run Xen guests without Xen 179 R.A. Harper, A.N. Aliguori & M.D. Day Djprobe—Kernel probing with the smallest overhead 189 M. Hiramatsu and S. Oshima Desktop integration of Bluetooth 201 Marcel Holtmann How virtualization makes power management different 205 Yu Ke Ptrace, Utrace, Uprobes: Lightweight, Dynamic Tracing of User Apps 215 J. -
NOVA: a Log-Structured File System for Hybrid Volatile/Non
NOVA: A Log-structured File System for Hybrid Volatile/Non-volatile Main Memories Jian Xu and Steven Swanson, University of California, San Diego https://www.usenix.org/conference/fast16/technical-sessions/presentation/xu This paper is included in the Proceedings of the 14th USENIX Conference on File and Storage Technologies (FAST ’16). February 22–25, 2016 • Santa Clara, CA, USA ISBN 978-1-931971-28-7 Open access to the Proceedings of the 14th USENIX Conference on File and Storage Technologies is sponsored by USENIX NOVA: A Log-structured File System for Hybrid Volatile/Non-volatile Main Memories Jian Xu Steven Swanson University of California, San Diego Abstract Hybrid DRAM/NVMM storage systems present a host of opportunities and challenges for system designers. These sys- Fast non-volatile memories (NVMs) will soon appear on tems need to minimize software overhead if they are to fully the processor memory bus alongside DRAM. The result- exploit NVMM’s high performance and efficiently support ing hybrid memory systems will provide software with sub- more flexible access patterns, and at the same time they must microsecond, high-bandwidth access to persistent data, but provide the strong consistency guarantees that applications managing, accessing, and maintaining consistency for data require and respect the limitations of emerging memories stored in NVM raises a host of challenges. Existing file sys- (e.g., limited program cycles). tems built for spinning or solid-state disks introduce software Conventional file systems are not suitable for hybrid mem- overheads that would obscure the performance that NVMs ory systems because they are built for the performance char- should provide, but proposed file systems for NVMs either in- acteristics of disks (spinning or solid state) and rely on disks’ cur similar overheads or fail to provide the strong consistency consistency guarantees (e.g., that sector updates are atomic) guarantees that applications require. -
High Velocity Kernel File Systems with Bento
High Velocity Kernel File Systems with Bento Samantha Miller, Kaiyuan Zhang, Mengqi Chen, and Ryan Jennings, University of Washington; Ang Chen, Rice University; Danyang Zhuo, Duke University; Thomas Anderson, University of Washington https://www.usenix.org/conference/fast21/presentation/miller This paper is included in the Proceedings of the 19th USENIX Conference on File and Storage Technologies. February 23–25, 2021 978-1-939133-20-5 Open access to the Proceedings of the 19th USENIX Conference on File and Storage Technologies is sponsored by USENIX. High Velocity Kernel File Systems with Bento Samantha Miller Kaiyuan Zhang Mengqi Chen Ryan Jennings Ang Chen‡ Danyang Zhuo† Thomas Anderson University of Washington †Duke University ‡Rice University Abstract kernel-level debuggers and kernel testing frameworks makes this worse. The restricted and different kernel programming High development velocity is critical for modern systems. environment also limits the number of trained developers. This is especially true for Linux file systems which are seeing Finally, upgrading a kernel module requires either rebooting increased pressure from new storage devices and new demands the machine or restarting the relevant module, either way on storage systems. However, high velocity Linux kernel rendering the machine unavailable during the upgrade. In the development is challenging due to the ease of introducing cloud setting, this forces kernel upgrades to be batched to meet bugs, the difficulty of testing and debugging, and the lack of cloud-level availability goals. support for redeployment without service disruption. Existing Slow development cycles are a particular problem for file approaches to high-velocity development of file systems for systems. -
Container-Based Virtualization for Byte-Addressable NVM Data Storage
2016 IEEE International Conference on Big Data (Big Data) Container-Based Virtualization for Byte-Addressable NVM Data Storage Ellis R. Giles Rice University Houston, Texas [email protected] Abstract—Container based virtualization is rapidly growing Storage Class Memory, or SCM, is an exciting new in popularity for cloud deployments and applications as a memory technology with the potential of replacing hard virtualization alternative due to the ease of deployment cou- drives and SSDs as it offers high-speed, byte-addressable pled with high-performance. Emerging byte-addressable, non- volatile memories, commonly called Storage Class Memory or persistence on the main memory bus. Several technologies SCM, technologies are promising both byte-addressability and are currently under research and development, each with dif- persistence near DRAM speeds operating on the main memory ferent performance, durability, and capacity characteristics. bus. These new memory alternatives open up a new realm of These include a ReRAM by Micron and Sony, a slower, but applications that no longer have to rely on slow, block-based very large capacity Phase Change Memory or PCM by Mi- persistence, but can rather operate directly on persistent data using ordinary loads and stores through the cache hierarchy cron and others, and a fast, smaller spin-torque ST-MRAM coupled with transaction techniques. by Everspin. High-speed, byte-addressable persistence will However, SCM presents a new challenge for container-based give rise to new applications that no longer have to rely on applications, which typically access persistent data through slow, block based storage devices and to serialize data for layers of block based file isolation. -
Filesystems HOWTO Filesystems HOWTO Table of Contents Filesystems HOWTO
Filesystems HOWTO Filesystems HOWTO Table of Contents Filesystems HOWTO..........................................................................................................................................1 Martin Hinner < [email protected]>, http://martin.hinner.info............................................................1 1. Introduction..........................................................................................................................................1 2. Volumes...............................................................................................................................................1 3. DOS FAT 12/16/32, VFAT.................................................................................................................2 4. High Performance FileSystem (HPFS)................................................................................................2 5. New Technology FileSystem (NTFS).................................................................................................2 6. Extended filesystems (Ext, Ext2, Ext3)...............................................................................................2 7. Macintosh Hierarchical Filesystem − HFS..........................................................................................3 8. ISO 9660 − CD−ROM filesystem.......................................................................................................3 9. Other filesystems.................................................................................................................................3