The Aurora Operating System

Total Page:16

File Type:pdf, Size:1020Kb

The Aurora Operating System The Aurora Operating System Revisiting the Single Level Store Emil Tsalapatis Ryan Hancock Tavian Barnes RCS Lab, University of Waterloo RCS Lab, University of Waterloo RCS Lab, University of Waterloo [email protected] [email protected] [email protected] Ali José Mashtizadeh RCS Lab, University of Waterloo [email protected] ABSTRACT KEYWORDS Applications on modern operating systems manage their single level stores, transparent persistence, snapshots, check- ephemeral state in memory, and persistent state on disk. En- point/restore suring consistency between them is a source of significant developer effort, yet still a source of significant bugs inma- ACM Reference Format: ture applications. We present the Aurora single level store Emil Tsalapatis, Ryan Hancock, Tavian Barnes, and Ali José Mash- (SLS), an OS that simplifies persistence by automatically per- tizadeh. 2021. The Aurora Operating System: Revisiting the Single sisting all traditionally ephemeral application state. With Level Store. In Workshop on Hot Topics in Operating Systems (HotOS recent storage hardware like NVMe SSDs and NVDIMMs, ’21), June 1-June 3, 2021, Ann Arbor, MI, USA. ACM, New York, NY, Aurora is able to continuously checkpoint entire applications USA, 8 pages. https://doi.org/10.1145/3458336.3465285 with millisecond granularity. Aurora is the first full POSIX single level store to han- dle complex applications ranging from databases to web 1 INTRODUCTION browsers. Moreover, by providing new ways to interact with Single level storage (SLS) systems provide persistence of and manipulate application state, it enables applications to applications as an operating system service. Their advantage provide features that would otherwise be prohibitively dif- lies in removing the semantic gap between the in-memory ficult to implement. We argue that this provides strong evi- representation and the serialized on-disk representation that dence that manipulation and persistence of application state uses file IO APIs. This gap often leads to increased code naturally belong in an operating system. complexity and software bugs [13, 41]. Instead, applications solely use memory and the operating system persists this CCS CONCEPTS state to disk. Developers design programs as if they never crash and thus do not write code for persistence and recovery. • Software and its engineering ! Operating systems; • After a crash, the SLS restores the application, including Computer systems organization ! Secondary storage all state (i.e., CPU registers, OS state, and memory), which organization; Reliability; Dependable and fault-tolerant continues executing oblivious to the interruption. systems and networks; • Information systems ! Stor- SLSes have been impractical to build for decades for per- age architectures. formance reasons, but this has changed with the advent of new storage technologies. Past systems suffered from a large performance gap between memory and disks in terms of bandwidth and latency. This was compounded by write- Permission to make digital or hard copies of part or all of this work for amplification due to the tracking of memory modifications personal or classroom use is granted without fee provided that copies are at page granularity, and the overhead of CPU and OS state. not made or distributed for profit or commercial advantage and that copies Modern flash, coupled with fast PCIe Gen 4–5, has largely bear this notice and the full citation on the first page. Copyrights for third- closed the performance gap with memory. party components of this work must be honored. For all other uses, contact We introduce the Aurora Operating System, a novel non- the owner/author(s). traditional single level storage system that enables persis- HotOS ’21, June 1-June 3, 2021, Ann Arbor, MI, USA © 2021 Copyright held by the owner/author(s). tence and manipulation of execution state. Aurora is based ACM ISBN 978-1-4503-8438-4/21/05. on the FreeBSD kernel and is the first SLS that can run un- https://doi.org/10.1145/3458336.3465285 modified POSIX applications. Aurora provides persistence at HotOS ’21, June 1-June 3, 2021, Ann Arbor, MI, USA Emil Tsalapatis, Ryan Hancock, Tavian Barnes, and Ali José Mashtizadeh the granularity of process trees or containers, and supports incremental checkpointing [51] to persist applications at multi-process applications with nearly all POSIX primitives. regular intervals with runtime and/or application specific This allows for the persistence of complex applications like hooks. These systems were severely limited by the speed of Firefox, a popular web browser. storage devices at the time, e.g., the EROS research OS spent Aurora differs from previous systems in several ways. a large effort on masking spinning disk latency [45]. EROS [45] requires application cooperation to achieve per- Existing persistent-by-default designs like The Machine [31] formance and its main contributions are optimizing check- and Twizzler [17] are not transparent and depend on special pointing and swap for spinning disks. IBM’s AS/400 uses hardware. The Machine was an attempt to build a supercom- runtime and compile time hooks along with application hints puter based on memristors, while Twizzler is an OS that uses to achieve good performance [46]. only NVDIMMs for storage. These systems break compati- Aurora revisits the single level store with three main con- bility with existing systems in that they depend on the byte tributions. First, we depend on improvements in hardware to addressability of persistent storage. Single level stores like achieve performance and functionality including new flash Aurora conversely use regular DRAM and disks, and hide storage and large virtual address spaces. Second, we develop the distinction between the two from the application. an architecture designed to support both unmodified and Aurora makes a key observation that device bandwidth modified POSIX applications. Third, we expand the concept and latency has improved to rival the memory bus. Mod- of a single level store with new primitives for the manipulat- ern CPUs can provide an aggregate PCIe bandwidth up to ing execution state to enable novel applications. 256 GB/s, more than that of memory [16]. New Intel 3D Aurora also accurately captures application state by treat- XPoint SSD’s reduce IO latency to 10 `B [8], within two ing all POSIX primitives (e.g., Unix domain sockets, System orders of magnitude of memory. The combination of high V shared memory, and file descriptors) as first class objects, bandwidth and low latency makes transparent persistence rather than as parts of a process. This allows Aurora to han- possible without needing byte addressability. dle applications composed of processes that share memory or Popular checkpoint/restore mechanisms have been used files in arbitrary ways, without duplicating work or leaving for scientific computing to recover from failures and migrate edge cases unhandled. Using this approach, Aurora supports workloads [29, 42, 47]. These systems do not checkpoint complex programs like the Firefox web browser, the RocksDB frequently enough to provide transparent persistence and key value store, and the Mosh remote shell. the resulting checkpoints are not self contained. Aurora provides a system level service for manipulating Checkpointing of virtual machines (VM) has enjoyed a arbitrary application state. It goes much further than tradi- lot of popularity and applications. VMs package the applica- tional SLSes by blurring the line between applications and tion and all dependencies into a portable checkpoint. Live data. Users can operate on running applications to persist, migration and incremental checkpointing have enabled dis- copy, revert, or transfer them the same way they would a tributed resource management, fault tolerance and other file. Aurora makes state manipulation an explicit operation, applications [5, 24, 36, 38]. which programs often need to do in an ad hoc manner by Containers, which have less overhead than virtualization, themselves. Aurora creates application checkpoints that en- have traditionally lacked these features. Providing the same capsulate all information required to recreate the application, functionality for containers enables the same distributed even across reboots and machines. resource management and fault tolerance applications. Sys- We argue that application persistence and manipulation tems like CRIU [6], the standard for Linux container migra- of execution state naturally belong in the operating system, tion [40], piece together application state by querying the which enables novel applications to modern systems. Aurora kernel through system calls and the proc file system. allows us to solve a wide range of complex systems problems, While CRIU’s performance is tolerable for migration, its from reducing startup times and increasing density of server- overheads are prohibitive for other applications including less computing, to improving debugging and simplifying transparent persistence. Even research systems that optimize database design. memory checkpointing in CRIU have failed to reduce over- heads enough [50]. Furthermore, CRIU is incredibly complex, requiring 7 years to properly add UNIX socket support [7]. 2 BACKGROUND Aurora is different from previous systems in two ways. Single level stores have existed for decades both in industry First, rather than checkpointing objects exposed at the sys- and academia [20, 23, 32, 45, 46]. These systems simplified tem call boundary it
Recommended publications
  • Checkpoint and Restoration of Micro-Service in Docker Containers
    3rd International Conference on Mechatronics and Industrial Informatics (ICMII 2015) Checkpoint and Restoration of Micro-service in Docker Containers Chen Yang School of Information Security Engineering, Shanghai Jiao Tong University, China 200240 [email protected] Keywords: Lightweight Virtualization, Checkpoint/restore, Docker. Abstract. In the present days of rapid adoption of micro-service, it is imperative to build a system to support and ensure the high performance and high availability for micro-services. Lightweight virtualization, which we also called container, has the ability to run multiple isolated sets of processes under a single kernel instance. Because of the possibility of obtaining a low overhead comparable to the near-native performance of a bare server, the container techniques, such as openvz, lxc, docker, they are widely used for micro-service [1]. In this paper, we present the high availability of micro-service in containers. We investigate capabilities provided by container (docker, openvz) to model and build the Micro-service infrastructure and compare different checkpoint and restore technologies for high availability. Finally, we present preliminary performance results of the infrastructure tuned to the micro-service. Introduction Lightweight virtualization, named the operating system level virtualization technology, partitions the physical machines resource, creating multiple isolated user-space instances. Each container acts exactly like a stand-alone server. A container can be rebooted independently and have root access, users, IP address, memory, processes, files, etc. Unlike traditional virtualization with the hypervisor layer, containerization takes place at the kernel level. Most modern operating system kernels now support the primitives necessary for containerization, including Linux with openvz, vserver and more recently lxc, Solaris with zones, and FreeBSD with Jails [2].
    [Show full text]
  • Benchmarking, Analysis, and Optimization of Serverless Function Snapshots
    Benchmarking, Analysis, and Optimization of Serverless Function Snapshots Dmitrii Ustiugov∗ Plamen Petrov Marios Kogias† University of Edinburgh University of Edinburgh Microsoft Research United Kingdom United Kingdom United Kingdom Edouard Bugnion Boris Grot EPFL University of Edinburgh Switzerland United Kingdom ABSTRACT CCS CONCEPTS Serverless computing has seen rapid adoption due to its high scala- • Computer systems organization ! Cloud computing; • In- bility and flexible, pay-as-you-go billing model. In serverless, de- formation systems ! Computing platforms; Data centers; • velopers structure their services as a collection of functions, spo- Software and its engineering ! n-tier architectures. radically invoked by various events like clicks. High inter-arrival time variability of function invocations motivates the providers KEYWORDS to start new function instances upon each invocation, leading to cloud computing, datacenters, serverless, virtualization, snapshots significant cold-start delays that degrade user experience. To reduce ACM Reference Format: cold-start latency, the industry has turned to snapshotting, whereby Dmitrii Ustiugov, Plamen Petrov, Marios Kogias, Edouard Bugnion, and Boris an image of a fully-booted function is stored on disk, enabling a Grot. 2021. Benchmarking, Analysis, and Optimization of Serverless Func- faster invocation compared to booting a function from scratch. tion Snapshots . In Proceedings of the 26th ACM International Conference on This work introduces vHive, an open-source framework for Architectural Support for Programming Languages and Operating Systems serverless experimentation with the goal of enabling researchers (ASPLOS ’21), April 19–23, 2021, Virtual, USA. ACM, New York, NY, USA, to study and innovate across the entire serverless stack. Using 14 pages. https://doi.org/10.1145/3445814.3446714 vHive, we characterize a state-of-the-art snapshot-based serverless infrastructure, based on industry-leading Containerd orchestra- 1 INTRODUCTION tion framework and Firecracker hypervisor technologies.
    [Show full text]
  • Qualifikationsprofil #10309
    QUALIFIKATIONSPROFIL #10309 ALLGEMEINE DATEN Geburtsjahr: 1972 Ausbildung: Abitur Diplom, Informatik, (TU, Kaiserslautern) Fremdsprachen: Englisch, Französisch Spezialgebiete: Kubernetes KENNTNISSE Tools Active Directory Apache Application Case CATIA CVS Eclipse Exchange Framework GUI Innovator ITIL J2EE JMS LDAP Lotus Notes make MS Exchange MS Outlook MS-Exchange MS-Office MS-Visual Studio NetBeans OSGI RACF SAS sendmail Together Turbine UML VMWare .NET ADS ANT ASP ASP.NET Flash GEnie IDES Image Intellij IDEA IPC Jackson JBOSS Lex MS-Visio ODBC Oracle Application Server OWL PGP SPSS SQS TesserAct Tivoli Toolbook Total Transform Visio Weblogic WebSphere YACC Tätigkeiten Administration Analyse Beratung Design Dokumentation KI Konzeption Optimierung Support Vertrieb Sprachen Ajax Basic C C# C++ Cobol Delphi ETL Fortran Java JavaScript Natural Perl PHP PL/I PL-SQL Python SAL Smalltalk SQL ABAP Atlas Clips Delta FOCUS HTML Nomad Pascal SPL Spring TAL XML Detaillierte Komponenten AM BI FS-BA MDM PDM PM BW CO FI LO PP Datenbanken Approach IBM Microsoft Object Store Oracle Progress Sybase DMS ISAM JDBC mySQL DC/Netzwerke ATM DDS Gateway HBCI Hub Internet Intranet OpenSSL SSL VPN Asynchronous CISCO Router DNS DSL Firewall Gateways HTTP RFC Router Samba Sockets Switches Finance Business Intelligence Excel Konsolidierung Management Projektleiter Reporting Testing Wertpapiere Einkauf CAD Systeme CATIA V5 sonstige Hardware Digital HP PC Scanner Siemens Spark Teradata Bus FileNet NeXT SUN Switching Tools, Methoden Docker Go Kubernetes Rational RUP
    [Show full text]
  • Gvisor Is a Project to Restrict the Number of Syscalls That the Kernel and User Space Need to Communicate
    SED 820 Transcript EPISODE 820 [INTRODUCTION] [0:00:00.3] JM: The Linux operating system includes user space and kernel space. In user space, the user can create and interact with a variety of applications directly. In kernel space, the Linux kernel provides a stable environment in which device drivers interact with hardware and manage low-level resources. A Linux container is a virtualized environment that runs within user space. To perform an operation, a process in a container in user space makes a syscall, which is a system call into kernel space. This allows the container to have access to resources like memory and disk. Kernel space must be kept secure to ensure the operating system’s integrity. Linux includes hundreds of syscalls. Each syscall represents an interface between the user space and the kernel space. Security vulnerabilities can emerge from this wide attack surface of different syscalls and most applications only need a small number of syscalls to provide their required functionality. gVisor is a project to restrict the number of syscalls that the kernel and user space need to communicate. gVisor is a runtime layer between the user space container and the kernel space. gVisor reduces the number of syscalls that can be made into kernel space. The security properties of gVisor make it an exciting project today, but it is the portability features of gVisor that hint at a huge future opportunity. By inserting an interpreter interface between containers and the Linux kernel, gVisor presents the container world with an opportunity to run on operating systems other than Linux.
    [Show full text]
  • Surviving Software Dependencies
    practice DOI:10.1145/3347446 is additional code a programmer wants Article development led by queue.acm.org to call. Adding a dependency avoids repeating work: designing, testing, de- bugging, and maintaining a specific Software reuse is finally here unit of code. In this article, that unit of but comes with risks. code is referred to as a package; some systems use the terms library and mod- BY RUSS COX ule instead. Taking on externally written depen- dencies is not new. Most programmers have at one point in their careers had to go through the steps of manually Surviving installing a required library, such as C’s PCRE or zlib; C++’s Boost or Qt; or Java’s JodaTime or JUnit. These pack- ages contain high-quality, debugged Software code that required significant exper- tise to develop. For a program that needs the functionality provided by one of these packages, the tedious work of manually downloading, in- Dependencies stalling, and updating the package is easier than the work of redeveloping that functionality from scratch. The high fixed costs of reuse, however, mean manually reused packages tend to be big; a tiny package would be easier to reimplement. FOR DECADES, DISCUSSION of software reuse was more A dependency manager (a.k.a. pack- common than actual software reuse. Today, the situation age manager) automates the download- ing and installation of dependency is reversed: developers reuse software written by others packages. As dependency managers every day, in the form of software dependencies, and the make individual packages easier to download and install, the lower fixed situation goes mostly unexamined.
    [Show full text]
  • Firecracker: Lightweight Virtualization for Serverless Applications
    Firecracker: Lightweight Virtualization for Serverless Applications Alexandru Agache, Marc Brooker, Andreea Florescu, Alexandra Iordache, Anthony Liguori, Rolf Neugebauer, Phil Piwonka, and Diana-Maria Popa, Amazon Web Services https://www.usenix.org/conference/nsdi20/presentation/agache This paper is included in the Proceedings of the 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’20) February 25–27, 2020 • Santa Clara, CA, USA 978-1-939133-13-7 Open access to the Proceedings of the 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’20) is sponsored by Firecracker: Lightweight Virtualization for Serverless Applications Alexandru Agache Marc Brooker Andreea Florescu Amazon Web Services Amazon Web Services Amazon Web Services Alexandra Iordache Anthony Liguori Rolf Neugebauer Amazon Web Services Amazon Web Services Amazon Web Services Phil Piwonka Diana-Maria Popa Amazon Web Services Amazon Web Services Abstract vantage over traditional server provisioning processes: mul- titenancy allows servers to be shared across a large num- Serverless containers and functions are widely used for de- ber of workloads, and the ability to provision new func- ploying and managing software in the cloud. Their popularity tions and containers in milliseconds allows capacity to be is due to reduced cost of operations, improved utilization of switched between workloads quickly as demand changes. hardware, and faster scaling than traditional deployment meth- Serverless is also attracting the attention of the research com- ods. The economics and scale of serverless applications de- munity [21,26,27,44,47], including work on scaling out video mand that workloads from multiple customers run on the same encoding [13], linear algebra [20, 53] and parallel compila- hardware with minimal overhead, while preserving strong se- tion [12].
    [Show full text]
  • Evaluating and Improving LXC Container Migration Between
    Evaluating and Improving LXC Container Migration between Cloudlets Using Multipath TCP By Yuqing Qiu A thesis submitted to the Faculty of Graduate and Postdoctoral Affairs in partial fulfillment of the requirements for the degree of Master of Applied Science in Electrical and Computer Engineering Carleton University Ottawa, Ontario © 2016, Yuqing Qiu Abstract The advent of the Cloudlet concept—a “small data center” close to users at the edge is to improve the Quality of Experience (QoE) of end users by providing resources within a one-hop distance. Many researchers have proposed using virtual machines (VMs) as such service-provisioning servers. However, seeing the potentiality of containers, this thesis adopts Linux Containers (LXC) as Cloudlet platforms. To facilitate container migration between Cloudlets, Checkpoint and Restore in Userspace (CRIU) has been chosen as the migration tool. Since the migration process goes through the Wide Area Network (WAN), which may experience network failures, the Multipath TCP (MPTCP) protocol is adopted to address the challenge. The multiple subflows established within a MPTCP connection can improve the resilience of the migration process and reduce migration time. Experimental results show that LXC containers are suitable candidates for the problem and MPTCP protocol is effective in enhancing the migration process. i Acknowledgement I would like to express my sincerest gratitude to my principal supervisor Dr. Chung-Horng Lung who has provided me with valuable guidance throughout the entire research experience. His professionalism, patience, understanding and encouragement have always been my beacons of light whenever I go through difficulties. My gratitude also goes to my co-supervisor Dr.
    [Show full text]
  • Architectural Implications of Function-As-A-Service Computing
    Architectural Implications of Function-as-a-Service Computing Mohammad Shahrad Jonathan Balkind David Wentzlaff Princeton University Princeton University Princeton University Princeton, USA Princeton, USA Princeton, USA [email protected] [email protected] [email protected] ABSTRACT Network Serverless computing is a rapidly growing cloud application model, popularized by Amazon’s Lambda platform. Serverless cloud ser- Scheduling vices provide fine-grained provisioning of resources, which scale Platform (priorwork) automatically with user demand. Function-as-a-Service (FaaS) appli- Queueing Management cations follow this serverless model, with the developer providing 35% decrease in IPC Interference their application as a set of functions which are executed in response due to interference 6x variation due to to a user- or system-generated event. Functions are designed to Memory BW invocation pattern 20x MPKI for be short-lived and execute inside containers or virtual machines, Branch MPKI >10x exec time short functions introducing a range of system-level overheads. This paper studies for short functions Cold Start Server the architectural implications of this emerging paradigm. Using (500ms cold start) Up to 20x (thispaper) Container the commercial-grade Apache OpenWhisk FaaS platform on real slowdown servers, this work investigates and identifies the architectural im- Native plications of FaaS serverless computing. The workloads, along with Execution Figure 1: We characterize the server-level overheads of the way that FaaS inherently interleaves short functions from many Function-as-a-Service applications, compared to native exe- tenants frustrates many of the locality-preserving architectural cution. This contrasts with prior work [2–5] which focused structures common in modern processors.
    [Show full text]
  • Checkpoint and Restore of Singularity Containers
    Universitat politecnica` de catalunya (UPC) - BarcelonaTech Facultat d'informatica` de Barcelona (FIB) Checkpoint and restore of Singularity containers Grado en ingenier´ıa informatica´ Tecnolog´ıas de la Informacion´ Memoria 25/04/2019 Director: Autor: Jordi Guitart Fernandez Enrique Serrano G´omez Departament: Arquitectura de Computadors 1 Abstract Singularity es una tecnolog´ıade contenedores software creada seg´unlas necesidades de cient´ıficos para ser utilizada en entornos de computaci´onde altas prestaciones. Hace ya 2 a~nosdesde que los usuarios empezaron a pedir una integraci´onde la fun- cionalidad de Checkpoint/Restore, con CRIU, en contenedores Singularity. Esta inte- graci´onayudar´ıaen gran medida a mejorar la gesti´onde los recursos computacionales de las m´aquinas. Permite a los usuarios guardar el estado de una aplicaci´on(ejecut´andose en un contenedor Singularity) para poder restaurarla en cualquier momento, sin perder el trabajo realizado anteriormente. Por lo que la posible interrupci´onde una aplicaci´on, debido a un fallo o voluntariamente, no es una p´erdidade tiempo de computaci´on. Este proyecto muestra como es posible realizar esa integraci´on. Singularity ´esuna tecnologia de contenidors software creada segons les necessitats de cient´ıfics,per ser utilitzada a entorns de computaci´od'altes prestacions. Fa 2 anys desde que els usuaris van comen¸cara demanar una integraci´ode la funcional- itat de Checkpoint/Restore, amb CRIU, a contenidors Singularity. Aquesta integraci´o ajudaria molt a millorar la gesti´odels recursos computacionals de les m`aquines.Permet als usuaris guardar l'estat d'una aplicaci´o(executant-se a un contenidor Singularity) per poder restaurar-la en qualsevol moment, sense perdre el treball realitzat anteriorment.
    [Show full text]
  • Containerd: Integration
    containerd: integration Lantao Liu (Google) Wei Fu (Alibaba Cloud) containerd status containerd matures ● 5th project to graduate from CNCF ● Broad support from companies ● All major cloud providers using containerd ● Support Linux and Windows platform Architecture Client-Server Design Client - High level operations using client - New functionality, interfaces may change (rarely) Server - Low level interfaces to resources over GRPC - Stable API, guaranteed 1.x compatibility Backend Service Interface - Provides access to all components - Low level components wrapped by metadata store - Provides namespacing (content/Snapshotter/Image/Container) Snapshotter Snapshotters - COW filesystems - Union FS and Block Device implementations - Container RW Layer Metrics Metric API - Metrics exposed through Prometheus API - Exposes metrics for containerd process & container level metrics Kubernetes Runtime Support Kubernetes Runtime Support - CRI gRPC API exposed from containerd - Kubelet can be configured to use containerd as runtime Summary ● Stable gRPC interface ● Kubernetes Runtime Support Smart Client Model gRPC API Smart Client - Mirrors internal component interfaces - General-Purpose interface - Snapshots, Content, Containers, Task, Events, etc - Direct access to the component (e.g. Snapshots) Pull Image Diff Content Snapshotter Image Registry Client Service Service Service Service Get manifest store manifest each layer Get layer store layer each layer prepare snapshot apply diff read layer mount & unpack layer descriptor commit snapshot create
    [Show full text]
  • Linux Kernel Debugging Advanced Operating Systems 2020/2021
    Linux Kernel Debugging Advanced Operating Systems 2020/2021 Vlastimil Babka Linux Kernel Developer, SUSE Labs [email protected] Agenda – Debugging Scenarios • Debugging production kernels – Post-mortem analysis: interpreting kernel oops/panic output, creating and analyzing kernel crash dumps – Kernel observability – dynamic debug, tracing, alt-sysrq dumps, live crash session • Debugging during individual kernel development – Debug prints – printk() facilitiy – Debugger (gdb) support • Finding (latent) bugs during collaborative development – Optional runtime checks configurable during build – Testing and fuzzing – Static analysis 2 Enterprise Linux Distro and Bugs (incl. Kernel) • The software installation (e.g. ISO) itself is free (and open source, obviously) • Customers pay for support subscription – Delivery of (tested by QA) package updates – fixing known bugs, CVE’s… but not more! – Getting reported bugs fixed • Bugs specific to customer’s workload, hardware, “luck”, large number of machines… • Upstream will also fix reported bugs, but only with latest kernel and no effort guarantees • Dealing with kernel bugs, e.g. crashes – Find out the root cause (buggy code) with limited possibilities (compared to local development) • Typically no direct access to customer’s system or workload • Long turnarounds for providing a modified debug kernel and reproducing the bug – Write and deliver a fix (upstream first!) or workaround; fix goes to next update • Possibly a livepatch in some cases – Is a lot of fun ;-) 3 Kernel Oops - The Real World Example
    [Show full text]
  • Bring Security to the Container World, and More
    01,1001.01 100 010 , , 100 , 1 100 , 1 “All problems in computer science can be solved by another level of indirection, except of course for the problem of too many indirections.” ----David Wheeler .- • :.’:. • : . • ...,:. • :....,:.: .,-.,. ., • .’. What’s Kata Containers • /// • // • /.,/ ./A • /.,/. • ,/.,/// Kata Containers is Virtualized Container :- • • : : • - • : • • app centric OS feature package runtime .,- • -,, • • • , , • , , system view OS supervisor isolation hardware aware* Containerization Virtualization app centric OS feature system view OS supervisor package runtime isolation hardware aware* View Point From application to OS; The From hardware to OS; intercept environment of an app the instructions of guest system Design Goal Kernel ABI, including syscall A machine like the host, could interfaces, procfs/sysfs, etc. run a kernel on top of it Implementati Inside kernel, a feature of the A supervisor of the kernel, a on scheduler scheduler of the schedulers • Namespace Virtual Machine – • Process Process Process – Process VM is a type of container Guest Kernel indeed Host Kernel * Industry Compatible Enhanced Networking Direct Device Support Run Custom Kernel More Secure :-:-: • : ! - ::: ! :-:- - ! :- • : " ,-:-:- Docker " --B Image runC " : -: : : -- - : Docker Docker Image runC Image * Intel® Clear Containers Announced project Release 1.0 May 2015 Dec 2017 May 2018 *Other names and brands may be claimed as the property of others. Multi Architecture Direct
    [Show full text]