Kpatch • Urgent Security and Stability Fixes, Cves, Driver Issues and Kernel Development

Total Page:16

File Type:pdf, Size:1020Kb

Kpatch • Urgent Security and Stability Fixes, Cves, Driver Issues and Kernel Development Achieve a 0-downtime CERN Database infrastructure Summer Student: Varsha Rao Supervisor: Borja Aparicio Cotarelo IT-DB-IMS 1 Why is it required? • Security vulnerabilities • Difficulty in scheduling downtime • Hardware consolidation Varsha Rao, Borja Aparicio Cotarelo 2 What can be done? Varsha Rao, Borja Aparicio Cotarelo 3 What can be done? Varsha Rao, Borja Aparicio Cotarelo 4 Live Kernel Patching “Hot patching, also known as live patching or dynamic software updating, is the application of patches without shutting down and restarting the system or the program concerned.” - Wikipedia Definition Varsha Rao, Borja Aparicio Cotarelo 5 Comparison Use Cases Ksplice • Runtime security vulnerabilities and stability bugs. Kpatch • Urgent security and stability fixes, CVEs, driver issues and kernel development. Varsha Rao, Borja Aparicio Cotarelo 6 Comparison License Ksplice • It was originally GPLV2, after Oracle acquisition it is no longer. Kpatch • GPLV2 Varsha Rao, Borja Aparicio Cotarelo 7 Comparison Is it merged into Kernel Mainline? Ksplice • No Live Patching Technology Kpatch Ksplice kGraft Kpatch • Yes Oracle - 2009 SUSE - 2014 RedHat - 2014 Varsha Rao, Borja Aparicio Cotarelo 8 Comparison Status Ksplice • Production ready Kpatch • Not production ready Varsha Rao, Borja Aparicio Cotarelo 9 Comparison Operating System Ksplice • Support available only for Oracle Linux. • Community Edition is provided for Fedora and Ubuntu. Kpatch • Support is only available for RHEL. • Available on major Linux distros. Varsha Rao, Borja Aparicio Cotarelo 10 Comparison Installation For both of them, the installation process is • Easy and quick • Well documented Varsha Rao, Borja Aparicio Cotarelo 11 What did we do • Understand both Kpatch and Ksplice Varsha Rao, Borja Aparicio Cotarelo 12 What did we do • Understand both Kpatch and Ksplice • Install both of them Varsha Rao, Borja Aparicio Cotarelo 13 What did we do • Understand both Kpatch and Ksplice • Install both of them • Test and Verify the features Varsha Rao, Borja Aparicio Cotarelo 14 Features Ksplice Kpatch License Initially GPLV2 GPLV2 Patch Provisioning Lacks transparency Patch can be built easily Dependency Yes, Oracle Updates Optional, RedHat Updates Varsha Rao, Borja Aparicio Cotarelo 15 Ksplice { Install Command } Varsha Rao, Borja Aparicio Cotarelo 16 Kpatch { Builds Patch Module} { Loads Patch Module } Varsha Rao, Borja Aparicio Cotarelo 17 Conclusion • Live Kernel Patching Works • Requires long term evaluation • General Kernel Upgrade is not possible • Depends on sysadmin requirements Varsha Rao, Borja Aparicio Cotarelo 18 Thanks!! [email protected] [email protected] Varsha Rao, Borja Aparicio Cotarelo 19.
Recommended publications
  • Red Hat Enterprise Linux 7 7.1 Release Notes
    Red Hat Enterprise Linux 7 7.1 Release Notes Release Notes for Red Hat Enterprise Linux 7 Red Hat Customer Content Services Red Hat Enterprise Linux 7 7.1 Release Notes Release Notes for Red Hat Enterprise Linux 7 Red Hat Customer Content Services Legal Notice Copyright © 2015 Red Hat, Inc. This document is licensed by Red Hat under the Creative Commons Attribution-ShareAlike 3.0 Unported License. If you distribute this document, or a modified version of it, you must provide attribution to Red Hat, Inc. and provide a link to the original. If the document is modified, all Red Hat trademarks must be removed. Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law. Red Hat, Red Hat Enterprise Linux, the Shadowman logo, JBoss, MetaMatrix, Fedora, the Infinity Logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries. Linux ® is the registered trademark of Linus Torvalds in the United States and other countries. Java ® is a registered trademark of Oracle and/or its affiliates. XFS ® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries. MySQL ® is a registered trademark of MySQL AB in the United States, the European Union and other countries. Node.js ® is an official trademark of Joyent. Red Hat Software Collections is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
    [Show full text]
  • Oracle® Linux Administrator's Solutions Guide for Release 6
    Oracle® Linux Administrator's Solutions Guide for Release 6 E37355-64 August 2017 Oracle Legal Notices Copyright © 2012, 2017, Oracle and/or its affiliates. All rights reserved. This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable: U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S.
    [Show full text]
  • Adaptive Android Kernel Live Patching
    Adaptive Android Kernel Live Patching Yue Chen Yulong Zhang Zhi Wang Liangzhao Xia Florida State University Baidu X-Lab Florida State University Baidu X-Lab Chenfu Bao Tao Wei Baidu X-Lab Baidu X-Lab Abstract apps contain sensitive personal data, such as bank ac- counts, mobile payments, private messages, and social Android kernel vulnerabilities pose a serious threat to network data. Even TrustZone, widely used as the se- user security and privacy. They allow attackers to take cure keystore and digital rights management in Android, full control over victim devices, install malicious and un- is under serious threat since the compromised kernel en- wanted apps, and maintain persistent control. Unfortu- ables the attacker to inject malicious payloads into Trust- nately, most Android devices are never timely updated Zone [42, 43]. Therefore, Android kernel vulnerabilities to protect their users from kernel exploits. Recent An- pose a serious threat to user privacy and security. droid malware even has built-in kernel exploits to take Tremendous efforts have been put into finding (and ex- advantage of this large window of vulnerability. An ef- ploiting) Android kernel vulnerabilities by both white- fective solution to this problem must be adaptable to lots hat and black-hat researchers, as evidenced by the sig- of (out-of-date) devices, quickly deployable, and secure nificant increase of kernel vulnerabilities disclosed in from misuse. However, the fragmented Android ecosys- Android Security Bulletin [3] in recent years. In ad- tem makes this a complex and challenging task. dition, many kernel vulnerabilities/exploits are publicly To address that, we systematically studied 1;139 An- available but never reported to Google or the vendors, droid kernels and all the recent critical Android ker- let alone patched (e.g., exploits in Android rooting nel vulnerabilities.
    [Show full text]
  • Fast and Scalable VMM Live Upgrade in Large Cloud Infrastructure
    Fast and Scalable VMM Live Upgrade in Large Cloud Infrastructure Xiantao Zhang Xiao Zheng Zhi Wang Alibaba Group Alibaba Group Florida State University [email protected] [email protected] [email protected] Qi Li Junkang Fu Yang Zhang Tsinghua University Alibaba Group Alibaba Group [email protected] [email protected] [email protected] Yibin Shen Alibaba Group [email protected] Abstract hand over passthrough devices to the new KVM instance High availability is the most important and challenging prob- without losing any ongoing (DMA) operations. Our evalua- lem for cloud providers. However, virtual machine mon- tion shows that Orthus can reduce the total migration time itor (VMM), a crucial component of the cloud infrastruc- and downtime by more than 99% and 90%, respectively. We ture, has to be frequently updated and restarted to add secu- have deployed Orthus in one of the largest cloud infrastruc- rity patches and new features, undermining high availabil- tures for a long time. It has become the most effective and ity. There are two existing live update methods to improve indispensable tool in our daily maintenance of hundreds of the cloud availability: kernel live patching and Virtual Ma- thousands of servers and millions of VMs. chine (VM) live migration. However, they both have serious CCS Concepts • Security and privacy → Virtualization drawbacks that impair their usefulness in the large cloud and security; • Computer systems organization → Avail- infrastructure: kernel live patching cannot handle complex ability. changes (e.g., changes to persistent data structures); and VM live migration may incur unacceptably long delays when Keywords virtualization; live upgrade; cloud infrastructure migrating millions of VMs in the whole cloud, for example, ACM Reference Format: to deploy urgent security patches.
    [Show full text]
  • Protecting Your Linux Systems with Oracle Ksplice
    Staying Ahead of Cyberthreats: Protecting Your Linux Systems with Oracle Ksplice The Advantages Of Zero-Downtime Patching April 23, 2020 Copyright © 2020, Oracle and/or its affiliates Public TABLE OF CONTENTS Introduction 2 Why Patching Matters 2 About Oracle Ksplice 3 Other Benefits 3 Conclusion 4 Learn More 4 1 WHITE PAPER | Staying Ahead of Cyberthreats: Protecting Your Linux Systems Copyright © 2020, Oracle and/or its affiliates |Public INTRODUCTION IT systems require regular patching for security, performance, and compliance reasons. For Linux operating system (OS) kernel updates, which include “Availability requirements important new security enhancements and bug fixes, releases happen about 1 are on the rise for once per month. These updates help keep systems current with the latest organizations undergoing innovations. However, manually patching systems has many inherent digital transformations. challenges and difficulties which tends to delay their timely application. For this Downtimes are costly, reason, zero-downtime patching solutions for Linux, like Oracle Ksplice, are with unplanned becoming essential tools. In this paper, Oracle Ksplice’s capabilities and many infrastructure downtimes advantages are explained. costing $100,000 per hour on an average. With Why Patching Matters the possibility of every organization being a Inadequate patch management can leave loopholes in the IT infrastructure leading to target for cyberattacks various security and performance issues. Ideally, patches should be applied shortly after and attackers moving very release to ensure the latest system protections. Patching typically requires downtime quickly to exploit system which, depending on operations, can require weeks or months of advanced planning. vulnerabilities, IDC Most Linux patching also traditionally happens at the disk level for file systems, which has recommends several disadvantages.
    [Show full text]
  • Live Kernel Patching Using Kgraft
    SUSE Linux Enterprise Server 12 SP4 Live Kernel Patching Using kGraft SUSE Linux Enterprise Server 12 SP4 This document describes the basic principles of the kGraft live patching technology and provides usage guidelines for the SLE Live Patching service. kGraft is a live patching technology for runtime patching of the Linux kernel, without stopping the kernel. This maximizes system uptime, and thus system availability, which is important for mission-critical systems. By allowing dynamic patching of the kernel, the technology also encourages users to install critical security updates without deferring them to a scheduled downtime. A kGraft patch is a kernel module, intended for replacing whole functions in the kernel. kGraft primarily oers in-kernel infrastructure for integration of the patched code with base kernel code at runtime. SLE Live Patching is a service provided on top of regular SUSE Linux Enterprise Server maintenance. kGraft patches distributed through SLE Live Patching supplement regular SLES maintenance updates. Common update stack and procedures can be used for SLE Live Patching deployment. Publication Date: 09/24/2021 Contents 1 Advantages of kGraft 3 2 Low-level Function of kGraft 3 1 Live Kernel Patching Using kGraft 3 Installing kGraft Patches 4 4 Patch Lifecycle 6 5 Removing a kGraft Patch 6 6 Stuck Kernel Execution Threads 6 7 The kgr Tool 7 8 Scope of kGraft Technology 7 9 Scope of SLE Live Patching 8 10 Interaction with the Support Processes 8 11 GNU Free Documentation License 8 2 Live Kernel Patching Using kGraft 1 Advantages of kGraft Live kernel patching using kGraft is especially useful for quick response in emergencies (when serious vulnerabilities are known and should be xed when possible or there are serious system stability issues with a known x).
    [Show full text]
  • Kshot: Live Kernel Patching with SMM and SGX
    KShot: Live Kernel Patching with SMM and SGX Lei Zhou∗y, Fengwei Zhang∗, Jinghui Liaoz, Zhengyu Ning∗, Jidong Xiaox Kevin Leach{, Westley Weimer{ and Guojun Wangk ∗Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China, zhoul2019,zhangfw,ningzy2019 @sustech.edu.cn f g ySchool of Computer Science and Engineering, Central South University, Changsha, China zDepartment of Computer Science, Wayne State University, Detroit, USA, [email protected] xDepartment of Computer Science, Boise State University, Boise, USA, [email protected] Department of Computer Science and Engineering, University of Michigan, Ann Arbor, USA, kjleach,weimerw @umich.edu { f g kSchool of Computer Science and Cyber Engineering, Guangzhou University, Guangzhou, China, [email protected] Abstract—Live kernel patching is an increasingly common kernel vulnerabilities also merit patching. Organizations often trend in operating system distributions, enabling dynamic up- use rolling upgrades [3], [6], in which patches are designed dates to include new features or to fix vulnerabilities without to affect small subsystems that minimize unplanned whole- having to reboot the system. Patching the kernel at runtime lowers downtime and reduces the loss of useful state from running system downtime, to update and patch whole server systems. applications. However, existing kernel live patching techniques However, rolling upgrades do not altogether obviate the need (1) rely on specific support from the target operating system, to restart software or reboot systems; instead, dynamic hot and (2) admit patch failures resulting from kernel faults. We patching (live patching) approaches [7]–[9] aim to apply present KSHOT, a kernel live patching mechanism based on patches to running software without having to restart it.
    [Show full text]
  • Porting Linux Embedded Linux Conference (Europe)
    Porting Linux Embedded Linux Conference (Europe) Porting Linux About Jon Masters ● Been playing with Linux for 14 years (and the kernel for 13 of those), since the age of 13. ● Built embedded NMR scientific instruments, worked with Montavista UK, now at Red Hat. ● Author of the LKML Summary Podcast and the kernel column in Linux User & Developer. ● Co-author of Building Embedded Linux Systems (second edition) – O'Reilly (2008) ● My car still has an empeg :) Porting Linux Overview ● Why port Linux anyway? ● Background pre-requisities ● Early board work ● Bootloader bringup ● Initial kernel bringup ● Debugging ● Working with Upstream ● Trends Porting Linux Why port Linux anyway? ● Linux is very portable ● Supports 23 architectures in the upstream “mainline” kernel tree of Linus Torvalds. ● Kernel is mostly written in C, with some assembly (most architectures only need a dozen such files) ● Split between high-level generic functions and low- level functions to abstract architectural differences. Porting Linux Why port Linux anyway? ● Linux is competitive ● The number of Linux kernel developers contributing to the official kernel has tripled since 2005. ● Feature growth continues with an average of 10K new lines of source code added every day. ● In the hour you spend here 5.45 patches will on average be added to the upstream Linux kernel. ● Source: Linux Foundation analysis Porting Linux Why port Linux anyway? ● Linux is cost effective. ● A large amount of code to build upon. ● Large (growing) community of developers. ● I think we all know the rest. Porting Linux Background pre-requisities ● Hardware ● Development board or simulator – Optional debugger, some kind of UART – Boards range in value from $200-$crazy – Implement the same architecture and platform as the final design but maybe with a number of hacks.
    [Show full text]
  • Think ALL Distros Offer the Best Linux Devsecops Environment?
    Marc Staimer, Dragon Slayor Consulting WHITE PAPER Think All Distros Offer the Best Linux DevSecOps What You’re Not Being Told About Environment? Database as a Service (DBaaS) Think Again! WHITE PAPER • Think Again! Think All Distros Provide the Best Linux DevSecOps Environment? Think Again! Introduction DevOps is changing. Developing code with after the fact bolt-on security is dangerously flawed. When that bolt-on fails to correct exploitable code vulnerabilities, it puts the entire organization at risk. Security has been generally an afterthought for many doing DevOps. It was often assumed the IT organization’s systemic multiple layers of security measures and appliances would protect any new code from malware or breaches. And besides, developing code with security built in, adds tasks and steps to development and testing time. More tasks and steps delay time-to-market. Multi-tenant clouds have radically changed the market. Any vulnerability in a world with increasing cyber-attacks, can put millions of user’s data at risk. Those legacy DevOps attitudes are unsound. They are potentially quite costly in the current environment. Consider that nearly every developed and most developing countries have enacted laws and regulation protecting personally identifiable information or PII1. PII is incredibly valuable to cybercriminals. Stealing PII enables them to commit many cybercrimes including the cybertheft of identities, finances, intellectual property, admin privileges, and much more. PII can also be sold on the web. Those PII laws and regulations are meant to force IT organizations to protect PII. Non-compliance of these laws and regulations often carry punitive financial penalties.
    [Show full text]
  • Fast and Live Hypervisor Replacement
    Fast and Live Hypervisor Replacement Spoorti Doddamani Piush Sinha Hui Lu Binghamton University Binghamton University Binghamton University New York, USA New York, USA New York, USA [email protected] [email protected] [email protected] Tsu-Hsiang K. Cheng Hardik H. Bagdi Kartik Gopalan Binghamton University Binghamton University Binghamton University New York, USA New York, USA New York, USA [email protected] [email protected] [email protected] Abstract International Conference on Virtual Execution Environments (VEE Hypervisors are increasingly complex and must be often ’19), April 14, 2019, Providence, RI, USA. ACM, New York, NY, USA, updated for applying security patches, bug fixes, and feature 14 pages. https://doi.org/10.1145/3313808.3313821 upgrades. However, in a virtualized cloud infrastructure, up- 1 Introduction dates to an operational hypervisor can be highly disruptive. Before being updated, virtual machines (VMs) running on Virtualization-based server consolidation is a common prac- a hypervisor must be either migrated away or shut down, tice in today’s cloud data centers [2, 24, 43]. Hypervisors host resulting in downtime, performance loss, and network over- multiple virtual machines (VMs), or guests, on a single phys- head. We present a new technique, called HyperFresh, to ical host to improve resource utilization and achieve agility transparently replace a hypervisor with a new updated in- in resource provisioning for cloud applications [3, 5–7, 50]. stance without disrupting any running VMs. A thin shim Hypervisors must be often updated or replaced for various layer, called the hyperplexor, performs live hypervisor re- purposes, such as for applying security/bug fixes [23, 41] placement by remapping guest memory to a new updated adding new features [15, 25], or simply for software reju- hypervisor on the same machine.
    [Show full text]
  • Oracle Linux System Administration I
    Oracle Linux System Administration I Dieser Kurs im Web Die Schulung hilft Ihnen, eine Reihe von Fähigkeiten zu entwickeln, einschließlich der Installation, der Verwendung des Unbreakable Enterprise Kernel, der Alle tagesaktuellen Informationen Konfiguration von Linux-Diensten, der Vorbereitung des Systems für die Oracle- und Möglichkeiten zur Bestellung finden Sie unter dem folgenden Link: Datenbank, der Überwachung und der Fehlerbehebung. www.experteach.de/go/026S Nach diesem Kurs verfügen Sie über das Wissen und die Fähigkeiten, typische Vormerkung Probleme von Administratoren zu lösen, und verstehen das Kernel- Sie können auf unserer Website einen Platz Entwicklungsmodell und die Linux-Distributionen. Machen Sie sich damit vertraut, kostenlos und unverbindlich für 7 Tage reservieren. wie Oracle Linux Ihnen die neuesten Linux-Innovationen bietet, die extreme Dies geht auch telefonisch unter 06074 4868-0. Leistung, erweiterte Skalierbarkeit und Zuverlässigkeit für Garantierte Kurstermine Unternehmensanwendungen und -systeme ermöglichen. Für Ihre Planungssicherheit bieten wir stets eine große Auswahl garantierter Kurstermine an. Kursinhalt • Course Introduction Ihr Kurs maßgeschneidert • Introduction to Oracle Linux Diesen Kurs können wir für Ihr Projekt exakt an • Installing Oracle Linux 7 Ihre Anforderungen anpassen. • Oracle Linux 7 Boot Process • System Configuration • Package Management • Ksplice • Automate Tasks • Kernel Module Configuration • User and Group Administration • Partitions, File Systems, and Swap • Implementing the XFS File System • Implementing the Btrfs File System • Storage Administration • Network Configuration • File Sharing • OpenSSH Service • Security Administration • Oracle on Oracle • System Monitoring E-Book Sie erhalten die englischsprachigen Original-Unterlagen in Form eines Oracle University eKits. Zielgruppe • System Administrator • Linux Administrator • Entwickler Voraussetzungen • Arten von Benutzerkonten & Arbeiten mit Dateien und Verzeichnissen unter Unix • Textbearbeitung mit vi & Unix-Prozesskontrolle Training Preise zzgl.
    [Show full text]
  • (12) United States Patent (10) Patent No.: US 8,124,082 B2 Fong Et Al
    USOO8124082B2 (12) United States Patent (10) Patent No.: US 8,124,082 B2 Fong et al. (45) Date of Patent: Feb. 28, 2012 (54) HUMANIZED ANTI-BETA7 ANTAGONISTS 3.68 A 13 3. SEetC ren al. et al.1 AND USES THEREFOR 5,624,821 A 4/1997 Winter et al. 5,648,260 A 7, 1997 Winter et al. (75) Inventors: Sherman Fong, Alameda, CA (US); 5,658,727 A 8, 1997 Barbas et al. Mark S. Dennis, San Carlos, CA (US) 5,693,762 A 12/1997 Queen et al. 5,712.374. A 1/1998 Kuntsmann et al. 5,714,586 A 2, 1998 Kunstmann et al. (73) Assignee: Genentech, Inc., South San Francisco, 5,731, 168 A 3, 1998 Carter et al. CA (US) 5,733,743 A 3/1998 Johnson et al. 5,739,116 A 4, 1998 Hamann et al. (*) Notice: Subject to any disclaimer, the term of this 5,750,373 A 5/1998 Garrard et al. patent is extended or adjusted under 35 5,767.285 A 6/1998 Hamann et al. U.S.C. 154(b) by 119 days 5,770,701 A 6/1998 McGahren et al. M YW- y yS. 5,770,710 A 6/1998 McGahren et al. 5,773,001 A 6/1998 Hamann et al. (21) Appl. No.: 12/390,730 5,837.242 A 1 1/1998 Holliger et al. 5,877,296 A 3, 1999 Hamann et al. (22) Filed: Feb. 23, 2009 5,969,108 A 10/1999 McCafferty et al. 9 6,172,197 B1 1/2001 McCafferty et al.
    [Show full text]