DOCSLIB.ORG

Buildroot Using Embedded Tools to Build Container Images

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Buildroot Using Embedded Tools to Build Container Images Buildroot Using embedded tools to build container images Jérémy Rosen A couple of disclaimers I come from the embedded world More precisely the embedded, industrial world. Not the phone/tablet world Quite dierent to the consumer product world I don’t really know anything about datacenter problems This is not about container conguration Nothing about namespaces Nothing about CGroups Nothing about network conguration We will only talk about rootfs generation. 2/10 Containers in the embedded world Containers have a limited but real use in the embedded world No thin provisionning, no dynamic deployment Applications are highly hardware dependant Limited use-case for multiversionning of libraries (and old/closed software) Limited use-case as “package manager” The DevOps approch is a hard sell... Rapid deployment doesn’t work on embedded device Exponential number of versions oating around All source-code need to be archived and traced for licence compliance Rollback/failed upgrades are much more frightening in the embedded world Containers as a security feature is...complicated 3/10 Building an image from source What we want Complete tracability to the line of code Everything is checksumed Complete archival of the source code All source tarballs are archived All patches are archived All overlays are archived All scripts run during build are archived Indepence from the host/Reproducible build Build/archive the build-tools 4/10 Buildroot : An image building tool Buildroot is an “automated Linux From Scratch” based on advanced makeles Automated build process Handle dependencies Patch sources Install Download Congure Check SHA Build Image generation (Fakeroot based) Tweak permissions Creates disk images Add les and content License collection Creates FS images Source-code collection Cross-compilation friendly Since 2001 We love our “single command from source to image” approch 5/10 A complete buildroot container build Minimal image based on buildroot’s default conguration git clone git ://git.buildroot.net/buildroot -b 2019.05.1 make menuconfig Target ) target architecture ) x86_64 make sudo tar xvf output/images/rootfs.tar sudo systemd-nspawn -D <the rootfs> -b What we have here busybox, µClibc, no kernel cross-gcc, host-autotools, host-makedevs, host-fakeroot... “heavy” container (it has busybox init) Built in 12min (excluding download), 1.6M total 1 This image is very similar to an alpine-linux built from scratch. 6/10 1. 1min and 6.3M with a glibc pre-compiled toolchain A more realistic example : Light container with Apache Changing a couple of options in the KCong Pre-built GCC from bootlin (glibc based) Add Apache Remove busybox, init-system support, shell support After rebuild, we have a functionnal light container No shell, no init No customisation Built in 3m, 11M image apachectl doesn’t work...because apachectl is a shell script sudo systemd-nspawn -D <the rootfs> --as-pid2 httpd -D FOREGROUND 7/10 Customising the image : overlay As an example of customisation, let’s transform our container into a portable service To do that, we need an apache.service le added in the rootfs. System Configuration ) Root filesystem overlay directory $(TOPDIR)/overlay mkdir -p overlay/etc/systemd/system add apache.service in that directory [Service] ExecStart=/usr/bin/httpd -D FOREGROUND make etc. portablectl attach -p trusted apache systemctl start apache The overlay directory makes it trivial to see what has changed in your image (and git it) You can have multiple overlay directories 8/10 other customisation tools patch directory : easily patch any package from upstream. overlay directory : most of the customisation go here You can have multiple overlay directories post-rootfs script : adjust permission in fakeroot. post-image script : adjust things at image/disk generation time These tools are here to allow you to never manually tweak the image. All changes should be done through buildroot to keep changes trackable and image-builds reproducible 9/10 Conclusion : why use buildroot What it brings Source based, signatures included in the recipes No need to be root to build images Trivial to understand, easy to master. Archival/tracability/license-compliance Highly independant from the host system All patches are visible and easy to manage Reduced attack surface Reproducible builds (experimental) Easy to build portable services/light-containers Easy to debug (all sources, debug symbols, build artifacts are available) Drawbacks Need to understand Linux. No hand-holding by the distro. Hard to debug (no shell, remote gdb) 10/10 Build time..
Load more

Recommended publications
  • The Linux Kernel Module Programming Guide
    The Linux Kernel Module Programming Guide Peter Jay Salzman Michael Burian Ori Pomerantz Copyright © 2001 Peter Jay Salzman 2007−05−18 ver 2.6.4 The Linux Kernel Module Programming Guide is a free book; you may reproduce and/or modify it under the terms of the Open Software License, version 1.1. You can obtain a copy of this license at http://opensource.org/licenses/osl.php. This book is distributed in the hope it will be useful, but without any warranty, without even the implied warranty of merchantability or fitness for a particular purpose. The author encourages wide distribution of this book for personal or commercial use, provided the above copyright notice remains intact and the method adheres to the provisions of the Open Software License. In summary, you may copy and distribute this book free of charge or for a profit. No explicit permission is required from the author for reproduction of this book in any medium, physical or electronic. Derivative works and translations of this document must be placed under the Open Software License, and the original copyright notice must remain intact. If you have contributed new material to this book, you must make the material and source code available for your revisions. Please make revisions and updates available directly to the document maintainer, Peter Jay Salzman <[email protected]>. This will allow for the merging of updates and provide consistent revisions to the Linux community. If you publish or distribute this book commercially, donations, royalties, and/or printed copies are greatly appreciated by the author and the Linux Documentation Project (LDP).
    [Show full text]
  • CNTR: Lightweight OS Containers
    CNTR: Lightweight OS Containers Jorg¨ Thalheim, Pramod Bhatotia Pedro Fonseca Baris Kasikci University of Edinburgh University of Washington University of Michigan Abstract fundamental to achieve high efficiency in virtualized datacenters and enables important use-cases, namely Container-based virtualization has become the de-facto just-in-time deployment of applications. Moreover, standard for deploying applications in data centers. containers significantly reduce operational costs through However, deployed containers frequently include a higher consolidation density and power minimization, wide-range of tools (e.g., debuggers) that are not required especially in multi-tenant environments. Because of all for applications in the common use-case, but they these advantages, it is no surprise that containers have seen are included for rare occasions such as in-production wide-spread adoption by industry, in many cases replacing debugging. As a consequence, containers are significantly altogether traditional virtualization solutions [17]. larger than necessary for the common case, thus increasing the build and deployment time. Despite being lightweight, deployed containers often include a wide-range of tools such as shells, editors, CNTR1 provides the performance benefits of lightweight coreutils, and package managers. These additional tools containers and the functionality of large containers by are usually not required for the application’s core function splitting the traditional container image into two parts: the — the common operational use-case — but they are “fat” image — containing the tools, and the “slim” image included for management, manual inspection, profiling, — containing the main application. At run-time, CNTR and debugging purposes [64]. In practice, this significantly allows the user to efficiently deploy the “slim” image and increases container size and, in turn, translates into then expand it with additional tools, when and if necessary, slower container deployment and inefficient datacenter by dynamically attaching the “fat” image.
    [Show full text]
  • Version 7.8-Systemd
    Linux From Scratch Version 7.8-systemd Created by Gerard Beekmans Edited by Douglas R. Reno Linux From Scratch: Version 7.8-systemd by Created by Gerard Beekmans and Edited by Douglas R. Reno Copyright © 1999-2015 Gerard Beekmans Copyright © 1999-2015, Gerard Beekmans All rights reserved. This book is licensed under a Creative Commons License. Computer instructions may be extracted from the book under the MIT License. Linux® is a registered trademark of Linus Torvalds. Linux From Scratch - Version 7.8-systemd Table of Contents Preface .......................................................................................................................................................................... vii i. Foreword ............................................................................................................................................................. vii ii. Audience ............................................................................................................................................................ vii iii. LFS Target Architectures ................................................................................................................................ viii iv. LFS and Standards ............................................................................................................................................ ix v. Rationale for Packages in the Book .................................................................................................................... x vi. Prerequisites
    [Show full text]
  • Securing Embedded Systems: Analyses of Modern Automotive Systems and Enabling Near-Real Time Dynamic Analysis
    Securing Embedded Systems: Analyses of Modern Automotive Systems and Enabling Near-Real Time Dynamic Analysis Karl Koscher A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy University of Washington 2014 Reading Committee: Tadayoshi Kohno, Chair Gaetano Borriello Shwetak Patel Program Authorized to Offer Degree: Computer Science and Engineering © Copyright 2014 Karl Koscher University of Washington Abstract Securing Embedded Systems: From Analyses of Modern Automotive Systems to Enabling Dynamic Analysis Karl Koscher Chair of the Supervisory Committee: Associate Professor Tadayoshi Kohno Department of Computer Science and Engineering Today, our life is pervaded by computer systems embedded inside everyday products. These embedded systems are found in everything from cars to microwave ovens. These systems are becoming increasingly sophisticated and interconnected, both to each other and to the Internet. Unfortunately, it appears that the security implications of this complexity and connectivity have mostly been overlooked, even though ignoring security could have disastrous consequences; since embedded systems control much of our environment, compromised systems could be used to inflict physical harm. This work presents an analysis of security issues in embedded systems, including a comprehensive security analysis of modern automotive systems. We hypothesize that dynamic analysis tools would quickly discover many of the vulnerabilities we found. However, as we will discuss, there
    [Show full text]
  • Linux for Zseries: Device Drivers and Installation Commands (March 4, 2002) Summary of Changes
    Linux for zSeries Device Drivers and Installation Commands (March 4, 2002) Linux Kernel 2.4 LNUX-1103-07 Linux for zSeries Device Drivers and Installation Commands (March 4, 2002) Linux Kernel 2.4 LNUX-1103-07 Note Before using this document, be sure to read the information in “Notices” on page 207. Eighth Edition – (March 2002) This edition applies to the Linux for zSeries kernel 2.4 patch (made in September 2001) and to all subsequent releases and modifications until otherwise indicated in new editions. © Copyright International Business Machines Corporation 2000, 2002. All rights reserved. US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Summary of changes .........v Chapter 5. Linux for zSeries Console || Edition 8 changes.............v device drivers............27 Edition 7 changes.............v Console features .............28 Edition 6 changes ............vi Console kernel parameter syntax .......28 Edition 5 changes ............vi Console kernel examples ..........28 Edition 4 changes ............vi Usingtheconsole............28 Edition 3 changes ............vii Console – Use of VInput ..........30 Edition 2 changes ............vii Console limitations ............31 About this book ...........ix Chapter 6. Channel attached tape How this book is organized .........ix device driver ............33 Who should read this book .........ix Tapedriverfeatures...........33 Assumptions..............ix Tape character device front-end........34 Tape block
    [Show full text]
  • Linux Kernel and Driver Development Training Slides
    Linux Kernel and Driver Development Training Linux Kernel and Driver Development Training © Copyright 2004-2021, Bootlin. Creative Commons BY-SA 3.0 license. Latest update: October 9, 2021. Document updates and sources: https://bootlin.com/doc/training/linux-kernel Corrections, suggestions, contributions and translations are welcome! embedded Linux and kernel engineering Send them to [email protected] - Kernel, drivers and embedded Linux - Development, consulting, training and support - https://bootlin.com 1/470 Rights to copy © Copyright 2004-2021, Bootlin License: Creative Commons Attribution - Share Alike 3.0 https://creativecommons.org/licenses/by-sa/3.0/legalcode You are free: I to copy, distribute, display, and perform the work I to make derivative works I to make commercial use of the work Under the following conditions: I Attribution. You must give the original author credit. I Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under a license identical to this one. I For any reuse or distribution, you must make clear to others the license terms of this work. I Any of these conditions can be waived if you get permission from the copyright holder. Your fair use and other rights are in no way affected by the above. Document sources: https://github.com/bootlin/training-materials/ - Kernel, drivers and embedded Linux - Development, consulting, training and support - https://bootlin.com 2/470 Hyperlinks in the document There are many hyperlinks in the document I Regular hyperlinks: https://kernel.org/ I Kernel documentation links: dev-tools/kasan I Links to kernel source files and directories: drivers/input/ include/linux/fb.h I Links to the declarations, definitions and instances of kernel symbols (functions, types, data, structures): platform_get_irq() GFP_KERNEL struct file_operations - Kernel, drivers and embedded Linux - Development, consulting, training and support - https://bootlin.com 3/470 Company at a glance I Engineering company created in 2004, named ”Free Electrons” until Feb.
    [Show full text]
  • Root Filesystem
    >>> Operating Systems And Applications For Embedded Systems >>> Root Filesystem Name: Mariusz Naumowicz Date: 27 sierpnia 2018 [~]$ _ [1/21] >>> Plan 1. Root Filesystem Useful System Filesystem Hierarchy Standard (FHS) Staging directory 2. Programs The init program Shell Utilities BusyBox ToyBox Libraries Reducing size by stripping Device nodes The proc and sysfs flesystems Mounting flesystems Additional reading Standalone ramdisk Minimizing size Booting with QEMU Additional reading [~]$ _ [2/21] >>> Useful System * init: The program that starts everything off, usually by running a series of scripts. * shell: Needed to give you a command prompt but, more importantly, to run the shell scripts called by init and other programs. * daemons: Various server programs, started by init. * libraries: Usually, the programs mentioned so far are linked with shared libraries which must be present in the root filesystem. * Configuration files: The configuration for init and other daemons is stored in a series of ASCII text files, usually in the /etc directory. * Device nodes: The special files that give access to various device drivers. * /proc and /sys: Two pseudo filesystems that represent kernel data structures as a hierarchy of directories and files. Many programs and library functions read these files. * kernel modules: If you have configured some parts of your kernel to be modules, they will be here, usually in /lib/modules/[kernel version]. [1. Root Filesystem]$ _ [3/21] >>> Filesystem Hierarchy Standard (FHS) * /bin: programs essential for all
    [Show full text]
  • El Núcleo Linux
    El núcleo Linux Josep Jorba Esteve PID_00174426 GNUFDL • PID_00174426 El núcleo Linux Índice Introducción ................................................... ....... 5 Objetivos ................................................... ............ 6 1. El núcleo del sistema GNU/Linux .............................. 7 2. Personalización o actualización del núcleo................... 15 3. Proceso de configuración y compilación ...................... 18 3.1. Compilación antigua de la rama 2.4.x del núcleo . 20 3.2. Migración de 2.4 a la rama 2.6.x del núcleo. 25 3.3. Compilación de la rama 2.6.x del núcleo . 27 3.4. Compilación del núcleo en Debian (Debian Way)............. 30 4. Aplicación de parches al núcleo ................................ 35 5. Módulos del núcleo ............................................... 38 5.1. DKMS: módulos recompilados dinámicamente . 41 6. Virtualización en el núcleo ..................................... 44 6.1. KVM ............................................ .............. 46 7. Futuro del núcleo y alternativas ............................... 53 8. Taller de configuración del núcleo a las necesidades del usuario................................................... ...... 58 8.1. Configuración del núcleo en Debian . 58 8.2. Configuración del núcleo en Fedora/Red Hat . 61 8.3. Configuración de un núcleo genérico . 63 Resumen ................................................... ............ 66 Actividades ................................................... ......... 67 Bibliografía ..................................................
    [Show full text]
  • Yocto-Slides.Pdf
    Yocto Project and OpenEmbedded Training Yocto Project and OpenEmbedded Training © Copyright 2004-2021, Bootlin. Creative Commons BY-SA 3.0 license. Latest update: October 6, 2021. Document updates and sources: https://bootlin.com/doc/training/yocto Corrections, suggestions, contributions and translations are welcome! embedded Linux and kernel engineering Send them to [email protected] - Kernel, drivers and embedded Linux - Development, consulting, training and support - https://bootlin.com 1/296 Rights to copy © Copyright 2004-2021, Bootlin License: Creative Commons Attribution - Share Alike 3.0 https://creativecommons.org/licenses/by-sa/3.0/legalcode You are free: I to copy, distribute, display, and perform the work I to make derivative works I to make commercial use of the work Under the following conditions: I Attribution. You must give the original author credit. I Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under a license identical to this one. I For any reuse or distribution, you must make clear to others the license terms of this work. I Any of these conditions can be waived if you get permission from the copyright holder. Your fair use and other rights are in no way affected by the above. Document sources: https://github.com/bootlin/training-materials/ - Kernel, drivers and embedded Linux - Development, consulting, training and support - https://bootlin.com 2/296 Hyperlinks in the document There are many hyperlinks in the document I Regular hyperlinks: https://kernel.org/ I Kernel documentation links: dev-tools/kasan I Links to kernel source files and directories: drivers/input/ include/linux/fb.h I Links to the declarations, definitions and instances of kernel symbols (functions, types, data, structures): platform_get_irq() GFP_KERNEL struct file_operations - Kernel, drivers and embedded Linux - Development, consulting, training and support - https://bootlin.com 3/296 Company at a glance I Engineering company created in 2004, named ”Free Electrons” until Feb.
    [Show full text]
  • MANAGING a REAL-TIME EMBEDDED LINUX PLATFORM with BUILDROOT John Diamond, Kevin Martin Fermi National Accelerator Laboratory, Batavia, IL 60510
    MANAGING A REAL-TIME EMBEDDED LINUX PLATFORM WITH BUILDROOT John Diamond, Kevin Martin Fermi National Accelerator Laboratory, Batavia, IL 60510 Desktop distributions are an awkward Buildroot + ucLibc + Busybox + RTAI Quantitative Results implementation of an Embedded RTOS Buildroot – downloads, unpacks, • Whole build process is automated resulting in • Architecture-dependent binary configures, compiles and installs system much quicker build times (hours not days) software automatically • Kernel and root filesystem size: 3.5 MB – 20 packages uClibc – Small-footprint standard C library MB (reduction of 99%) • Loaded with unnecessary software Busybox – all-in-one UNIX utilities and shell • Boot-time: ~9 seconds • Huge footprints RTAI – Real-Time Linux extensions = Qualitative Results • Allows integration with revision control into First Try: Build Linux from Source the platform development process, making it • Success! But.. 2. Buildroot’s menuconfig generates a package configuration file easier to manage an ecosystem of targets • Is as difficult as it sounds and kernel configuration file • Community support for x86 & ARM targets Linux Kernel • Overwhelming number of packages and Configuration gives us confidence that future targets can be patches Package supported without much effort 1. Developer Configuration • No version control configures build via Buildroot’s • Cross-compile even more headaches menuconfig Internet Build Process Power Supply Control Quench Protection Git / CVS / SVN and Regulation for the System for Tevatron Did not do what we needed: Fermilab Linac Electron Lens (TEL II) 3. The build process pulls 4. The output from the software packages from build process is a kernel • Small-footprint network bootable image the internet and custom bzImage bzImage file with an softare packages from a integrated root filesystem ARM Cortex A-9 source code repository file PC/104 AMD • Automated build system Geode SBC Beam Position Monitor Power Supply Control prototype for Fermilab and Regulation for • Support for multiple architectures 5.
    [Show full text]
  • Cross-Compiler Bipartite Vulnerability Search
    electronics Article Cross-Compiler Bipartite Vulnerability Search Paul Black * and Iqbal Gondal Internet Commerce Security Laboratory (ICSL), Federation University, Ballarat 3353, Australia; [email protected] * Correspondence: [email protected] Abstract: Open-source libraries are widely used in software development, and the functions from these libraries may contain security vulnerabilities that can provide gateways for attackers. This paper provides a function similarity technique to identify vulnerable functions in compiled programs and proposes a new technique called Cross-Compiler Bipartite Vulnerability Search (CCBVS). CCBVS uses a novel training process, and bipartite matching to filter SVM model false positives to improve the quality of similar function identification. This research uses debug symbols in programs compiled from open-source software products to generate the ground truth. This automatic extraction of ground truth allows experimentation with a wide range of programs. The results presented in the paper show that an SVM model trained on a wide variety of programs compiled for Windows and Linux, x86 and Intel 64 architectures can be used to predict function similarity and that the use of bipartite matching substantially improves the function similarity matching performance. Keywords: malware similarity; function similarity; binary similarity; machine-learning; bipar- tite matching 1. Introduction Citation: Black, P.; Gondal, I. Cross-Compiler Bipartite Function similarity techniques are used in the following activities, the triage of mal- Vulnerability Search. Electronics 2021, ware [1], analysis of program patches [2], identification of library functions [3], analysis of 10, 1356. https://doi.org/10.3390/ code authorship [4], the identification of similar function pairs to reduce manual analysis electronics10111356 workload, [5], plagiarism analysis [6], and for vulnerable function identification [7–9].
    [Show full text]
  • Enabling Hardware Accelerated Video Decode on Intel® Atom™ Processor D2000 and N2000 Series Under Fedora 16
    Enabling Hardware Accelerated Video Decode on Intel® Atom™ Processor D2000 and N2000 Series under Fedora 16 Application Note October 2012 Order Number: 509577-003US INFORMATIONLegal Lines and Disclaimers IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT. A “Mission Critical Application” is any application in which failure of the Intel Product could result, directly or indirectly, in personal injury or death. SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS FOR ANY SUCH MISSION CRITICAL APPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, AND THE DIRECTORS, OFFICERS, AND EMPLOYEES OF EACH, HARMLESS AGAINST ALL CLAIMS COSTS, DAMAGES, AND EXPENSES AND REASONABLE ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY CLAIM OF PRODUCT LIABILITY, PERSONAL INJURY, OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION, WHETHER OR NOT INTEL OR ITS SUBCONTRACTOR WAS NEGLIGENT IN THE DESIGN, MANUFACTURE, OR WARNING OF THE INTEL PRODUCT OR ANY OF ITS PARTS. Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked “reserved” or “undefined”.
    [Show full text]