DOCSLIB.ORG

Functional Package Management with Guix Ludovic Courtès

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Functional Package Management with Guix Ludovic Courtès Functional Package Management with Guix Ludovic Courtès To cite this version: Ludovic Courtès. Functional Package Management with Guix. European Lisp Symposium, Jun 2013, Madrid, Spain. hal-00824004 HAL Id: hal-00824004 https://hal.inria.fr/hal-00824004 Submitted on 20 May 2013 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Functional Package Management with Guix Ludovic Courtès Bordeaux, France [email protected] ABSTRACT 1. INTRODUCTION We describe the design and implementation of GNU Guix, a GNU Guix1 is a purely functional package manager for the purely functional package manager designed to support a com- GNU system [20], and in particular GNU/Linux. Pack- plete GNU/Linux distribution. Guix supports transactional age management consists in all the activities that relate upgrades and roll-backs, unprivileged package management, to building packages from source, honoring the build-time per-user profiles, and garbage collection. It builds upon the and run-time dependencies on packages, installing, removing, low-level build and deployment layer of the Nix package man- and upgrading packages in user environments. In addition ager. Guix uses Scheme as its programming interface. In to these standard features, Guix supports transactional up- particular, we devise an embedded domain-specific language grades and roll-backs, unprivileged package management, (EDSL) to describe and compose packages. We demonstrate per-user profiles, and garbage collection. Guix comes with a how it allows us to benefit from the host general-purpose distribution of user-land free software packages. programming language while not compromising on expres- siveness. Second, we show the use of Scheme to write build Guix seeks to empower users in several ways: by offering the programs, leading to a \two-tier" programming system. uncommon features listed above, by providing the tools that allow users to formally correlate a binary package and the Categories and Subject Descriptors \recipes" and source code that led to it|furthering the spirit D.4.5 [Operating Systems]: Reliability; D.4.5 [Operating of the GNU General Public License|, by allowing them to Systems]: System Programs and Utilities; D.1.1 [Software]: customize the distribution, and by lowering the barrier to Applicative (Functional) Programming entry in distribution development. The keys toward these goals are the implementation of a General Terms purely functional package management paradigm, and the use Languages, Management, Reliability of both declarative and lower-level programming interfaces (APIs) embedded in Scheme. To that end, Guix reuses Keywords the package storage and deployment model implemented by Functional package management, Scheme, Embedded domain- the Nix functional package manager [8]. On top of that, it specific language provides Scheme APIs, and in particular embedded domain- specific languages (EDSLs) to describe software packages and their build system. Guix also uses Scheme for programs and libraries that implement the actual package build processes, leading to a \two-tier" system. This paper focuses on the programming techniques imple- mented by Guix. Our contribution is twofold: we demon- strate that use of Scheme and EDSLs achieves expressiveness comparable to that of Nix's DSL while providing a richer and extensible programming environment; we further show that Scheme is a profitable alternative to shell tools when it comes to package build programs. Section 2 first gives some background on functional package management and Copyright c 2013 Ludovic Courtès its implementation in Nix. Section 3 describes the design Permission is granted to copy, distribute and/or modify this document un- and implementation of Guix's programming and packaging der the terms of the GNU Free Documentation License, Version 1.3 or any interfaces. Section 4 provides an evaluation and discussion of later version published by the Free Software Foundation; with no Invariant the current status of Guix. Section 5 presents related work, Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the li- and Section 6 concludes. cense is available at http://www.gnu.org/licenses/gfdl.html. The source of this document is available from http://git.sv.gnu. org/cgit/guix/maintenance.git. European Lisp Symposium 2013, Madrid, Spain 1http://www.gnu.org/software/guix/ 2. BACKGROUND not transactional; interrupting them may leave the system in This section describes the functional package management an undefined, or even unusable state, where some of the files paradigm and its implementation in Nix. It then shows how have been altered. Third, rolling back to a previous system Guix differs, and what the rationale is. configuration is practically impossible, due to the absence of a mechanism to formally describe the system's configuration. 2.1 Functional Package Management Functional package management is a paradigm whereby the Nix attempts to address these shortcomings through the build and installation process of a package is considered as a functional software deployment paradigm: installed packages pure function, without any side effects. This is in contrast are immutable, and build processes are regarded as pure with widespread approaches to package build and installation functions, as explained before. Thanks to this property, where the build process usually has access to all the software it implements transparent source/binary deployment: the installed on the machine, regardless of what its declared directory name of a build result encodes all the inputs of its inputs are, and where installation modifies files in place. build process, so if a trusted server provides that directory, then it can be directly downloaded from there, avoiding the Functional package management was pioneered by the Nix need for a local build. package manager [8], which has since matured to the point of managing a complete GNU/Linux distribution [9]. To allow Each user has their own profile, which contains symbolic links build processes to be faithfully regarded as pure functions, to the /nix/store directories of installed packages. Thus, Nix can run them in a chroot environment that only contains users can install packages independently, and the actual the inputs it explicitly declared; thus, it becomes impossible storage is shared when several users install the very same for a build process to use, say, Perl, if that package was package in their profile. Nix comes with a garbage collector, not explicitly declared as an input of the build process. In which has two main functions: with conservative scanning, addition, Nix maps the list of inputs of a build process to a it can determine what packages a build output refers to; and statistically unique file system name; that file name is used to upon user request, it can delete any packages not referenced identify the output of the build process. For instance, a par- via any user profile. ticular build of GNU Emacs may be installed in /nix/store/- v9zic07iar8w90zcy398r745w78a7lqs-emacs-24.2, based on To describe and compose build processes, Nix implements a cryptographic hash of all the inputs to that build process; its own domain-specific language (DSL), which provides a changing the compiler, configuration options, build scripts, convenient interface to the build and storage mechanisms or any other inputs to the build process of Emacs yields described above. The Nix language is purely functional, a different name. This is a form of on-disk memoization, lazy, and dynamically typed; it is similar to that of the Vesta with the /nix/store directory acting as a cache of \function software configuration system [11]. It comes with a handful of results"|i.e., a cache of installed packages. Directories under built-in data types, and around 50 primitives. The primitive /nix/store are immutable. to describe a build process is derivation. This direct mapping from build inputs to the result's directory 1: derivation { name is basis of the most important properties of a functional 2: name = "example-1.0"; package manager. It means that build processes are regarded 3: builder = "${./static-bash}"; as referentially transparent. To put it differently, instead 4: args = [ "-c" "echo hello > $out" ]; of merely providing pre-built binaries and/or build recipes, 5: system = "x86_64-linux"; functional package managers provide binaries, build recipes, 6: } and in effect a guarantee that a given binary matches a given build recipe. Figure 1: Call to the derivation primitive in the Nix language. 2.2 Nix The idea of purely functional package started by making an Figure 1 shows code that calls the derivation function with analogy between programming language paradigms and soft- one argument, which is a dictionary. It expects at least ware deployment techniques [8]. The authors observed that, the four key/value pairs shown above; together, they define in essence, package management tools typically used on free the build process and its inputs. The result is a derivation, operating systems, such as RPM and Debian's APT, imple- which is essentially the promise of a build. The derivation ment an imperative software deployment paradigm. Package has a low-level on-disk representation independent of the Nix installation, removal, and upgrade are all done in-place, by language|in other words, derivations are to the Nix language mutating the operating system's state. Likewise, changes what assembly is to higher-level programming languages. to the operating system's configuration are done in-place by When this derivation is instantiated|i.e., built|, it runs changing configuration files. the command static-bash -c "echo hello > $out" in a chroot that contains nothing but the static-bash file; in This imperative approach has several drawbacks.
Load more

Recommended publications
  • Glibc and System Calls Documentation Release 1.0
    Glibc and System Calls Documentation Release 1.0 Rishi Agrawal <[email protected]> Dec 28, 2017 Contents 1 Introduction 1 1.1 Acknowledgements...........................................1 2 Basics of a Linux System 3 2.1 Introduction...............................................3 2.2 Programs and Compilation........................................3 2.3 Libraries.................................................7 2.4 System Calls...............................................7 2.5 Kernel.................................................. 10 2.6 Conclusion................................................ 10 2.7 References................................................ 11 3 Working with glibc 13 3.1 Introduction............................................... 13 3.2 Why this chapter............................................. 13 3.3 What is glibc .............................................. 13 3.4 Download and extract glibc ...................................... 14 3.5 Walkthrough glibc ........................................... 14 3.6 Reading some functions of glibc ................................... 17 3.7 Compiling and installing glibc .................................... 18 3.8 Using new glibc ............................................ 21 3.9 Conclusion................................................ 23 4 System Calls On x86_64 from User Space 25 4.1 Setting Up Arguements......................................... 25 4.2 Calling the System Call......................................... 27 4.3 Retrieving the Return Value......................................
    [Show full text]
  • Red Hat Enterprise Linux 6 Developer Guide
    Red Hat Enterprise Linux 6 Developer Guide An introduction to application development tools in Red Hat Enterprise Linux 6 Dave Brolley William Cohen Roland Grunberg Aldy Hernandez Karsten Hopp Jakub Jelinek Developer Guide Jeff Johnston Benjamin Kosnik Aleksander Kurtakov Chris Moller Phil Muldoon Andrew Overholt Charley Wang Kent Sebastian Red Hat Enterprise Linux 6 Developer Guide An introduction to application development tools in Red Hat Enterprise Linux 6 Edition 0 Author Dave Brolley [email protected] Author William Cohen [email protected] Author Roland Grunberg [email protected] Author Aldy Hernandez [email protected] Author Karsten Hopp [email protected] Author Jakub Jelinek [email protected] Author Jeff Johnston [email protected] Author Benjamin Kosnik [email protected] Author Aleksander Kurtakov [email protected] Author Chris Moller [email protected] Author Phil Muldoon [email protected] Author Andrew Overholt [email protected] Author Charley Wang [email protected] Author Kent Sebastian [email protected] Editor Don Domingo [email protected] Editor Jacquelynn East [email protected] Copyright © 2010 Red Hat, Inc. and others. The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version. Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
    [Show full text]
  • Download the Specification
    Internationalizing and Localizing Applications in Oracle Solaris Part No: E61053 November 2020 Internationalizing and Localizing Applications in Oracle Solaris Part No: E61053 Copyright © 2014, 2020, Oracle and/or its affiliates. License Restrictions Warranty/Consequential Damages Disclaimer This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. Warranty Disclaimer The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. Restricted Rights Notice If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable: U.S. GOVERNMENT END USERS: Oracle programs (including any operating system, integrated software, any programs embedded, installed or activated on delivered hardware, and modifications of such programs) and Oracle computer documentation or other Oracle data delivered to or accessed by U.S. Government end users are "commercial
    [Show full text]
  • Bringing GNU Emacs to Native Code
    Bringing GNU Emacs to Native Code Andrea Corallo Luca Nassi Nicola Manca [email protected] [email protected] [email protected] CNR-SPIN Genoa, Italy ABSTRACT such a long-standing project. Although this makes it didactic, some Emacs Lisp (Elisp) is the Lisp dialect used by the Emacs text editor limitations prevent the current implementation of Emacs Lisp to family. GNU Emacs can currently execute Elisp code either inter- be appealing for broader use. In this context, performance issues preted or byte-interpreted after it has been compiled to byte-code. represent the main bottleneck, which can be broken down in three In this work we discuss the implementation of an optimizing com- main sub-problems: piler approach for Elisp targeting native code. The native compiler • lack of true multi-threading support, employs the byte-compiler’s internal representation as input and • garbage collection speed, exploits libgccjit to achieve code generation using the GNU Com- • code execution speed. piler Collection (GCC) infrastructure. Generated executables are From now on we will focus on the last of these issues, which con- stored as binary files and can be loaded and unloaded dynamically. stitutes the topic of this work. Most of the functionality of the compiler is written in Elisp itself, The current implementation traditionally approaches the prob- including several optimization passes, paired with a C back-end lem of code execution speed in two ways: to interface with the GNU Emacs core and libgccjit. Though still a work in progress, our implementation is able to bootstrap a func- • Implementing a large number of performance-sensitive prim- tional Emacs and compile all lexically scoped Elisp files, including itive functions (also known as subr) in C.
    [Show full text]
  • Debian Installation Manual
    Powered by Universal Speech Solutions LLC MRCP Deb Installation Manual Administrator Guide Revision: 70 Created: February 7, 2015 Last updated: March 15, 2021 Author: Arsen Chaloyan Powered by Universal Speech Solutions LLC | Overview 1 Table of Contents 1 Overview ............................................................................................................................................... 3 1.1 Applicable Versions ............................................................................................................ 3 1.2 Supported Distributions ...................................................................................................... 3 1.3 Authentication ..................................................................................................................... 3 2 Installing Deb Packages Using Apt-Get ............................................................................................... 4 2.1 Repository Configuration ................................................................................................... 4 2.2 GnuPG Key ......................................................................................................................... 4 2.3 Repository Update .............................................................................................................. 4 2.4 UniMRCP Client Installation .............................................................................................. 5 2.5 UniMRCP Server Installation ............................................................................................
    [Show full text]
  • Reference Architecture Specification
    Linux based 3G Multimedia Mobile-phone Reference Architecture Specification Draft 1.0 NEC Corporation Panasonic Mobile Communication Ltd. CE Linux Forum Technical Document Contents Preface......................................................................................................................................iii 1. Introduction.............................................................................................................................1 2. Scope .....................................................................................................................................1 3. Reference................................................................................................................................1 4. Definitions and abbreviations ....................................................................................................1 5. Architecture ............................................................................................................................3 5.1 A-CPU ..................................................................................................................................3 5.2 C-CPU..................................................................................................................................5 6. Description of functional entities ..............................................................................................5 6.1 Kernel....................................................................................................................................5
    [Show full text]
  • GNU Guix Cookbook Tutorials and Examples for Using the GNU Guix Functional Package Manager
    GNU Guix Cookbook Tutorials and examples for using the GNU Guix Functional Package Manager The GNU Guix Developers Copyright c 2019 Ricardo Wurmus Copyright c 2019 Efraim Flashner Copyright c 2019 Pierre Neidhardt Copyright c 2020 Oleg Pykhalov Copyright c 2020 Matthew Brooks Copyright c 2020 Marcin Karpezo Copyright c 2020 Brice Waegeneire Copyright c 2020 Andr´eBatista Copyright c 2020 Christine Lemmer-Webber Copyright c 2021 Joshua Branson Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled \GNU Free Documentation License". i Table of Contents GNU Guix Cookbook ::::::::::::::::::::::::::::::: 1 1 Scheme tutorials ::::::::::::::::::::::::::::::::: 2 1.1 A Scheme Crash Course :::::::::::::::::::::::::::::::::::::::: 2 2 Packaging :::::::::::::::::::::::::::::::::::::::: 5 2.1 Packaging Tutorial:::::::::::::::::::::::::::::::::::::::::::::: 5 2.1.1 A \Hello World" package :::::::::::::::::::::::::::::::::: 5 2.1.2 Setup:::::::::::::::::::::::::::::::::::::::::::::::::::::: 8 2.1.2.1 Local file ::::::::::::::::::::::::::::::::::::::::::::: 8 2.1.2.2 `GUIX_PACKAGE_PATH' ::::::::::::::::::::::::::::::::: 9 2.1.2.3 Guix channels ::::::::::::::::::::::::::::::::::::::: 10 2.1.2.4 Direct checkout hacking:::::::::::::::::::::::::::::: 10 2.1.3 Extended example ::::::::::::::::::::::::::::::::::::::::
    [Show full text]
  • Using Ptxdist on Mac OS Based on Ptxdist 2012.04
    Using PTXdist on Mac OS based on PTXdist 2012.04 Bernhard Walle∗ 2012-04-20 ∗[email protected] Contents 1. Motivation 3 2. Basics 4 2.1. Getting OpenSource Software on Mac OS.........................4 2.2. Preparing the Hard Disk..................................5 2.2.1. Creating and Using a Sparse Bundle ........................6 3. Installing PTXdist7 3.1. Requirements........................................7 3.1.1. Mac OS.......................................7 3.1.2. Host compiler....................................7 3.2. GNU Tools..........................................7 3.3. Installation..........................................8 3.4. First setup..........................................9 4. Building an OSELAS.Toolchain 11 5. Building an Embedded Linux project 12 5.1. Using OSELAS.BSP-Pengutronix-Generic......................... 12 5.1.1. Getting the BSP................................... 12 5.1.2. Building FSF GCC.................................. 13 5.1.3. Building Qemu................................... 13 5.1.4. Running the System................................ 13 5.2. Using real Hardware.................................... 14 6. Limitations 15 6.1. Building UBI or JFFS2 images................................ 15 6.2. Linux kernel......................................... 15 6.3. Bootloader.......................................... 15 7. Using a Mac for Embedded Linux Development 16 7.1. Using a serial console.................................... 16 7.2. Exporting directories via NFS............................... 16 7.3. Mounting ext2 partitions.................................. 18 A. About this Document 19 B. Change History 20 Bibliography 21 2 1. Motivation PTXdist (http://www.ptxdist.org) is a great way to build Embedded Linux systems. It downloads all required components, conVgures them for cross-compilation and Vnally builds a target image and/or target packages. In addition, it provides an easy way to build a cross-toolchain for most common processors. Read [2] for a description how to use PTXdist.
    [Show full text]
  • 2004 USENIX Annual Technical Conference
    USENIX Association Proceedings of the FREENIX Track: 2004 USENIX Annual Technical Conference Boston, MA, USA June 27–July 2, 2004 © 2004 by The USENIX Association All Rights Reserved For more information about the USENIX Association: Phone: 1 510 528 8649 FAX: 1 510 548 5738 Email: [email protected] WWW: http://www.usenix.org Rights to individual papers remain with the author or the author's employer. Permission is granted for noncommercial reproduction of the work for educational or research purposes. This copyright notice must be included in the reproduced paper. USENIX acknowledges all trademarks herein. The NetBSD Update System Alistair Crooks, The NetBSD Project 9th April 2004 Abstract driving force behind the use of pkgsrc or NetBSD - rather, this is a description of a facility which is This paper explains the needs for a binary patch and used in NetBSD and which can be used on any other update system, and explains the background and im- operating system to augment the standard facilities plementation of NetBSD-update, a binary update fa- which are in place. cility for NetBSD. The implementation is then anal- ysed, and some lessons drawn for others who may be interested in implementing their own binary up- Driving Forces for a Binary Patch and date system using the NetBSD pkgsrc tools, which Update System are available for many operating systems and envi- ronments already. It is now common to nd rewalls in large and small organisations, preventing malign access, and protect- ing the organisation from intrusion and other attacks. The NetBSD Binary Update Sys- It would not be prudent to have a C compiler in- tem stalled on such a machine - its use should be that of a gatekeeper, as a bouncer with an attitude, keep- Unix, Linux and the BSD operating systems have ing anything suspicious out, and not allowing anyone traditionally been distributed in source format, and who does manage to penetrate the defences to use users and administrators have had a long tradition any tools to break further into the infrastructure.
    [Show full text]
  • Wisp: Whitespace to Lisp
    wisp: Whitespace to Lisp Dr. Arne Babenhauserheide <2013-03-26 Di> » I love the syntax of Python, but crave the simplicity and power of Lisp.« display "Hello World!" 7! (display "Hello World!") define : factorial n (define (factorial n) if : zero? n 7! (if (zero? n) . 1 1 * n : factorial {n - 1} (* n (factorial {n - 1})))) Wisp basics • Wisp turns indentation into lisp expressions. • Why Wisp? • Get it – from its Mercurial repository: hg clone https://hg.sr.ht/~arnebab/wisp – Or via GNU Guix: guix install guile guile-wisp – Or via the package guile-wisp-hg for Arch Linux. – Or via ./configure; make install from the releases. • See more examples and tests. 1 »ArneBab’s alternate sexp syntax is best I’ve seen; pythonesque, hides parens but keeps power« — Christopher Webber in twitter, in identi.ca and in their blog: Wisp: Lisp, minus the parentheses ~ wow ~ »Wisp allows people to see code how Lispers perceive it. Its structure becomes apparent.« — Ricardo Wurmus in IRC, paraphrasing the wisp statement from his talk at FOSDEM 2019 about Guix for reproducible science in HPC. ^¨ Yay! ^¨ with (open-file "with.w" "r") as port format #t "~a\n" : read port Familiar with-statement in 25 lines. 2 ↓ skip updates and releases ↓ Update (2021-05-21): If you use GNU Guix, you can get an instant wisp shell with a single command: guix environment --ad-hoc guile guile-wisp -- wisp That’s the guix wisp insta-REPL ^¨ $ guix environment --ad-hoc guile guile-wisp -- wisp GNU Guile 3.0.7 Copyright (C) 1995-2021 Free Software Foundation, Inc.
    [Show full text]
  • Bash Guide for Beginners
    Bash Guide for Beginners Machtelt Garrels Garrels BVBA <tille wants no spam _at_ garrels dot be> Version 1.11 Last updated 20081227 Edition Bash Guide for Beginners Table of Contents Introduction.........................................................................................................................................................1 1. Why this guide?...................................................................................................................................1 2. Who should read this book?.................................................................................................................1 3. New versions, translations and availability.........................................................................................2 4. Revision History..................................................................................................................................2 5. Contributions.......................................................................................................................................3 6. Feedback..............................................................................................................................................3 7. Copyright information.........................................................................................................................3 8. What do you need?...............................................................................................................................4 9. Conventions used in this
    [Show full text]
  • Introduction to the Nix Package Manager
    Introduction Nix concepts Usage examples Conclusion Introduction to the Nix Package Manager Millian Poquet 2021-05-12 — Datamove (Inria) seminar 1 / 16 Introduction Nix concepts Usage examples Conclusion Why Nix? Control your software environment! Programs/libraries/scripts/configurations + versions Why is it important for us? Use/develop/test/distribute software Manually install many dependencies? No, just type nix-shell Shared env for whole team (tunable) and test machines Bug only on my machine? Means this is hardware or OS related Reproducible research Repeat experiment in exact same environment Introduce or test variation 2 / 16 Introduction Nix concepts Usage examples Conclusion What is Nix? Nix: package manager Download and install packages Shell into well-defined environment (like virtualenv) Transactional (rollback works) Cross-platform: Linux, macOS, Windows (WSL) Nix: programming language Define packages Define environments (set of packages) Functional, DSL NixOS: Linux distribution Declarative system configuration Uses the Nix language Transactional (rollback still works) 3 / 16 Introduction Nix concepts Usage examples Conclusion Nix in numbers Started in 2003 Nix 1: 10k commits, 28k C++ LOC Nixpkgs 2: 285k commits, 55k packages 3 1. https://github.com/NixOS/nix 2. https://github.com/NixOS/nixpkgs 3. https://repology.org/repositories/statistics 4 / 16 Introduction Nix concepts Usage examples Conclusion Presentation summary 2 Nix concepts 3 Usage examples 4 Conclusion 5 / 16 Introduction Nix concepts Usage examples Conclusion Traditional
    [Show full text]