Masaryk University Faculty of Informatics Modern development workflow for Fedora GNU/Linux distribution Master’s Thesis Bc. František Lachman Brno, Spring 2019 Declaration Hereby I declare that this paper is my original authorial work, which I have worked out on my own. All sources, references, and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source. Bc. František Lachman Advisor: Ing. Milan Brož, Ph.D. i Acknowledgements I would like to express my appreciation to my supervisor, Doctor Milan Brož, for his thorough mentoring and all the valuable advice he gave me. I would like to thank my technical consultant, Tomáš Tomeček, for guiding me during the last year. For all discussions, reviews and suggestions. I really appreciate it. Also, I would like to thank all members of the Packit team for being very helpful to me. Finally, I would like to thank my wife for support and understanding during the work on the thesis. The thesis would not exist without any of you. iii Abstract In this thesis, we will look at the current state of the software packaging in the current GNU/Linux distributions. Using three representatives (Debian, Arch Linux and Fedora), we will see various parts of the workflow and compare the different approaches to the same problems. The goal of the thesis is to propose a modern development workflow for Fedora GNU/Linux distribution. We will describe related tools and partial solutions, that exist for this distribution. Finally, we will discuss the proposed solution and examples of the real setup of the prototype. The thesis is licensed under a Creative Commons Attribution 4.0 International license. iv Keywords GNU/Linux, UNIX, packaging, distributions, git, Fedora, RPM v Contents Introduction 1 1 Current state of three Linux distributions 3 1.1 Quick overview of the distributions ..............3 1.2 Package Specification .....................5 1.3 Storing distribution data ...................8 1.4 Distribution versioning scheme, repositories ......... 11 1.5 Packager workflow ...................... 14 1.6 Building packages ....................... 16 1.7 Security, formal requirements ................. 18 1.8 Package sources, editing of the distribution packages ..... 20 2 Existing solutions 21 2.1 Similar or related tools .................... 21 2.2 Custom solutions ....................... 26 3 Proposed solution for Fedora ecosystem 31 3.1 Key principles ......................... 31 3.2 Source-git ........................... 34 3.3 Upstream-downstream format conversion .......... 36 3.4 Workflow ........................... 38 4 Implementation 43 4.1 Security and compliance ................... 51 4.2 Implemented requirements .................. 53 5 Use cases 55 Conclusion 59 Bibliography 61 Glossary 69 A Appendices 75 A.1 Packit CLI ........................... 75 A.2 Sources ............................ 83 vii List of Figures 1.1 Debian FTP server 8 1.2 Arch package SVN tree 9 1.3 dist-git scheme 9 1.4 Pagure over dist-git 10 2.1 Spec file template for rpkg-util (content from [68]) 24 2.2 RDO packaging overview ([63]) 29 3.1 pull-request workflow 32 3.2 source-git repository 34 3.3 Commits in the source-git repository 35 3.4 Transformation of the source-git to dist-git 37 3.5 Automatically created pull-request in dist-git 39 3.6 Automatically created downstream commit 39 3.7 Fedora CI system status for the dist-git pull-request 40 3.8 Synchronize downstream to upstream 40 3.9 Packit-as-a-service integration with Copr 42 4.1 Dependency graph 49 4.2 Packit-as-a-service GitHub application 50 5.1 systemd source-git 58 ix Introduction One of the benefits of using Linux distributions is the ability to install the software on the system easily. There is a significant amount of work that needs to be done after a release of some program to be able to install it as a package on the system. The conversion from the source code to the installable package is not only about compilation. Today’s package managers are responsible for sharing and resolving dependencies, and tracking installed files. There are various ways to achieve these responsibilities. Server distrowatch.com tracks more than three hundred of different Linux distributions1. Many of them share the packaging system, some of them builds on top of others or are influenced by others, but there are many very different ideas and approaches how the packaging canbe done. In the first section of the thesis, we will look at the three important representatives. We will see the description of their solution from the perspective of a maintainer, a person who is responsible for the conversion from source code to the installable package and taking care of that package. The goal of the thesis is to propose a new maintainer workflow for Fedora, one of the Linux distributions. Here are the requirements for the prototype: • Transform the content of the upstream project into a format which Fedora distribution build system understands. • The transformation process should distinguish between the pristine upstream archive and additional changes layered on top; they should not be mixed together by default. • The process has to be configurable by a user. • Submit the changes as a new pull request in Fedora distribution’s code repository. The proposed workflow has to fulfil the following points: • An easy way to contribute: a user can propose a change with a minimal amount of steps. • Maintainers can easily collaborate with upstream projects. The tooling would enable downstream maintainers to pick up 1. By the time of writing of the thesis. 1 fixes from upstream and maintainers are able touse the prototype to report issues upstream. • Provenance: the commits are signed, and it is possible to track down the origin of a certain artefact. • Continuous Integration: contributors are getting feedback for their proposed changes. The second section will describe tools that try to simplify or automate the parts of the maintainers’ workflow. These are used for getting convenient principles and approaches for the Fedora Linux distribution, the target Linux distribution of the proposed solution. The prototype will be described in the third section of the thesis. The section will cover principles and ideas that were used during the implementation. The implementation will be described in the Section 4 and will describe used tools and libraries, internal structure of the code and the conclusion of the requirements. The fifth, and the last, section will show the setup of the projects that were used for demonstrating the use of the prototype. 2 1 Current state of three Linux distributions In this chapter, we are going to show the current state of the art of three distributions of GNU/Linux, the UNIX-like operating system. After a short introduction of the used examples, we are going to discuss several packaging topics to see differences among distributions’ approaches. • As a first model, we are going to use Debian as a representative of deb-based Linux distributions like Ubuntu and Linux Mint. • For the second point of view, Arch Linux is going to be used. • The third example will be Fedora representing distributions based on RPM (e.g. RHEL, Centos or SUSE). Since Fedora will also be used in the implementation part, there is a strong accent on this distribution in the following parts. We will describe its approach more deeply and with more technicalities. 1.1 Quick overview of the distributions 1.1.1 Debian and deb-based distributions Debian is the operating system by The Debian Project, the association who have made common cause to create a free1 operating system [77]. Debian was created in August 1993 and named after the creator Ian Murdock, and his wife, Debra. It uses GNU/Linux or FreeBSD kernel; there is also ongoing work on supporting GNU/Hurd [78]. According to the server distrowatch.com, Debian is used as a base for nearly four hundred distributions2, of which 135 are now active3. We can name Ubuntu, elementary OS, Linux Mint or Kali Linux for a reference. 1. The meaning of the term free in this context and relation to the Open Source can be found in [79]. 2. A current list of Debian derivates can be seen in the following URL: https: //distrowatch.com/search.php?basedon=Debian 3. Active derivates are listed in this URL: https://distrowatch.com/search.php? basedon=Debian&status=Active 3 1. Current state of three Linux distributions The packaging system of Debian, which the derivates inherit will be described later in this chapter. 1.1.2 Arch Linux Arch is a Linux distribution, which is aiming to be simple, agile and lightweight. It was founded in 2002 and was inspired by the Linux distributions like Slackware or CRUX. [55] It is now (29 April 2019) the 16th most popular distribution according to distrowatch.com. Arch is a rolling-release system that is highly configurable and assembled by the user. The “simple” in The Arch Way means without unnecessary additions, modifications, or complications. (It is “simple” from technical, not usability, point of view.) [55] The Arch way4 can be defined (based on [22]) with five core principles: • simplicity, • code-correctness over convenience, • user-centric, • openness, • freedom. 1.1.3 Fedora and RPM-based distributions The history of the Fedora Linux distribution is loosely coupled to Red Hat Linux. According to [59], it started as a repository of newer and more experimental software for Red Hat Linux, one of the earliest Linux distributions. The start of the real distribution (called Fedora Core) was caused by the move from Red Hat Linux to Red Hat Enterprise Linux (RHEL). The Fedora Core was not governed by the Red Hat, but by the newly created Fedora Project. The project was lately renamed to just Fedora and is driven by community with support from Red Hat, which can use Fedora as a test bed for new technologies and application, which would later be included in Red Hat Enterprise Linux.