MASARYK UNIVERSITY FACULTY}w¡¢£¤¥¦§¨ OF I !"#$%&'()+,-./012345<yA|NFORMATICS Open Build Service migration to Fedora DIPLOMA THESIS Bc. Josef Stˇríbný Brno, Fall 2014 Declaration Hereby I declare, that this paper is my original authorial work, which I have worked out by my own. All sources, references and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source. Advisor: Mgr. Marek Grác, Ph.D. ii Acknowledgement I would love to thank my parents for their great support throughout my university studies. Their hard work was essential part of making this thesis a reality. I would also like to thank Marek Grác, Mirek Suchý and Adrian Schröter for their inputs and consultation during the writing of this thesis. iii Abstract The aim of this thesis is to migrate the Open Build Service (OBS) to Fedora. The text part describes the current tools for building the Fe- dora RPM packages, the OBS build system and the necessary changes to be done for OBS to be used within the Fedora Project. The practical part deals with the packaging of OBS for Fedora and its infrastruc- ture. iv Keywords OBS, RPM, build system, Fedora v Contents 1 Introduction ............................3 2 RPM-based Distributions ....................5 2.1 RPM .............................5 2.2 rpmbuild ........................... 10 2.3 createrepo .......................... 11 3 Fedora Build Systems and Tooling ............... 13 3.1 Yum .............................. 13 3.2 DNF .............................. 14 3.3 Mock ............................. 14 3.4 Koji .............................. 17 3.4.1 Koji-Hub . 17 3.4.2 Kojid . 17 3.4.3 Koji-Web . 18 3.4.4 Koji-Client . 18 3.4.5 Kojira . 19 3.5 fedpkg ............................ 19 3.6 Bodhi ............................. 19 3.7 Copr ............................. 20 4 Open Build Service ........................ 22 4.1 OBS Server .......................... 22 4.1.1 Repository Server . 23 4.1.2 Scheduler . 23 4.1.3 Dispatcher . 23 4.1.4 Worker . 24 4.1.5 Publisher . 24 4.1.6 Source Server . 24 4.2 OBS API ........................... 24 4.3 obs-build ........................... 27 4.4 obs-signd ........................... 28 4.5 osc ............................... 29 5 Analyses and Design ....................... 31 5.1 Fedora Packaging ...................... 31 5.1.1 Dependencies . 31 5.1.2 OBS . 33 5.2 Fedora System Integration ................. 35 1 5.2.1 systemd . 35 5.2.2 SELinux . 36 5.3 Fedora Tooling Integration ................. 40 6 Conclusion ............................. 41 A Installation and Setup ...................... 45 A.1 Installation .......................... 45 A.2 Setup ............................. 45 B Attachements ........................... 47 2 1 Introduction Fedora is a Linux distribution1 that uses RPM package format as its preferred method of distributing software. To create such binary packages for the whole distribution in a meaningful way Fedora Pro– ject2 developed build system3 for building RPM packages called Koji. Koji is used successfully for years, but the development is rather slow and it misses features that can be already found in the much more profound, general, and feature-complete Open Build Service (OBS) project, a build system originally created for building openSUSE op- erating system packages. The first real consideration to port OBS for the Fedora operat- ing system came from Mirek Suchý, a new maintainer of the Copr project[1], who was preparing the design and architecture of the new Copr build system for the Fedora Project. One of the variants was to leverage the work behind the OBS project and built on top of it since it already offered a full feature set for Copr requirements[2, 3]. Apart from the features provided by OBS, the project has reached a certain maturity level and quite impressive community including companies such as Intel, Nokia, SGI, CRAY and Dell since its pre- miere at FOSDEM in 2006 [27]. As Mirek Suchý later mentions on the Fedora devel mailing list from September 2013 [4], OBS does not have established community within the Fedora Project, it is unfamiliar to the Fedora packagers, lacks the support for Fedora-specific tooling, and therefore it is de- cided not to use OBS as the Copr backend. But the idea of having OBS in Fedora and reconsider its use in the official Fedora Project build systems later stayed alive. Moreover the decision of using the obs- signd, an OBS optional component for signing the built packages, for the Copr package signing was made[5]. So to say, if Fedora as a project should reconsider the use of OBS, the OBS needs to be 1. An operating system created as a collection of packaged software based on the Linux kernel and mostly around a package management system of some sort. 2. A community-based project sponsored by Red Hat to co-ordinate the develop- ment of the free and open source Fedora operating system. 3. Software made to create binary packages for Linux distribution involving com- piling source code into binary code, running automated tests and packaging. 3 1. INTRODUCTION packaged for Fedora with the support for the Fedora specific tool- ing and some awareness of OBS within the Fedora packagers needs to be raised. The objective of this thesis is therefore to cover the Fedora’s cur- rent tooling and build systems, the OBS project description with its differences to the tools used within Fedora, and an analyses of the OBS packaging for the Fedora operating system. The practical outcome of this thesis should be the RPM packages of OBS dependencies, and RPM packages of OBS for Fedora including the optional obs-signd component as this component will be used in the Fedora Copr build system. This thesis is also meant to be used as a future reference for the reconsideration of OBS as a build system for the Fedora distribu- tion. 4 2 RPM-based Distributions On an RPM-based Linux distribution (such as Fedora or openSUSE), all basic software comes pre-installed in a form of binary RPM pack- age including the Linux kernel. Distributing software as such pack- ages helps to skip the compilation step during an installation, re- duce the size (as sources are not needed), verify the origin of the files through package signing and provide a meaningful way how to man- age, maintain and upgrade installed software within the operating system. Even though RPM is a package manager itself, Fedora fea- tures other package managers such as Yum and DNF that work on top of RPM and provide a single user interface to package manage- ment featuring advanced dependency solving. 2.1 RPM The RPM Package Manager (RPM) is a powerful command line driven package management system capable of installing, uninstalling, veri- fying, querying, and updating computer software packages, and also the name for the associated file format[6, 28]. Each RPM package con- sists of an archive of files along with additional information about the package including helper scripts and file attributes to install or erase the package. Although RPM can be used directly to install, re- move or update a package, management of RPM packages are on modern RPM-based operating systems left to other tools like Yum or DNF. This is mainly due to the support for package dependency solving and quering external repositories. Hovewer, RPM is still used directly for quering and verifying installed packages: # Find configuration files for Apache HTTP server $ rpm -qc httpd /etc/httpd/conf.d/autoindex.conf /etc/httpd/conf.d/userdir.conf /etc/httpd/conf.d/welcome.conf /etc/httpd/conf.modules.d/00-base.conf /etc/httpd/conf.modules.d/00-dav.conf /etc/httpd/conf.modules.d/00-lua.conf 5 2. RPM-BASED DISTRIBUTIONS /etc/httpd/conf.modules.d/00-mpm.conf /etc/httpd/conf.modules.d/00-proxy.conf /etc/httpd/conf.modules.d/00-systemd.conf /etc/httpd/conf.modules.d/01-cgi.conf /etc/httpd/conf/httpd.conf /etc/httpd/conf/magic /etc/logrotate.d/httpd /etc/sysconfig/htcacheclean /etc/sysconfig/httpd An RPM package is always created from its Source RPM (SRPM) which contains the package sources (upstream source files, patches) and an RPM specification file. This specification file can carry impor- tant metadata such as license, software category, package run-time and build-time dependencies, and contains the information on how to build the RPM packages. A specification preamble could look like the following: %global gem_name activerecord Summary: Implements the~ActiveRecord pattern for ORM Name: rubygem-%{gem_name} Epoch: 1 Version: 4.1.1 Release: 1%{?dist} Group: Development/Languages License: MIT URL: http://www.rubyonrails.org Source0: http://rubygems.org/downloads/ activerecord-%{version}.gem # git clone http://github.com/rails/rails.git # cd rails/activerecord/ # git checkout v4.1.1 # tar czvf activerecord-4.1.1-tests.tgz test/ Source1: activerecord-%{version}-tests.tgz BuildRequires: rubygems-devel BuildRequires: rubygem(bcrypt) BuildRequires: rubygem(activesupport) = %{version} 6 2. RPM-BASED DISTRIBUTIONS BuildRequires: rubygem(activemodel) = %{version} BuildRequires: rubygem(sqlite3) BuildRequires: rubygem(erubis) BuildRequires: rubygem(mocha) BuildRequires: rubygem(arel) >= 5.0.0 BuildRequires: rubygem(arel) < 6.0.0 BuildRequires: rubygem(minitest) BuildRequires: rubygem(rack) BuildArch: noarch %description Implements the~ActiveRecord pattern (Fowler, PoEAA ) for ORM. It ties database tables and classes together for business objects, like Customer or Subscription, that can find, save, and destroy themselves without resorting to manual SQL. %package doc Summary: Documentation for %{name} Group: Documentation Requires: %{name} = %{epoch}:%{version}-%{release} BuildArch: noarch %description doc Documentation for %{name} The specification file is full of macros[7] that look like %{macro} or %macro. Macros are meaningful placeholders that encapsulate some common logic or name. This way we can define the name of our package at the top of the specification file with %global and change the name of the package always at one place. Other globally defined macros come from /usr/lib/rpm/macros, /etc/rpm/macros.* and /.rpmmacros files and are also used to provide a unified logic or represent a naming convention in various parts of the build process.