MASARYK UNIVERSITY FACULTY}w¡¢£¤¥¦§¨ OF I !"#$%&'()+,-./012345 Open Build Service migration to Fedora DIPLOMATHESIS Bc. Josef Stˇríbný Brno, Fall 2014 Declaration Hereby I declare, that this paper is my original authorial work, which I have worked out by my own. All sources, references and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source. Advisor: Mgr. Marek Grác, Ph.D. ii Acknowledgement I would love to thank my parents for their great support throughout my university studies. Their hard work was essential part of making this thesis a reality. I would also like to thank Marek Grác, Mirek Suchý and Adrian Schröter for their inputs and consultation during the writing of this thesis. iii Abstract The aim of this thesis is to migrate the Open Build Service (OBS) to Fedora. The text part describes the current tools for building the Fe- dora RPM packages, the OBS build system and the necessary changes to be done for OBS to be used within the Fedora Project. The practical part deals with the packaging of OBS for Fedora and its infrastruc- ture. iv Keywords OBS, RPM, build system, Fedora v Contents 1 Introduction ...... 3 2 RPM-based Distributions ...... 5 2.1 RPM ...... 5 2.2 rpmbuild ...... 10 2.3 createrepo ...... 11 3 Fedora Build Systems and Tooling ...... 13 3.1 Yum ...... 13 3.2 DNF ...... 14 3.3 Mock ...... 14 3.4 Koji ...... 17 3.4.1 Koji-Hub ...... 17 3.4.2 Kojid ...... 17 3.4.3 Koji-Web ...... 18 3.4.4 Koji-Client ...... 18 3.4.5 Kojira ...... 19 3.5 fedpkg ...... 19 3.6 Bodhi ...... 19 3.7 Copr ...... 20 4 Open Build Service ...... 22 4.1 OBS Server ...... 22 4.1.1 Repository Server ...... 23 4.1.2 Scheduler ...... 23 4.1.3 Dispatcher ...... 23 4.1.4 Worker ...... 24 4.1.5 Publisher ...... 24 4.1.6 Source Server ...... 24 4.2 OBS API ...... 24 4.3 obs-build ...... 27 4.4 obs-signd ...... 28 4.5 osc ...... 29 5 Analyses and Design ...... 31 5.1 Fedora Packaging ...... 31 5.1.1 Dependencies ...... 31 5.1.2 OBS ...... 33 5.2 Fedora System Integration ...... 35 1 5.2.1 systemd ...... 35 5.2.2 SELinux ...... 36 5.3 Fedora Tooling Integration ...... 40 6 Conclusion ...... 41 A Installation and Setup ...... 45 A.1 Installation ...... 45 A.2 Setup ...... 45 B Attachements ...... 47 2 1 Introduction Fedora is a Linux distribution1 that uses RPM package format as its preferred method of distributing software. To create such binary packages for the whole distribution in a meaningful way Fedora Pro– ject2 developed build system3 for building RPM packages called Koji. Koji is used successfully for years, but the development is rather slow and it misses features that can be already found in the much more profound, general, and feature-complete Open Build Service (OBS) project, a build system originally created for building openSUSE op- erating system packages. The first real consideration to port OBS for the Fedora operat- ing system came from Mirek Suchý, a new maintainer of the Copr project[1], who was preparing the design and architecture of the new Copr build system for the Fedora Project. One of the variants was to leverage the work behind the OBS project and built on top of it since it already offered a full feature set for Copr requirements[2, 3]. Apart from the features provided by OBS, the project has reached a certain maturity level and quite impressive community including companies such as Intel, Nokia, SGI, CRAY and Dell since its pre- miere at FOSDEM in 2006 [27]. As Mirek Suchý later mentions on the Fedora devel mailing list from September 2013 [4], OBS does not have established community within the Fedora Project, it is unfamiliar to the Fedora packagers, lacks the support for Fedora-specific tooling, and therefore it is de- cided not to use OBS as the Copr backend. But the idea of having OBS in Fedora and reconsider its use in the official Fedora Project build systems later stayed alive. Moreover the decision of using the obs- signd, an OBS optional component for signing the built packages, for the Copr package signing was made[5]. So to say, if Fedora as a project should reconsider the use of OBS, the OBS needs to be 1. An operating system created as a collection of packaged software based on the Linux kernel and mostly around a package management system of some sort. 2. A community-based project sponsored by Red Hat to co-ordinate the develop- ment of the free and open source Fedora operating system. 3. Software made to create binary packages for Linux distribution involving com- piling source code into binary code, running automated tests and packaging. 3 1. INTRODUCTION packaged for Fedora with the support for the Fedora specific tool- ing and some awareness of OBS within the Fedora packagers needs to be raised. The objective of this thesis is therefore to cover the Fedora’s cur- rent tooling and build systems, the OBS project description with its differences to the tools used within Fedora, and an analyses of the OBS packaging for the Fedora operating system. The practical outcome of this thesis should be the RPM packages of OBS dependencies, and RPM packages of OBS for Fedora including the optional obs-signd component as this component will be used in the Fedora Copr build system. This thesis is also meant to be used as a future reference for the reconsideration of OBS as a build system for the Fedora distribu- tion. 4 2 RPM-based Distributions On an RPM-based Linux distribution (such as Fedora or openSUSE), all basic software comes pre-installed in a form of binary RPM pack- age including the Linux kernel. Distributing software as such pack- ages helps to skip the compilation step during an installation, re- duce the size (as sources are not needed), verify the origin of the files through package signing and provide a meaningful way how to man- age, maintain and upgrade installed software within the operating system. Even though RPM is a package manager itself, Fedora fea- tures other package managers such as Yum and DNF that work on top of RPM and provide a single user interface to package manage- ment featuring advanced dependency solving. 2.1 RPM The RPM Package Manager (RPM) is a powerful command line driven package management system capable of installing, uninstalling, veri- fying, querying, and updating computer software packages, and also the name for the associated file format[6, 28]. Each RPM package con- sists of an archive of files along with additional information about the package including helper scripts and file attributes to install or erase the package. Although RPM can be used directly to install, re- move or update a package, management of RPM packages are on modern RPM-based operating systems left to other tools like Yum or DNF. This is mainly due to the support for package dependency solving and quering external repositories. Hovewer, RPM is still used directly for quering and verifying installed packages: # Find configuration files for Apache HTTP server $ rpm -qc httpd /etc/httpd/conf.d/autoindex.conf /etc/httpd/conf.d/userdir.conf /etc/httpd/conf.d/welcome.conf /etc/httpd/conf.modules.d/00-base.conf /etc/httpd/conf.modules.d/00-dav.conf /etc/httpd/conf.modules.d/00-lua.conf 5 2. RPM-BASED DISTRIBUTIONS /etc/httpd/conf.modules.d/00-mpm.conf /etc/httpd/conf.modules.d/00-proxy.conf /etc/httpd/conf.modules.d/00-systemd.conf /etc/httpd/conf.modules.d/01-cgi.conf /etc/httpd/conf/httpd.conf /etc/httpd/conf/magic /etc/logrotate.d/httpd /etc/sysconfig/htcacheclean /etc/sysconfig/httpd An RPM package is always created from its Source RPM (SRPM) which contains the package sources (upstream source files, patches) and an RPM specification file. This specification file can carry impor- tant metadata such as license, software category, package run-time and build-time dependencies, and contains the information on how to build the RPM packages. A specification preamble could look like the following: %global gem_name activerecord Summary: Implements the~ActiveRecord pattern for ORM Name: rubygem-%{gem_name} Epoch: 1 Version: 4.1.1 Release: 1%{?dist} Group: Development/Languages License: MIT URL: http://www.rubyonrails.org Source0: http://rubygems.org/downloads/ activerecord-%{version}.gem # git clone http://github.com/rails/rails.git # cd rails/activerecord/ # git checkout v4.1.1 # tar czvf activerecord-4.1.1-tests.tgz test/ Source1: activerecord-%{version}-tests.tgz BuildRequires: rubygems-devel BuildRequires: rubygem(bcrypt) BuildRequires: rubygem(activesupport) = %{version} 6 2. RPM-BASED DISTRIBUTIONS BuildRequires: rubygem(activemodel) = %{version} BuildRequires: rubygem(sqlite3) BuildRequires: rubygem(erubis) BuildRequires: rubygem(mocha) BuildRequires: rubygem(arel) >= 5.0.0 BuildRequires: rubygem(arel) < 6.0.0 BuildRequires: rubygem(minitest) BuildRequires: rubygem(rack) BuildArch: noarch %description Implements the~ActiveRecord pattern (Fowler, PoEAA ) for ORM. It ties database tables and classes together for business objects, like Customer or Subscription, that can find, save, and destroy themselves without resorting to manual SQL. %package doc Summary: Documentation for %{name} Group: Documentation Requires: %{name} = %{epoch}:%{version}-%{release} BuildArch: noarch %description doc Documentation for %{name} The specification file is full of macros[7] that look like %{macro} or %macro. Macros are meaningful placeholders that encapsulate some common logic or name. This way we can define the name of our package at the top of the specification file with %global and change the name of the package always at one place. Other globally defined macros come from /usr/lib/rpm/macros, /etc/rpm/macros.* and /.rpmmacros files and are also used to provide a unified logic or represent a naming convention in various parts of the build process. However, there are also macros defined by RPM such as %package, %description or %prep that are here for section definition and flow 7 2. RPM-BASED DISTRIBUTIONS control. Metadata specification for the package come in fields that are called tags in RPM terminology. Tags are those lines starting with RPM-defined tags followed by a colon. In the example above the pre– amble first specifies the package name and summary (short descrip- tion), its version, epoch and release. Epoch is a tool for downgrad- ing a package version. Since higher epoch is always preferred we can update packages to their older versions (this is helpful when rolling back wrong changes). Release is a distribution or repository specific information saying how many times have been the package rebuilt for the given version. The %?dist macro then specifies a dis- tribution suffix (.fc21 for Fedora 21) and it is not really important for the package to work, but indicates for which operating system has been the package intended. In the example above we do not have any Requires or Provides tags as there are auto-generated for RubyGems packages in Fedora 21 and higher. The most important part comes next by defining Source, Requires and BuildRequires tags. Package sources are defined by Source0 and up to SourceN tags. They usually specifies an URL or a local path, but technically only the last filename in the source line is relevant. Requires and BuildRequires then specify run-time and build-time dependencies which can be other package names, files on the filesys- tem or virtual provides. Virtual provides are additional provides that can be specified by Provides tag within the specification file. It is a way how to identify a dependency without the need to depend on a specific package name (whose name can change in future) or in a situation where there are more packages that provide the same functionality. At last the preamble specifies a sub-package for documentation files. It is possible to specify as many sub-packages as needed which is helpful for breaking the package into more pieces that does not need to be installed at once. Common approach is to separate header files, documentation files or components that can be installed and used on their own. The preamble is then followed by %prep, %build, %install, %check, %files and %changelog sections: %prep gem unpack %{SOURCE0} 8 2. RPM-BASED DISTRIBUTIONS %setup -q -D -T -n %{gem_name}-%{version} gem spec %{SOURCE0} -l --ruby > %{gem_name}. gemspec %build gem build %{gem_name}.gemspec %gem_install %install mkdir -p %{buildroot}%{gem_dir} cp -a .%{gem_dir}/* %{buildroot}%{gem_dir} %check %files %dir %{gem_instdir} %{gem_libdir} %doc %{gem_instdir}/MIT-LICENSE %exclude %{gem_cache} %{gem_spec} %files doc %doc %{gem_docdir} %doc %{gem_instdir}/CHANGELOG.md %doc %{gem_instdir}/README.rdoc %doc %{gem_instdir}/examples %changelog * Thu May 22 2014 Josef Stribny The %prep, %build, %install, %check sections are sort of shell scripts with the RPM macros defined that should prepare the sources by unpacking and (optionally) patching them, build the software and install the built binaries and other files in their appropriate places on the filesystem, and run the test suite of the program. These scripts are called build-time scripts as they are run during the build of a pack- 9 2. RPM-BASED DISTRIBUTIONS age. Important thing is that preparation and build part is done in BUILD whereas installation is done in BUILDROOT (here represented by %{buildroot} macro). Other notable optional sections are %pre, %post, %preun and %postun. They are scriptlets that become part of the RPM file and are executed upon installation or uninstallation of the package. These are used to set up the correct environment on the target operating system by creating users and groups, or starting a system service. Finally each sub-package comes with a list of files that comes in the final RPM and changelog entries can be found at the end repre- senting changes for each rebuild of the package. These files are part of an RPM file as an archive, usually cpio. rpm2cpio is a program that can be used to unpack the cpio archive from the RPM file and the cpio binary can then be used to unpack the archive itself. If the package is installed by the rpm binary file, it is added to the RPM database on the system, which can be found at /var/lib/rpm. 2.2 rpmbuild The rpmbuild binary is used for building the RPM and SRPM pack- ages from the specification files and sources referenced in them[28]. To build packages using rpmbuild as unpriviledge user it is needed to set up a top directory with BUILD, RPMS, SOURCES, SPECS and SRPMS subdirectories and tell the rpmbuild where to look for this top directory. Once set up, rpmbuild will expect all package sources in SOURCES and specification files in SPECS. BUILD is the root di- rectory in which the build will be performed and RPMS and SRPMS are directories in which the final RPM packages and SRPM packages will be put. The basic usage is: rpmbuild [options] SPECFILE The build can be trigger either on a package specification file or compressed tarball containing such a specification file and its stages can be controlled with the following additional options: -ba builds binary and source packages (with all previous stages). 10 2. RPM-BASED DISTRIBUTIONS -bb builds binary packages (with all previous stages). -bp executes the %prep stage from the specification file. This involves mainly unpacking the sources and applying any patches. -bc executes %prep and %build stages from the specification file. Build stage is for building the software., e.g. ’make’ or ’gem build’. -bi executes %prep, %build and %install stages from the speci- fication file. Installation would mean "make install" for CMake or "gem install" for RubyGems package. -bl does a list check. It verifies that %files section from the spec- ification file is intact by checking that all mentioned files exist. -bs builds just the source package. The rpmbuild program can be used directly, but calls to rpmbuild are often wrapped by other tools like mock that makes the build safer by isolating the build environment. 2.3 createrepo The createrepo utility is a small Python program that is able to take a directory of built RPM packages and create an RPM package repos- itory out of them[8]. This is done by adding various RPM metadata for future dependency solving and installation as set of XML files. These files are generated by createrepo according to the RPM meta- data specification which specifies the following files: repomd.xml as the index file of the other metadata files. This can be used by clients to check whether the repository con- tains new changes since repomd.xml lists timestamps and check- sums for the other XML files. It also ensures the repository in- tegrity. primary.xml.[gz] for storing the primary metadata informa- tion such as name, epoch, version, release, architecture, file size, file location, description, summary, format, dependen- cies, provides, conflicts or obsoletes. 11 2. RPM-BASED DISTRIBUTIONS filelists.xml.[gz] for storing the complete file and directory list- ings for all of the packages within the RPM repository. Each package is identified by name, epoch, version, release, archi- tecture and package ID. other.xml.[gz] for storing additional information such as the changelog data from packages. groups.xml.[gz] this file is tentatively described. The intention is for a common package-groups specification as well. There is still some sections for this format that need to be fleshed out. The createrepo_c1 is a more performant alternative written in C with the same capabilities. 1. See https://fedorahosted.org/createrepo_c/. 12 3 Fedora Build Systems and Tooling 3.1 Yum Yum is a software package manager that handles installing, updat- ing, and removing RPM packages and their dependencies on the RPM- based operating systems[9]. Yum works on top of RPM reposito- ries that contain the RPM packages with repository metadata. Those repositories can then be added into /etc/yum.repos.d in .repo files. $ cat /etc/yum.repos.d/fedora.repo [fedora] name=Fedora $releasever - $basearch failovermethod=priority baseurl=http://download.fedoraproject.org/pub/ fedora/linux/$releases/releasever/Everything/ $basearch/os/ metalink=https://mirrors.fedoraproject.org/ metalink?repo=fedora-$releasever&arch=$basearch enabled=1 metadata_expire=7d gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora- $releasever-$basearch skip_if_unavailable=False The example above shows an excerpt from an fedora.repo file which references the official Fedora repositories. Apart from the im- portant baseurl and metalink options that specify the path to the RPM repository, it is convenient to set up gpgcheck and gpgkey options to allow Yum to verify the packages after their installation. When Yum runs an operation, it checks the known repositories, calculates the package dependencies and runs the requested opera- tion in the transaction. This way Yum supports rolling back to previ- ous stages of the package list. Yum is still the default package manager in Fedora. 13 3. FEDORA BUILD SYSTEMSAND TOOLING 3.2 DNF DNF is a Yum successor and a future replacement[10, 11]. DNF is originally a Yum fork aiming at using a SAT solver for dependency resolving with clearer API, cleaner codebase, improved performance and better memory footprint generally providing very similar inter- face. DNF is built using Hawkey, a new package management library built on top of the libsolv library. SAT solver1 provided by libsolv is NP-complete, therefore it finds the best possible solution in a very reasonable time. Moreover the libsolv library is used in OBS project (and openSUSE package manager) for years and basing DNF on it brings a unification of the dependency solving across the RPM-based Linux distributions used today. DNF is scheduled to replace Yum in Fedora 22[13]. 3.3 Mock Mock is a chroot2 build environment manager for building binary RPMs and a Fedora project aiming to be an unified way of build- ing packages both for the local packager’s builds and for the official builds by Koji build system[14]. It is a rather simple tool which is re- sponsible for setting up the chroot for the build, populating the build with a minimal set of necessary packages and running the build it- self. The key reason for using mock is to reproduce the package builds. Mock depends on the Yum software package manager for in- stalling the minimal set of packages and RPM dependencies for the SRPM to be built to populate the chroot jail before building the SRPM. As this is often repetitive and bandwith-demanding task, Mock uses various caching mechanisms to speed this up (e.g. it caches the down- loaded RPM packages using the yum_cache plugin). Since Mock should address builds for various systems and ar- chitectures (although the builds for a given architecture needs to be 1. Algorithm to solve the boolean satisfiability problem by determining if an in- terpretation that satisfies a given boolean formula exists[12]. 2. Also known as changed root or jail is a different root directory then /root that you can switch to (chroot). A program that can switch a root directory on Fedora is also called chroot. 14 3. FEDORA BUILD SYSTEMSAND TOOLING done on the underlying machine with the same architecture) it reads the configuration files from /etc/mock to know how to make the builds. A mock configuration file looks a lot like a Yum configuration file containing the list of RPM repositories with some options determin- ing the architectures of host and target system, distribution and setup command among others: $ cat /etc/mock/fedora-20-x86_64.cfg config_opts[’root’] = ’fedora-20-x86_64’ config_opts[’target_arch’] = ’x86_64’ config_opts[’legal_host_arches’] = (’x86_64’,) config_opts[’chroot_setup_cmd’] = ’install @buildsys-build’ config_opts[’dist’] = ’fc20’# only useful for -resultdir variable subst config_opts[’extra_chroot_dirs’] = [ ’/run/lock’, ] config_opts[’releasever’] = ’20’ config_opts[’yum.conf’] = """ [main] cachedir=/var/cache/yum debuglevel=1 reposdir=/dev/null logfile=/var/log/yum.log retries=20 obsoletes=1 gpgcheck=0 assumeyes=1 syslog_ident=mock syslog_device= # repos [fedora] name=fedora metalink=https://mirrors.fedoraproject.org/ metalink?repo=fedora-$releasever&arch=$basearch 15 3. FEDORA BUILD SYSTEMSAND TOOLING failovermethod=priority gpgkey=file:///etc/pki/mock/RPM-GPG-KEY-fedora-20- primary gpgcheck=1 These configuration files are then referenced by an -r option. The build for Fedora 20 on x86_64 would be started with: $ mock -r fedora-20-x86_64 SRPM There is also a site-wide configuration file 3 which can be used to specify site-wide options. The shipped version of this file has no active options, but does have a list of all of the configuration options, examples of how to set them, and their default values. The results of a successful SRPM build are all binary RPM pack- ages defined in the SRPM’s spec file. Mock writes 3 different logs on the filesystem that can be used for investigating failures: state.log is a global log of the whole process. It is what goes to the standard output during the Mock build. root.log are logs of what has been installed via Yum. build.log is the log for the build itself, the results of executing the RPM spec file sections from %prep to %check (or %clean) including the final file lists checks. Mock also comes with a mockchain subprogram that can execute a chain of SRPM builds in which it builds each SRPM package at a time and adds the resulting binary packages into a local Yum repos- itory so that they can be used later in the chain builds. This process is then repeated until all source packages are built. Using the chroot has an advantage in the speed of the setup, but has a few disadvantages as well: Mock cannot be used to test system’s services. That means it cannot run System V init scripts or systemd services during the build. Any tests coming with the SRPM that rely on these services cannot be run in the %check section. This applies also for manual testing inside the created chroot. 3. Located at /etc/mock/site-defaults.cfg. 16 3. FEDORA BUILD SYSTEMSAND TOOLING Mock has a limited security since chroot is still very much connected with the main system and so the root user can still cause some damage. Mock cannot be used to test different operating systems than those built with RPM packages and it has a limitation of us- ing the same kernel of the host operating system for all of the builds due to the nature of chroot. That means that build- ing a SRPM package under Fedora 20 with Linux Kernel 3.15.x for Fedora 21 with the Linux Kernel 3.16.x doesn’t ensure that the build would pass with the newer Kernel. 3.4 Koji Koji is the build server developed for the Fedora Project[15]. It is a Python-based open source build system deployed by Fedora In- frastructure for building the official Fedora distribution packages. Koji is comprised by a few loosely coupled components; Koji-Hub, Kojid, Koji-Web, Koji-Client and Kojira. 3.4.1 Koji-Hub Koji-Hub is the heart of Koji build system. It is an server running un- der mod_wsgi in Apache exposing an XML-RPC interface for com- munication. This interface is then used by all other Koji components which are the clients for Koji-Hub server. Clients are those that ini- tialize the communication and Koji-Hub is the component that or- chestrate the process requests. It has direct access to the Koji database and write access to the file system. 3.4.2 Kojid Kojid is the build daemon that is meant to run on the build machines (machines dedicated to the actual build process for the packages). It receives build requests (or other tasks such as creating install images) and handles them. For the build requests it starts the builds in a new build root and do the actual build using Mock in the chroot jail. It 17 3. FEDORA BUILD SYSTEMSAND TOOLING is a kojid responsibility to ask Koji-Hub for the new jobs and it does that via Koji-Hub XML-RPC interface. 3.4.3 Koji-Web Koji-Web is the web interface to Koji-Hub. It is a client providing a visual interface for some of the administration tasks. It is meant to expose details and results about past and current builds includ- ing their logs and binary RPM packages of the successful builds. For the successfully built packages Koji-Web can list additional meta- data including the list of files, requires and provides (but this is not true for scratch-builds4). Essentially it is a set of Python scripts using the Cheetah templating engine that can be run in mod_python on Apache HTTP server. The official instance of Koji for building Fedora packages is avail- able at http://koji.fedoraproject.org/koji/. Most of the interface is present only for reading although with sufficient privileges can Fe- dora developers perform some additional actions such as canceling a build, resubmitting a failed task, and set up a notification. Signed users with administration privileges can also manage tags, targets and build hosts. 3.4.4 Koji-Client The Koji client is a command line interface for Koji backend that pro- vides a remote interface for Koji-Hub. Packagers can use the Koji client to initialize builds or query the data from the Koji server. In reality though, most of the communication requesting new builds comes from fedpkg tool. Fedora packagers should run the fedora-packager-setup tool if they wish to use Koji to build Fedora packages to do the initial setup for Koji authentication as this authentication requires the following three certificates to be set up: /.fedora.cert˜ is specific to the Fedora developer and can be generated by the fedora-cert tool. This is usually generated 4. Scratch-builds are just trial builds to see if the SRPM would build. The results are usually not saved for long. 18 3. FEDORA BUILD SYSTEMSAND TOOLING when someone becomes the Fedora maintainer and expires in time. /.fedora-upload-ca.cert˜ is the certificate to sign the user keys. /.fedora-server-ca.cert˜ is the certificate to sign the build sys- tem’s server keys. 3.4.5 Kojira Kojira is a daemon that keeps the build root repodata updated. It is responsible for removing redundant build roots and cleaning up after a build request is completed. 3.5 fedpkg The fedpkg tool is an open source program based on rpgk project[16, 17]. It provides Fedora specific functionality to rpkg on top of other tools like dist-git5, koji and rpmbuild. The most common packager processes are handled by fedpkg al- though they can be achieved by lower-level tooling. The purpose of fedpkg is to provide an easy to use interface for all packager’s actions such as checking the package sources from the Fedora dist-git, com- miting new changes, uploading new sources, requesting Koji build or scratch-build, and Koji update. 3.6 Bodhi Bodhi is a modular web-based system and command-line interface that handles the process of package updates for Fedora[18, 19]. The types of updates that can be submitted via Bodhi are: bugfix for bugfix update enhancement for general updates 5. dist-git refereces the distribution repositories based on git source control sys- tem. 19 3. FEDORA BUILD SYSTEMSAND TOOLING security for security updates (CVEs6) Each package update within Bodhi has one of the following states pending: package has not been pushed to the updates-testing or updates-stable repositories. testing: package is in the updates-testing repository for Fedora users to test it. stable: package has been added to the main updates reposi- tory. obsolete: package has been obsoleted by some other update. The bodhi-client command line interface can be used to submit new updates instead of web interface: $ bodhi --new --release FEDORA_RELEASE --type bugfix PACKAGE The fedpkg tool also encapsulates the bodhi call. The following can be invoked from the checked-out package repository: $ fedpkg update 3.7 Copr Copr is a latest addition to the Fedora build system ecosystem[20, 21]. Unlike for Koji, Copr is designed to be a lightweight build sys- tem that allows third-party contributors to easily create and publish additional packages outside of official repositories. These packages do not need to go through the package review process so Copr can provide also packages that cannot make to the official repositories 6. Common Vulnerabilities and Exposures (CVE) is a system that references publicly known security vulnerabilities maintained by MITRE Corporation. Such known vulnerability or exposure is assigned a CVE number. 20 3. FEDORA BUILD SYSTEMSAND TOOLING for some reason. As with Koji, packagers can build new packages in Copr against various Fedora and EPEL7 releases. Copr currently does not provide any source control, only a straight- forward way of building a set of source RPMs from remote locations. An RPM repository with built binaries and sources is then provided for every project. 7. Extra Packages for Enterprise Linux (EPEL) are additinal repositories of high quality not-conflicting packages for Enterprise Linux usually installed on Red Hat Enterprise Linux or CentOS and a special interest group within Fedora Project that maintains these repositories. 21 4 Open Build Service Open Build Service (originally openSUSE Build Service) is a gen- eral build system capable of building packages for various Linux distributions, for different package formats and architectures[22]. It is formed by various components. Most notable server-side compo- nents include repository server, scheduler, worker, dispatcher, pub- lisher and source server. Builds themselves are handled by the build component1 and signing of the built packages is done by the op- tional signer component2. OBS includes both web-based interface (and API) and a command-line tool called osc. 4.1 OBS Server Server part of OBS is written in Perl and includes various Perl mod- ules and scripts meant to be run as system daemons listening for RPC calls over network (eventually run as System V services). All compo- nents are somehow connected via the main source server component (which is always just one for one OBS server setup), one or more repository servers, followed up by schedulers deployed for each ar- chitecture users want to build against and as many workers as users need for parallel building. These components can be run in a single host system, but are otherwise made to run on different servers3. All of the them work on a file system (file system functions as a database server) and are writing the content to the various directories depend- ing on their job description. The communication between compo- nents always happens either by RPC call or a change on the file sys- tem (the /events directory holds the events that are picked up by respective components). 1. Also called obs-build, OBS Build script or just Build. 2. Also called obs-signd. 3. See https://github.com/openSUSE/open-build-service/blob/master/src/ backend/DESIGN. 22 4. OPEN BUILD SERVICE 4.1.1 Repository Server The repository server starts the build life-cycle by creating build tasks at /events/arch if something needs to be build. The tasks are there to be picked up by the schedulers. Then the builds are scheduled by a scheduler and dispatched by a dispacher to workers. Workers in- form the repository server about the job status and when the job is finished the repository server passes this information to the signer by writing to /events/signer. After the optional signing the state is changed to finished by writ- ing to /events/arch/.finished and this is the signal for the scheduler to move the result into /build. 4.1.2 Scheduler Scheduling in OBS is checking if there is any need for a new build. This happens with a change in source files which means that OBS schedules new builds automatically. Scheduler picks up what to build in /events/arch for the scheduler’s architecture, creates a project repository in /build, collect binaries and schedule the jobs (builds) for the project writing to /jobs/arch/. When it is informed from the repository server that the job has finished, schedules other jobs accordingly for the given project and when the scheduler detects that an entire repository has finished building, it prepares project’s repository4 and writes to /events/pub- lish for the builds to be published. Scheduler’s process is the only one writing into /build directory. 4.1.3 Dispatcher Dispatching means assigning a build job (a scheduled build) to a concrete worker. Dispacher is triggered by /events/dispatch for jobs to be dispatched from the list of jobs at /jobs and assigns the jobs to workers from /workers. Then the dispacher asks those workers via RPC to do the builds and writes the job statuses to /jobs. 4. In /srv/obs/build/$PROJECT/$REPO/$ARCH/:repo. 23 4. OPEN BUILD SERVICE 4.1.4 Worker Worker periodically registers itself at the repository server via RPC for a new job (build) and gets the job pushed by the publisher as a response if it is currently in the idle state, then asks for the binaries and/or preinstall image the repository server and on sources and SSL certificate the source server (again via RPC). Once the worker gets all needed sources, it verifies them and does a build in /build/ $PROJECT/$REPO/$ARCH sending state to the repository server (build started, build finished). The build itself is handled by the OBS Build script. When the build finishes the worker fetches the old package if any and does a diff of changes that are together with the results trans- ferred to the source server which works as a synchronization point. Scheduler can then update :repo inside /build which is picked up by the publisher who finally updates /repos directory with final repos- itories. 4.1.5 Publisher Publisher looks into /events/publish and does the publishing by hard linking files (binaries) from /build to /repos and updates in- dexes in /db/published. It is the only process that writes into /repos directory. 4.1.6 Source Server Source server is the only single component within OBS and there- fore acts as a synchronization point. It hosts the sources, binaries and configuration in /projects and /sources, and it keeps indexes in /d- b/source and /db/published. When asked by a worker via RPC, it returns the path to the source files. 4.2 OBS API The OBS frontend is a Ruby on Rails based application that provides the web user interface and API including the access control logic for 24 4. OPEN BUILD SERVICE osc5. Unlike for the Koji web interface, the OBS web interface allows users to manage almost anything regarding the package life-cycle from projects creation, uploads of package sources, branching and repositories to users access control. OBS API also takes care about user management and access rights for OBS backend. It stores all necessary information in a MySQL database. OBS differentiate from Koji in the package update process. Koji builds the updated packages the same way it does build the stable packages, but update submission and management happens through Bodhi (be it the command line client or web interface) and builds are always marked by different tags (e.g. f21-updates-testing for the build submitted as an update for F21 and not yet pushed to stable). On the other hand the product’s builds and update process in OBS is designed around creating different kind of projects[22]; GA Project is an initial release which gets frozen when re- leased. Release Project provides a release repository for the product. Nothing is built in this project. The sources and binaries of a product are only copied there on a product’s release. Update Project is essentially a Release Project providing up- dates for a product generated in a GA Project. Maintenance Project is a project without any sources and bina- ries. It is used for managing incidents (as project’s sub-projects), typically security updates. The incident sub-project then de- scribes the concrete issue and its solution. The patchinfo file caring the update information needs to be provided together with fixes and release request is created. After review period and acceptance of the fix the name gets extended by the inci- dent number and put along with the original package to the pack- age container leaving the main package to be linked to this incident package. 5. The osc tool is an OBS command-line based client for managing the source code repositories from remote OBS servers for local accounts. See Chapter 4.5 for more. 25 4. OPEN BUILD SERVICE An example of an OBS project can be "the next openSUSE distri- bution" called openSUSE:Factory6 (something as Fedora Rawhide) and an example of an maintenance project "The openSUSE Mainte- nance project" called openSUSE:Maintenance7 which contains inci- dents for all released openSUSE distributions. 6. See https://build.opensuse.org/project/show/openSUSE:Factory. 7. See https://build.opensuse.org/project/show/openSUSE:Maintenance. 26 4. OPEN BUILD SERVICE 4.3 obs-build OBS Build script is a Perl script used to run the build inside work- ers or locally. It is called either from OBS worker for the OBS server builds or from the osc command-line client for local builds on the pack- ager’s machine. Build script can also be called directly just as mock. By default Build sets up a chroot directory in /var/tmp/build-root similar to mock’s and uses rpmbuild for RPM builds as well. How- ever, it can be used to build packages for many other Linux distribu- tions as it supports the following binary formats: RPM (binary format of Fedora, openSUSE and derivates) DEB (binary format of Debian, Ubuntu and derivates) ARCH (binary format of Arch Linux) The build process can be run in a variety of targets to secure and isolate the build process from classic virtualization to cloud environ- ments. Apart from building in a chroot, Build supports: any custom emulator (shell script) KVM (classic virtualization) XEN (classic virtualization) LXC (Linux containers) OpenStack (cloud build) EC2 (cloud build) z/VM Hypervisor And it supports the following recipes8: spec9 8. A recipe contains the steps saying how to do the build. 9. Specification file from source RPM. 27 4. OPEN BUILD SERVICE dsc10 arch kiwi11 livebuild mock preinstallimage Adding the support for Mock to OBS has been discussed with OBS maintainers during the writing of this thesis and has been after- wards implemented by them. Mock support in Build is internally im- plemented as another build engine (currently there is either build or mock engine available) and via the build-recipe-mock file containing a recipe on how to build a mock configuration file on-the-fly before the build. Build script is developed by the openSUSE Project and used to cre- ate openSUSE Linux but can also be employed to build a variety of other Linux distributions. 4.4 obs-signd Signing of built packages is handled by optional obs-signd compo- nent. The setup for signing RPM packages within Fedora project is to al- ways copy all data to a signing server which can then, by calling gpg from GNUPG project, create the signatures for them and send them back for the signatures to be passed to rpmsign. The rpmsign binary can then add the generated signatures to the final RPM packages. In this standard setup big ISO or Docker images can became a bot- tleneck since they can be several gigabytes big in size. Therefore the OBS project addressed the signing process differently. Instead 10. Textual metadata file from .deb source package. 11. KIWI is an application for making a wide variety of image sets for Linux sup- ported hardware platforms as well as virtualisation systems including QEMU, Xen and VMware. 28 4. OPEN BUILD SERVICE of sending all data to the signing server, only the digest (created by sha256sum) is sent over. Since gpg is also used in this setup, the initial computation of the file hash (digest) done by gpg has to be skipped. If it would not gpg would in fact create a signature for the hash of the file instead of the file itself. This is done by a spe- cial option called ’–files-are-digest’ that was added by the OBS team to gpg (but only to OBS gpg package). This approach has two problems. It creates a security risk since it is bypassing the computation of the digest inside of gpg and because it is necessary to ensure that the network is safe and not reacheable from the outside. Also it requires to patch gpg and carry such patches. 4.5 osc The osc tool is an OBS command-line based client. It is mainly used for managing the source code repositories from remote OBS servers for local accounts. Each source code repository is actually a checked- out git repository of corresponding OBS source server repository for which osc provides one common interface for managing the reposi- tory and its sources, quering OBS server for build results and editing the metadata[23]. Apart from working with the OBS repositories, osc can be used to trigger the local builds of packages. This is done via the OBS Build script which is the same script called on OBS workers to do the builds. Various options can be passed to osc to define the local builds. When first used, osc asks on user’s OBS credentials and creates the .oscrc file in the user home directory. This file is then used for storing osc settings for local builds as well as credentials for each OBS server. OBS Build script is not a mandatory part of osc, it is only a soft de- pendency and it is required only for local builds. A packager needs to specify at least the URL for OBS API and their user credentials in .oscrc to connect to an OBS server: [general] apiurl = https://api.opensuse.org [https://api.opensuse.org] user = username 29 4. OPEN BUILD SERVICE pass = password Afterwards https://api.opensuse.org will be queried for all OBS requests from osc. If other OBS servers are defined, it is possible to call any OBS command specifically for that server using the -A [apiurl] switch. Since its 0.122 version, osc gained support for generic Python’s keyring library and it is now able to store passwords in those keyrings instead of .oscrc as plain text. In terms of the Fedora infrastructure, osc is to OBS what fedpkg is to Koji. Where fedpkg can interact with the Koji builds and can call rpmbuild or mock to do the local builds, osc interacts with OBS servers and calls OBS Build script to do the local builds. osc is written in Python and provides also a Python module to of- fer a native Python interface to osc capabilities for other Python pro- grams. 30 5 Analyses and Design 5.1 Fedora Packaging OBS server, API, Build script and sign daemon needs to be prop- erly packaged for Fedora. Since they have various Perl and Ruby dependencies, these need to be packaged and submitted for inclu- sion within Fedora official repositories first. As Fedora 21 is planned to be released around end of 2014, efforts of this thesis should be fo- cused on Fedora rawhide and 21 for OBS dependencies and Fedora 21 for OBS packages. A separate Copr repository should be created to accommodate dependencies that cannot be accepted for base Fe- dora at the moment and for OBS itself until all the dependencies get into official repositories. Such packaged dependencies need to follow Fedora guidelines to match the Fedora official packages quality in or- der to be approved through the package review process. osc tool is already packaged and maintained in the official Fedora repositories and could be updated if necessary. 5.1.1 Dependencies As the backend part of OBS is written in Perl, it depends on vari- ous Perl modules. Perl modules required by OBS and missing from base Fedora repositories are BSSolv and Net::RabbitMQ. BSSolv is a Perl wrapper around libsolv library for dependency solving and Net::RabbitMQ provides interaction with RabbitMQ1 over AMQP2 using librabbitmq. BSSolv and Net::RabbitMQ are standard Perl mod- ules found on CPAN3 and Fedora Perl Packaging Guidelines applies to them. As Net::RabbitMQ 0.2.7 bundles an incompatible version of librabbitmq with the version already packaged for Fedora, it cannot be currently included in base Fedora repositories (bundling libraries is against Fedora policies4) and only a Copr version of the library is packaged. Unbundling of the library is not trivial due to the version 1. Open source distributed message broker similar to Apache ActiveMQ 2. Advanced Message Queuing Protocol 3. Comprehensive Perl Archive Network 4. See https://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries. 31 5. ANALYSES AND DESIGN incompatibility and a lot of the C extension code would need to be changed. The table 5.1 shows the necessary dependencies to run the back- end part on Fedora 21 and their Red Hat Bugzilla ID associated with their package review process. Table 5.1: Backend dependencies dependency name bugzilla ID BSSolv 1065490 Net::RabbitMQ Copr OBS Signer 1128384 OBS Build Copr OBS API part is written using Ruby on Rails web framework and the dependencies are therefore Ruby gems. Since the latest stable ver- sion of OBS depends on older Rails version, a newer git revision that is built against Rails 4.1 (present in Fedora) needs to be selected. There are various other application dependencies that can be de- vided into four categories. The first one are runtime dependencies that are necessary for running the API Ruby code. The other one are assets providing static JavaScript or CSS stylesheets that are also re- quired to run the Rails application as expected. Last two groups are gems in the test group that are necessary to run the test suite and in the development group needed only for the application develop- ment. These groups are ignored in this packaging effort as they are not necessary for runtime and as it is not possible to easily execute OBS test suite during the build in mock due to the limitations de- scribed in chapter 3.3. Two of the necessary API dependencies that are not found in Fe- dora and also cannot be currently submitted or added to base Fe- dora are yajl-ruby and rails_tokeninput libraries. yajl-ruby has been already submitted for inclusion before, but has a vendoring issue5. yajl-ruby should be possible to replace by ffi-yajl gem which was cre- ated to address this. ffi-yajl is not yet in Fedora, but already submit- ted for inclusion by Julian C. Dunn6. And rails_tokeninput does not 5. See https://bugzilla.redhat.com/show_bug.cgi?id=823351. 6. See https://bugzilla.redhat.com/show_bug.cgi?id=1114146. 32 5. ANALYSES AND DESIGN state under what licensing conditions can be redistributed. Upstream ticket was opened7 so it is likely that licensing will be properly stated in future. The table 5.2 shows the list of dependencies needed to run OBS API on Fedora 21 and their Red Hat Bugzilla ID associated with their package review process. Table 5.2: API dependencies dependency name dependency of bugzilla ID cocoon runtime 1121084 clockwork runtime 1121082 pundit runtime 1121085 acts_as_list runtime 1143926 kaminari runtime 1143924 yajl-ruby runtime 823351 font-awesome-rails assets 1122941 jquery-ui-rails assets 1122939 jquery-datatables-rails assets 1122944 sprite-factory assets 1121080 rails_tokeninput assets Copr codemirror-rails assets 1169070 The jstribny/obs-for-fedora8 Copr project has been created and contains the builds that cannot be included in base Fedora at the mo- ment of writing. The RPM specification file for yajl-ruby has been taken from the review submission and it is the only one not created from scratch for the purpose of this thesis. 5.1.2 OBS Web applications such as OBS API should be ideally packaged into /usr/share and lib files such as OBS backend into directory speci- fied by %_libdir (which is either /usr/lib or /usr/lib64), but since OBS cannot yet be included in Fedora (due to the missing dependen- cies) and since /usr/lib is hard-coded everywhere in OBS sources, 7. See https://github.com/logical42/rails_tokeninput/issues/3. 8. Available at https://copr.fedoraproject.org/coprs/jstribny/obs-for-fedora/. 33 5. ANALYSES AND DESIGN the packaging currently follows the upstream packaging efforts and directory layout. 34 5. ANALYSES AND DESIGN 5.2 Fedora System Integration 5.2.1 systemd Starting Fedora 19, Fedora adopted systemd, a new system and ser- vice manager for Linux-based operating systems, as a default init system 9. systemd provides some advanced features as aggressive parallelization, usage of socket and D-Bus activation for starting ser- vices, on-demand starts of daemons, and many more. It is also possi- ble to use systemd as a drop-in replacement for System V init[24, 25]. Although openSUSE does have a stable release with systemd as a default (since openSUSE 12), OBS provides only System V init scripts for managing the OBS system services. As the scripts are then run by systemd, they shall therefore be replaced by proper systemd service files. Rewrite of those service files can be pretty straightforward, al- though there is one limitation that comes with the current design of OBS. System V init service files gets status for OBS components by calling an external scripts that read the actual status from the file system. And this cannot be done with systemd. In systemd it is not possible to redefine the status of a given service file. Status is simply a state of the unit file itself. Therefore in proposed packaging that uses systemd, systemd unit files calls out bin files that are created from the original OBS rc scripts by removing any System V specifics. This way they behave like nor- mal bin files and can be called from systemd unit files. To see the orig- inal status of the service (in terms of OBS component and not in terms of the systemd service) it is necessary to call these binary files di- rectly. So instead of: # To see the status of OBS repository server with System V service obsrepserver status there are two kinds of status to see, one for OBS component and one for systemd service: # To see the status of OBS repository server 9. A program that starts all other processes on Unix-like systems 35 5. ANALYSES AND DESIGN obsrepserver status # To see the status of OBS repository server service systemctl status obsrepserver The achieved state is functionally the same as letting systemd to run rc scripts directly, but it is one step further in adopting na- tive systemd services. It gives also the administrator the possibility to run these components as simple binary files: obssrcserver start|stop|status obsrepserver start|stop|status obsscheduler start|stop|status obsdispatcher start|stop|status obspublisher start|stop|status obswarden start|stop|status obsworker start|stop|status This could however be changed in the way that systemd unit files would call the OBS components directly. The code currently present in the original rc scripts and run just before calling OBS components files10 would most likely needed to be incorporated in those compo- nents directly. 5.2.2 SELinux SELinux (Security-Enhanced Linux) is a security enhancement to Linux kernel allowing for more access control beyond traditional access rights and Access Control List including United States Department of Defense–style mandatory access controls. It is available as a kernel module 11 and set of user-space tools[26]. Fedora includes SELinux already since Fedora Core 2 version mak- ing it one of the first adapters. SELinux is therefore a standard in Fedora 21 and its targeted mode is enabled by default. Thus Fedora ships many sample security policies in the base distribution designed to meet general-purpose security goals that can be extended by user- defined modules and other policies if necessary. 10. Files like bs_repserver. 11. The module has been merged into the Linux kernel in the 2.6 series. 36 5. ANALYSES AND DESIGN For OBS to work as expected on system running with enabled SELinux in targeted mode, certain security policies needs to be ex- plicitly enabled. These policies can be enabled manually by the sem- anage tool found on Fedora in policycoreutils-python package. What to properly label can be determined by examining the SELinux logs for denied operations found at /var/log/audit/audit.log. The au- dit2allow utility tool can generate a custom policy module to allow such operations based on the logged actions. Running OBS using mod_passanger and Apache HTTP server (the chosen deployment method for this migration) help to deter- mine denied actions and to correctly set up the SELinux policies. Regarding OBS on Fedora 21, the Apache HTTP server cannot ac- cess certificate files for reading and OBS API logs for writing. Apache process needs for files to be labeled as httpd_sys_content_t for read- ing and httpd_sys_rw_content_t for writing. Using semanage fcon- text this can be allowed with: semanage fcontext -a -t httpd_sys_content_t ’/srv/ obs/certs(/.*)’ semanage fcontext -a -t httpd_sys_rw_content_t ’/ srv/www/obs/api/log(/.*)’ This writes the rules to SELinux policy files, but do not change the label for files automatically. restorecon is the utility that can be used to relabel any files according to latest policy rules: restorecon -R ’/srv/obs/certs’ restorecon -R ’/srv/www/obs/api/log’ SELinux also prevents Apache process to access additional TCP ports used by OBS. This can be fixed by allowing these ports to be used by Apache using semanage port: semanage port -a -t http_port_t -p tcp 82 semanage port -a -t http_port_t -p tcp 5352 That would be written in policy files as portcon function calls: portcon tcp 8 system_u:object_r:http_port_t:s0 portcon tcp 82 system_u:object_r:http_port_t:s0 portcon tcp 5352 system_u:object_r:http_port_t:s0 37 5. ANALYSES AND DESIGN And finally mod_passenger needs also access to files used by Apache and to ports used by memcached. This policy can be auto- matically generated after hitting the issue using the following call to audit2allow: /var/log/audit/audit.log | audit2allow -mR local > local.te Once the .te file is generated, the SELinux module can be built and loaded: $ checkmodule -M -m -o local.mod local.te $ semodule_package -o local.pp -m local.mod $ semodule -i local.pp It is also possible to directly generate a final module using au- dit2allow: $ cat /var/log/audit/audit.log | audit2allow -MR local Generating type enforcment file: local.te Compiling policy: checkmodule -M -m -o local.mod local.te Building package: semodule_package -o local.pp -m local.mod ... $ semodule -i local.pp Since files labeling can be also included in the loadable policy module, it is convenient to ship OBS packages with only one policy module addressing all issues. This obspol module would be gener- ated from obspol.fc and obspol.te files which would contain the in- formation gathered during the test run: $ cat obspol.fc # Apache process needs to read certificates and writes to logs /srv/obs/certs/ system_u:object_r: httpd_sys_content_t:s0 /srv/obs/certs(/.*) system_u:object_r: httpd_sys_content_t:s0 /srv/www/obs/api/log system_u:object_r: httpd_sys_rw_content_t:s0 38 5. ANALYSES AND DESIGN /srv/www/obs/api/log(/.*) system_u:object_r: httpd_sys_rw_content_t:s0 $ cat obspol.te module obspol 1.0; require { type passenger_t; type memcache_port_t; type unreserved_port_t; type httpd_sys_content_t; type httpd_sys_rw_content_t; type logrotate_t; class tcp_socket name_connect; class dir { getattr search }; class dir read; class file { getattr open ioctl append }; } # ======passenger_t ======allow passenger_t httpd_sys_rw_content_t:dir { getattr search }; allow passenger_t httpd_sys_rw_content_t:file { getattr open ioctl append }; allow passenger_t memcache_port_t:tcp_socket name_connect; # !!!! This avc can be allowed using the boolean ’nis_enabled’ allow passenger_t unreserved_port_t:tcp_socket name_connect; # ======logrotate_t ======allow logrotate_t httpd_sys_content_t:dir read; The whole policy module together with the calls to allow the us- age of additional ports created as a part of this thesis are packaged as another OBS sub-package making it easier to install the SELinux support on systems with SELinux enabled. 39 5. ANALYSES AND DESIGN 5.3 Fedora Tooling Integration As mock is a key part in the process of making Fedora RPM pack- ages, it is necessary that mock works well with OBS. Since OBS Build script gained some sort of support for mock in form of an engine (in OBS Build script terminology), this should not be an issue anymore, but possibly more testing of this feature is required. 40 6 Conclusion This thesis discussed the basics of building RPM-based Linux dis- tribution, the Fedora tooling built around RPM including the Yum and DNF package managers, securing RPM package builds using mock, Koji – build system that is currently used to build official Fe- dora packages – and Copr – the new build system for building pack- ages out of official repositories. Afterwards the OBS build system was described to find out whether it could be a good fit as a Koji replacement. OBS is indeed superior feature-wise and already pro- vides basic mock support too, which is a condition for Fedora builds. As for the OBS migration to Fedora, all of the 14 necessary depen- dencies missing from the base Fedora were packaged for this thesis. They were submitted for package review process and built for Fe- dora rawhide and 21 upon review approval. Some of them cannot be included in Fedora at the moment and were therefore packaged only as a part of the OBS Copr repository. That is also the reason why OBS itself cannot be yet submitted for an inclusion to Fedora. Nevertheless the SRPM for the project was created based on the upstream spec file including a custom SELinux policy module so anyone can install OBS packages on Fedora 21. The packaging fol- lows upstream file layout as that is currently hard-coded in OBS source files, but some of that would need to change for a proper Fe- dora packaging. Since OBS packages for Fedora are now available OBS can be further tested and evaluated for the use as a build sys- tem for the Fedora Project. The optional obs-signd component was packaged too and included in base Fedora. Apart from its use within OBS project, obs-signd it is already used successfully in the develop- ment version of Copr project as well. Given the presented facts OBS is not yet ready to replace Koji. Apart from the problems with dependencies that can be likely ad- dressed by switching them to alternative projects and/or spending time on changes that would make them appropriate for base Fedora, the file layout should be made adjustable and all processes around packaging and package updates would have to adapt. 41 Bibliography [1] LWN.net [online]. [cit. 2014-03-14]. Fedora mining for COPR. Available from WWW: 42 6. CONCLUSION [11] FedoraProject [online]. [cit. 2014-10-19]. Features/DNF. Available from WWW: [12] Satisfiability and Maximum Satisfiability [on- line]. [cit. 2014-11-30]. Available from WWW: [13] FedoraProject [online]. [cit. 2014-03-14]. Changes/Re- placeYumWithDNF). Available from WWW: [14] Mock [online]. [cit. 2014-10-01]. Available from WWW: [15] Koji [online]. [cit. 2014-10-01]. Available from WWW: [16] fedpkg [online]. [cit. 2014-10-01]. Available from WWW: [17] FedoraProject [online]. [cit. 2014-10-19]. Pack- age maintenance guide. Available from WWW: [18] bodhi [online]. [cit. 2014-10-01]. Available from WWW: [19] FedoraProject [online]. [cit. 2014-10-19]. Bodhi. Available from WWW: [20] copr [online]. [cit. 2014-10-01]. Available from WWW: [21] FedoraProject [online]. [cit. 2014-10-19]. Category:Copr. Available from WWW: [22] Open Build Service [online]. [cit. 2014-09-14]. Open Build Service Reference Guide. Available from WWW: 43 6. CONCLUSION [23] Open Build Service [online]. [cit. 2014-09-14]. Open Build Service Best Practice Guide. Available from WWW: [24] systemd System and Service Manager [on- line]. [cit. 2014-10-01]. Available from WWW: [25] Pid Eins [online]. [cit. 2014-10-19]. systemd for Administrators, Part III. Available from WWW: [26] SELinux Project Wiki [online]. [cit. 2014-10-01]. Available from WWW: [27] SCHRÖDER, Michael; SCHRÖTER Adrian. Open Build Service: Facts, Features & Future [on- line]. 2012 [cit. 2014-03-14]. Available from WWW: [28] BAILEY, Edward. Maximum RPM. Indianapolis: Sams, 1997. ISBN 978-0672311055. 44 A Installation and Setup This section shows how to easily install the final packages available through Copr project and run OBS locally on fresh Fedora 21 install. A.1 Installation DNF has a Copr plugin that can be used directly for adding Copr repositories: $ sudo dnf install dnf-plugins-core -y $ sudo dnf copr enable jstribny/obs-for-fedora $ sudo dnf install obs* -y A.2 Setup OBS project ships with the obsapisetup file to set up the database used by OBS API. Before running it, it is necessary to swich OBS_API– _AUTOSETUP option to ’yes’ in the OBS server configuration file located at /etc/sysconfig/obs-server. Afterwards MariaDB can be started as a service and obsapisetup can be run: $ sudo systemctl start mariadb $ sudo obsapisetup start # For the generated cert files $ restorecon -R /srv/obs/certs Once the database is successfully set up and running, the various OBS backend services can be started: $ sudo systemctl start obssrcserver $ sudo systemctl start obsrepserver $ sudo systemctl start obsscheduler $ sudo systemctl start obsdispatcher $ sudo systemctl start obspublisher # Run for monitoring $ sudo systemctl start obswarden # Run worker locally 45 A.INSTALLATION AND SETUP $ sudo obsworker start # Finally start Apache HTTP server $ sudo systemctl start httpd If all is well, OBS web interface can be found at https://localhost/, the admin account can be accessed with username ’Admin’ and pass- word ’opensuse’ and https://localhost/monitor should show run- ning OBS services and one worker in the idle state. 46 B Attachements Attached compressed file contains all source RPM files that were cre- ated for the purpose of this thesis. rpm2cpio and cpio programs can be used to unpack the RPM archives of these files and therefore to see the RPM specification files. # To unpack the RPM archive of a given SRPM rpm2cpio SRPM | cpio -i 47
Open Build Service Migration to Fedora
