Effective Packaging

Eﬀective packaging

Petr Uzel L3/Maintenance [email protected] ptr uzl @ IRC About this talk

collection of tips and tricks that make (some) package maintenance tasks more eﬃcient no rocket science, no cool technologies based on experience there are deﬁnitely better ways

given a bug (e.g. /path/to/bin crashes), sometimes it is not obvious which package should be ﬁxed bugzilla does not track package info rpm -qf /path/to/file gives name of the RPM we need source package, not RPM

$ rpm -qf /lib64/libblkid.so.1 libblkid1-2.18-56.1.x86_64

$ osc search -s --package blkid No matches found for ’blkid’ in packages

$ rpm -q --queryformat ’%{SOURCERPM}\n’ \ libblkid1-2.18-56.1.x86_64 util-linux-2.18-56.1.src.rpm

Everything combined together (ﬁle → source package)

$ rpm -q --queryformat ’%{SOURCERPM}\n’ \ $(rpm -qf /lib64/libblkid.so.1) | \ sed -e ’s/$^.*$-.*-.*src.rpm/\1/’ util-linux

Where did I install the package from?

$ rpm -q --queryformat ’%{DISTURL}\n’ util-linux obs://build.opensuse.org/home:puzel:staging/openSUSE_11.3/6a[CUT]

With branching: slow: osc branch openSUSE:Factory foo; osc co home:*:branches:PRJ faster: osc branch --checkout openSUSE:Factory foo fastest: osc getpac foo or osc bco foo hyperspeed: osc getpac + osc bash completion

Without branching: function show_devel { osc meta pkg openSUSE:Factory ${1} | fgrep devel }

go directly to devel project

osc build in package working copy usually does the right thing more control: osc build [OPTS] REPOSITORY ARCH SPECFILE REPOSITORY, ARCH and SPECFILE detected automatically if not speciﬁed list REPO-ARCH combinations: osc repos [PRJ]

build debuginfo and debugsource packages: osc build --debuginfo/-d skip post build checks: osc build --nochecks delete old build root before build: osc build --clean use alternative project for build: osc --alternative-project=openSUSE:Factory use ccache to speed up build: osc build --ccache

foo depends on bar, which is not in buildservice yet but we have bar.rpm osc build -p/--prefer-pkgs DIR: prefer packages from DIR when installing buildroot

$ cd foo $ osc build -p /path/to_dir_with_bar.rpm foo.spec

osc build -x/--extra-pkgs PKG: add this package to buildroot

by default, buildroot is /var/tmp/build-root OSC BUILD ROOT overrides the default

$ export OSC_BUILD_ROOT=/var/tmp/build-root-temp $ osc build

use when parallelly building multiple packages

buildroot on tmpfs 5% speedup of parted build

$ grep build-root /etc/fstab tmpfs /var/tmp/build-root tmpfs size=5G 0 0

by default, osc build asks for root password; to avoid it set su-wrapper = sudo in ~/.oscrc add ‘‘yourlogin ALL = (root) NOPASSWD: /usr/bin/build’’ to sudoers ﬁle

build log is stored in $OSC BUILD ROOT/.build.log slow: less /var/tmp/build-root/.build.log faster: osc lbl/localbuildlog respects $OSC BUILD ROOT from working copy osc rbl/remotebuildlog PRJ PKG REPO ARCH - get remote build log

1 get package source 2 download tarball 3 verify tarball 4 repack tarball 5 check diﬀerences 6 adjust specﬁle 7 refresh patches 8 run test build 9 test the package 10 write changelog entry 11 commit

many projects publish tarballs on FTP sites ncftp is a command-line FTP client with bookmarking support useful to remember locations where upstream publishes tarballs supported by bash-completion

$ ncftp ut -> ncftp util-linux ncftp... ls ncftp... get fil -> filename

ncftpbookmarks: bookmark editor

packaging policy requires the tarball to be bzip2-ed (or xz-ed) sometimes upstream provides only .tar.gz

$ bznew parted-2.3.tar.gz $ ls *.bz2 parted-2.3.tar.bz2

Compare outputs from configure --help $ git clone git://gitorious.org/opensuse/pack-tools.git $ diffconf gnupg-2.0.10.tar.bz2 gnupg-2.0.16.tar.bz2

usually just open specﬁle and increase the version number small helper:

$ alias vs=’vim -p *.spec’

in vim (normal mode), press Ctrl+A/Ctrl+X to increase/decrease numbers

Quilt allows you to easily manage large numbers of patches by keeping track of the changes each patch makes. Patches can be applied, un-applied, refreshed, and more.

initialize a source tree from RPM spec ﬁle 1 unpack tarball 2 apply patches 3 un-apply patches (but remembers the series) IOW: %prep phase

$ quilt setup -v foo.spec $ cd foo-1.0.0

shortcut:

$ alias qs=’quilt setup -v *.spec’

now we are in the directory created by quilt setup (unpacked tarball) none of the patches is applied (but quilt “knows” about them) quilt push applies one patch (-a: all patches) quilt pop un-applies one patch (-a: all patches) quilt top prints name of the last applied patch quilt refresh refreshes last patch (so that it applies cleanly, without fuzz and oﬀsets) quilt series prints the names of all patches

$ quilt diff $ quilt diff --color=auto or quilt diff | colordiff $ quilt files or quilt diff | lsdiff $ quilt diff | diffstat $ quilt diff -U num

view diﬀerences in vimdiﬀ (side by side):

$ alias qdv=’quilt diff -p0 -C 9999 --color=never --diff=vimdiff’

1 quilt push -a steps 3 and 4 can be 2 quilt new combined to foo-fix-bug.patch quilt edit src/file.c 3 quilt add src/file.c the patch appears in parent directory (after 4 vim src/file.c quilt refresh) 5 hack, hack you have to add it to 6 quilt refresh specﬁle

goal: import patch (e.g. taken from upstream) to quilt 1 quilt push -a 2 quilt import /path/to/bugfix.patch 3 quilt push 4 quilt refresh

bugfix.patch appears in the parent directory -p num controls how many directory levels to strip (as with patch(1)) -P name overrides name of the patch multiple patches can be imported at once

when quilt push fails, use quilt push -f applies as much as possible, leaves rejects (file.c + file.c.rej)

$ quilt push Applying patch gnupg-files-are-digests.patch patching file g10/gpg.c Hunk #3 succeeded at 1998 (offset -4 lines). Hunk #4 succeeded at 2487 (offset -4 lines). patching file g10/options.h Hunk #1 FAILED at 194. 1 out of 1 hunk FAILED -- rejects in file g10/options.h patching file g10/sign.c Patch gnupg-files-are-digests.patch does not apply (enforce with -f)

failed hunks in *.c.rej have to be applied manually (vim *.c) then quilt refresh helper: vim quilt plugin (zypper install vim-plugin-quilt) (Contrib) vim foo.c - if there is also foo.c.rej - opens vertically split view easier to copy code from rejected hunks to original ﬁle the plugin can do more: :help quilt

$ alias q=’quilt’ $ alias qd=’quilt diff --color=always’ $ alias qdv=’quilt diff -p0 -C 9999 --color=never --diff=vimdiff’ $ alias qp=’quilt push --color=auto’ $ alias qs=’quilt setup -v *.spec’

$ cat ~/.quiltrc # show C function name in hunk header QUILT_DIFF_OPTS=’-p’

we have to tell osc about new/deleted ﬁles example: update foo-1.0 to foo-2.0 $ ls foo*bz2 foo-1.0.tar.bz2 foo-2.0.tar.bz2 $ osc status ? foo-2.0.tar.bz2 $ rm foo-1.0.tar.bz2 $ osc ar # osc addremove D foo-1.0.tar.bz2 A foo-2.0.tar.bz2 $ osc st # osc status D foo-1.0.tar.bz2 A foo-2.0.tar.bz2

osc add + osc remove = osc ar

osc diff displays changes in the package more readable:

$ alias osccd=’osc diff | colordiff | less’

osc vc opens foo.changes ﬁle (prepares new entry header, unless -e option is given) vim can open compressed tarballs: :tabnew foo-2.0.tar.bz2 search for Changelog, NEWS, . . . ; open copy, paste and edit changelog entry osc commit osc sr

to make changelog editing with vim more convenient, add to .vimrc:

set nocompatible filetype plugin on syntax on autocmd BufRead,BufNewFile *.changes.* set filetype=changes autocmd FileType changes set spelllang=en_us spell textwidth=67

supports completing: osc subcommand names (checkout, commit, . . . ) global long osc options (e.g. --quiet, --debug,...) long subcommand options project names package names (in given project) ﬁle names (in given project/package) repository names (openSUSE Factory,...) ... http://gitorious.org/opensuse/osc-bash-completion git clone git://gitorious.org/opensuse/osc-bash-completion.git see README

osc com => osc commit osc build --a => osc build --alternative-project osc ls Ar => osc ls Archiving osc ls Base:System pa => osc ls Base:System parted osc meta a => osc meta attribute osc request => osc request accept approvenew co decline checkout list log revoke show wipe ...

packager often needs to understand code written by somebody else application crashes patch does not apply to new version build fails there are tools that signiﬁcantly help: ctags cscope both work with C/C++, ctags supports many other languages zypper install ctags cscope

ctags creates database of symbols in the source code tree editor (or other tools) use this database to quickly search for symbols and “jump” to the deﬁnition to create the database (tags ﬁle): $ cd foo-1.0.0 $ ctags -R .

vim automatically uses ctags database if present Ctrl+] / Ctrl+L click jump to definition of symbol under cursor Ctrl+t / Ctrl+R click jump back :tag symbol jump to definition of “symbol” vim -t symbol open vim and jump directly to definition of “symbol” :help ctags

sometimes you forget exact name of the symbol bash completion becomes handy:

$ ctags -R . $ vim -t sym => vim -t symbol_name_full

http://vim.wikia.com/wiki/Using bash completion with ctags and Vim put it to ~/.bash completion zypper install bash completion

ctags can “only” jump to symbol definition; cscope can do more whith cscope, it is possible to quickly lookup e.g. where is the symbol defined (same as ctags) where is the symbol used what functions call this function what functions are called by this function which files include this file ... support in vim http://cscope.sourceforge.net/cscope vim tutorial.html

source code browser plugin for vim provides overview of the structure of source code ﬁles zypper install vim-plugin-taglist (Contrib) set up taglist binding in .vimrc:

nnoremap :Tlist

do not reinvent wheel - before writing patch, make sure it does not already exist Fedora uses git now:

# Credits: Pavol Rusnak function fedora_getpkg() { git clone git://pkgs.fedoraproject.org/${1}.git }

tig is ncurses interface for git (zypper install tig) run tig from the cloned repository, press h for help to ﬁnd and “extract” patch form git repository, add the following to ˜/.tigrc:

bind generic P !git format-patch %(commit)^..%(commit)

run tig (e.g. tig v2.2..) select the patch you want to extract hit P - the patch will be saved as sth like 0001-fix-name.patch

sometimes not obvious what rpm %macro does rpmbuild --showrc lists all deﬁned RPM macros rpm -E %macro prints expanded macro supported by bash completion: $ rpm -E %la => rpm -E %lang_package

%package lang Summary: Languages for package %{name} Group: System/Localization Requires: %{name} = %{version} Provides: %{name}-lang-all = %{version} Supplements: packageand(bundle-lang-other:%{name}) BuildArch: noarch %description lang Provides translations to the package %{name}

Questions?