Effective packaging Petr Uzel L3/Maintenance [email protected] ptr uzl @ IRC About this talk collection of tips and tricks that make (some) package maintenance tasks more efficient no rocket science, no cool technologies based on experience there are definitely better ways 2/1 c Novell Inc. All Right Reserved Which package is broken? given a bug (e.g. /path/to/bin crashes), sometimes it is not obvious which package should be fixed bugzilla does not track package info rpm -qf /path/to/file gives name of the RPM we need source package, not RPM $ rpm -qf /lib64/libblkid.so.1 libblkid1-2.18-56.1.x86_64 $ osc search -s --package blkid No matches found for 'blkid' in packages 3/1 c Novell Inc. All Right Reserved Source package name let's ask RPM: $ rpm -q --queryformat '%{SOURCERPM}\n' \ libblkid1-2.18-56.1.x86_64 util-linux-2.18-56.1.src.rpm Everything combined together (file ! source package) $ rpm -q --queryformat '%{SOURCERPM}\n' \ $(rpm -qf /lib64/libblkid.so.1) | \ sed -e 's/\(^.*\)-.*-.*src.rpm/\1/' util-linux Where did I install the package from? $ rpm -q --queryformat '%{DISTURL}\n' util-linux obs://build.opensuse.org/home:puzel:staging/openSUSE_11.3/6a[CUT] 4/1 c Novell Inc. All Right Reserved Get package source from OBS With branching: slow: osc branch openSUSE:Factory foo; osc co home:*:branches:PRJ faster: osc branch --checkout openSUSE:Factory foo fastest: osc getpac foo or osc bco foo hyperspeed: osc getpac + osc bash completion Without branching: function show_devel { osc meta pkg openSUSE:Factory ${1} | fgrep devel } go directly to devel project 5/1 c Novell Inc. All Right Reserved Building package osc build in package working copy usually does the right thing more control: osc build [OPTS] REPOSITORY ARCH SPECFILE REPOSITORY, ARCH and SPECFILE detected automatically if not specified list REPO-ARCH combinations: osc repos [PRJ] 6/1 c Novell Inc. All Right Reserved osc build options build debuginfo and debugsource packages: osc build --debuginfo/-d skip post build checks: osc build --nochecks delete old build root before build: osc build --clean use alternative project for build: osc --alternative-project=openSUSE:Factory use ccache to speed up build: osc build --ccache 7/1 c Novell Inc. All Right Reserved Use custom RPMs in buildroot foo depends on bar, which is not in buildservice yet but we have bar.rpm osc build -p/--prefer-pkgs DIR: prefer packages from DIR when installing buildroot $ cd foo $ osc build -p /path/to_dir_with_bar.rpm foo.spec osc build -x/--extra-pkgs PKG: add this package to buildroot 8/1 c Novell Inc. All Right Reserved Buildroot tips by default, buildroot is /var/tmp/build-root OSC BUILD ROOT overrides the default $ export OSC_BUILD_ROOT=/var/tmp/build-root-temp $ osc build use when parallelly building multiple packages buildroot on tmpfs 5% speedup of parted build $ grep build-root /etc/fstab tmpfs /var/tmp/build-root tmpfs size=5G 0 0 9/1 c Novell Inc. All Right Reserved Avoid password for building by default, osc build asks for root password; to avoid it set su-wrapper = sudo in ~/.oscrc add ``yourlogin ALL = (root) NOPASSWD: /usr/bin/build'' to sudoers file 10/1 c Novell Inc. All Right Reserved Build log build log is stored in $OSC BUILD ROOT/.build.log slow: less /var/tmp/build-root/.build.log faster: osc lbl/localbuildlog respects $OSC BUILD ROOT from working copy osc rbl/remotebuildlog PRJ PKG REPO ARCH - get remote build log 11/1 c Novell Inc. All Right Reserved Version update workflow 1 get package source 2 download tarball 3 verify tarball 4 repack tarball 5 check differences 6 adjust specfile 7 refresh patches 8 run test build 9 test the package 10 write changelog entry 11 commit 12/1 c Novell Inc. All Right Reserved Downloading tarball with ncftp many projects publish tarballs on FTP sites ncftp is a command-line FTP client with bookmarking support useful to remember locations where upstream publishes tarballs supported by bash-completion $ ncftp ut<tab> -> ncftp util-linux ncftp... ls ncftp... get fil<tab> -> filename ncftpbookmarks: bookmark editor 13/1 c Novell Inc. All Right Reserved Repacking tarball packaging policy requires the tarball to be bzip2-ed (or xz-ed) sometimes upstream provides only .tar.gz $ bznew parted-2.3.tar.gz $ ls *.bz2 parted-2.3.tar.bz2 14/1 c Novell Inc. All Right Reserved What's new in tarball? Compare outputs from configure --help $ git clone git://gitorious.org/opensuse/pack-tools.git $ diffconf gnupg-2.0.10.tar.bz2 gnupg-2.0.16.tar.bz2 15/1 c Novell Inc. All Right Reserved Change specfile usually just open specfile and increase the version number small helper: $ alias vs='vim -p *.spec' in vim (normal mode), press Ctrl+A/Ctrl+X to increase/decrease numbers 16/1 c Novell Inc. All Right Reserved Packaging with quilt about quilt Quilt allows you to easily manage large numbers of patches by keeping track of the changes each patch makes. Patches can be applied, un-applied, refreshed, and more. 17/1 c Novell Inc. All Right Reserved Packaging with quilt quilt setup initialize a source tree from RPM spec file 1 unpack tarball 2 apply patches 3 un-apply patches (but remembers the series) IOW: %prep phase $ quilt setup -v foo.spec $ cd foo-1.0.0 shortcut: $ alias qs='quilt setup -v *.spec' 18/1 c Novell Inc. All Right Reserved Packaging with quilt working with series of patches now we are in the directory created by quilt setup (unpacked tarball) none of the patches is applied (but quilt \knows" about them) quilt push applies one patch (-a: all patches) quilt pop un-applies one patch (-a: all patches) quilt top prints name of the last applied patch quilt refresh refreshes last patch (so that it applies cleanly, without fuzz and offsets) quilt series prints the names of all patches 19/1 c Novell Inc. All Right Reserved Packaging with quilt inspecting patches $ quilt diff $ quilt diff --color=auto or quilt diff | colordiff $ quilt files or quilt diff | lsdiff $ quilt diff | diffstat $ quilt diff -U num view differences in vimdiff (side by side): $ alias qdv='quilt diff -p0 -C 9999 --color=never --diff=vimdiff' 20/1 c Novell Inc. All Right Reserved Packaging with quilt creating new patch 1 quilt push -a steps 3 and 4 can be 2 quilt new combined to foo-fix-bug.patch quilt edit src/file.c 3 quilt add src/file.c the patch appears in parent directory (after 4 vim src/file.c quilt refresh) 5 hack, hack you have to add it to 6 quilt refresh specfile 21/1 c Novell Inc. All Right Reserved Packaging with quilt importing patch goal: import patch (e.g. taken from upstream) to quilt 1 quilt push -a 2 quilt import /path/to/bugfix.patch 3 quilt push 4 quilt refresh bugfix.patch appears in the parent directory -p num controls how many directory levels to strip (as with patch(1)) -P name overrides name of the patch multiple patches can be imported at once 22/1 c Novell Inc. All Right Reserved Packaging with quilt rejected patches when quilt push fails, use quilt push -f applies as much as possible, leaves rejects (file.c + file.c.rej) $ quilt push Applying patch gnupg-files-are-digests.patch patching file g10/gpg.c Hunk #3 succeeded at 1998 (offset -4 lines). Hunk #4 succeeded at 2487 (offset -4 lines). patching file g10/options.h Hunk #1 FAILED at 194. 1 out of 1 hunk FAILED -- rejects in file g10/options.h patching file g10/sign.c Patch gnupg-files-are-digests.patch does not apply (enforce with -f) 23/1 c Novell Inc. All Right Reserved Packaging with quilt rejected patches - continued failed hunks in *.c.rej have to be applied manually (vim *.c) then quilt refresh helper: vim quilt plugin (zypper install vim-plugin-quilt) (Contrib) vim foo.c - if there is also foo.c.rej - opens vertically split view easier to copy code from rejected hunks to original file the plugin can do more: :help quilt 24/1 c Novell Inc. All Right Reserved Packaging with quilt make quilt more friendly $ alias q='quilt' $ alias qd='quilt diff --color=always' $ alias qdv='quilt diff -p0 -C 9999 --color=never --diff=vimdiff' $ alias qp='quilt push --color=auto' $ alias qs='quilt setup -v *.spec' $ cat ~/.quiltrc # show C function name in hunk header QUILT_DIFF_OPTS='-p' 25/1 c Novell Inc. All Right Reserved osc status, osc addremove we have to tell osc about new/deleted files example: update foo-1.0 to foo-2.0 $ ls foo*bz2 foo-1.0.tar.bz2 foo-2.0.tar.bz2 $ osc status ? foo-2.0.tar.bz2 $ rm foo-1.0.tar.bz2 $ osc ar # osc addremove D foo-1.0.tar.bz2 A foo-2.0.tar.bz2 $ osc st # osc status D foo-1.0.tar.bz2 A foo-2.0.tar.bz2 osc add + osc remove = osc ar 26/1 c Novell Inc. All Right Reserved Committing changes osc diff displays changes in the package more readable: $ alias osccd='osc diff | colordiff | less' osc vc opens foo.changes file (prepares new entry header, unless -e option is given) vim can open compressed tarballs: :tabnew foo-2.0.tar.bz2 search for Changelog, NEWS, . ; open copy, paste and edit changelog entry osc commit osc sr 27/1 c Novell Inc. All Right Reserved Vim and changelogs to make changelog editing with vim more convenient, add to .vimrc: set nocompatible filetype plugin on syntax on autocmd BufRead,BufNewFile *.changes.* set filetype=changes autocmd FileType changes set spelllang=en_us spell textwidth=67 28/1 c Novell Inc.