I deb, you deb, everybody debs Debian packaging for beginners and experts alike Ondřej Surý • [email protected] • [email protected] • 25.­ 10. 2017 Contents ● .deb binary package structure ● Source package structure ● Basic toolchain ● Recommended toolchain ● Keeping the sources in git ● Clean build environment ● Misc... ● So how do I become Debian Developer? My Debian portfolio (since 2000) ● Mostly team maintained ● BIRD ● PHP + PECL (pkg-php) ● Cyrus SASL ○ Co-installable packages since 7.x ● Cyrus IMAPD ○ Longest serving PHP maintainer in Debian ● Apache2 + mod_md (fresh) (and still not crazy) ● ...other little stuff ● libjpeg-turbo ○ Transitioned Debian from IIJ JPEG (that Older work crazy guy) to libjpeg-turbo ● DNS Packaging Group ● GTK/GNOME/Freedesktop ○ CZ.NIC’s Knot DNS and Knot Resolver ● Redmine/Ruby ○ NLnet Lab’s NSD, Unbound, getdns, ldns Never again, it’s a straight road to madness ○ PowerDNS ○ ○ BIND 9 ● Berkeley DB and LMDB (pkg-db) ○ One Berkeley DB per release (yay!) Binary package structure ● ar archive consisting of: $ ar xv knot_2.0.1-4_amd64.deb x – debian-binary ○ debian-binary x – control.tar.gz x – data.tar.xz ■ .deb format version (2.0) ○ control.tar.gz $ dpkg-deb -X knot_2.0.1-4_amd64.deb output/ ./ ■ Package informatio (control) ./etc/ ■ Maintainer scripts […] ./usr/sbin/knotd ● {pre,post}{inst,rm} […] Misc (md5sum, conffiles) ■ $ dpkg-deb -e knot_2.0.1-4_amd64.deb DEBIAN/ ○ data.tar.xz $ ls DEBIAN/ conffiles control md5sums postinst postrm preinst ■ Actual content of the package prerm ■ This is what gets installed $ dpkg -I knot_2.0.1-4_amd64.deb ● Nástroje pro práci s .deb soubory new debian package, version 2.0. size 218134 bytes: control archive=2737 bytes. ○ dpkg-deb Package: knot ○ dpkg (wrapper around dpkg-deb) Version: 2.0.1-4 Architecture: amd64 ○ ar, tar Source package structure ● Source package metadata Format: 3.0 (quilt) Source: knot <pkg>_<dver>.dsc Binary: knot, [...], knot-doc Architecture: any all ○ Maintainer + Uploaders Version: 2.0.1-4 Build Dependencies and Conflicts Maintainer: Debian DNS Packaging ○ <[email protected]> ○ List of binary packages Uploaders: Ondřej Surý <[email protected]> Homepage: http://www.knot-dns.cz/ ○ Checksums (sha256) Standards-Version: 3.9.6 ○ And more… Vcs-Browser: https://anonscm.d.o/git/pkg-dns/knot.git Vcs-Git: git://anonscm.d.o/pkg-dns/knot.git ● Upstream tarball Build-Depends: debhelper (>= 9), [...], pkg-config Build-Depends-Indep: ghostscript, [...], python-sphinx <pkg>_<uver>.orig.?z Package-List: Original upstream unmodified tarball knot deb net optional arch=any ○ [...] ○ Or repacked due licensing reasons (dfsg) Checksums-Sha1: [...] Checksums-Sha256: ● Changes against the upstream tarball 183b[...]6ce6 958560 knot_2.0.1.orig.tar.xz 29b7[...]681c 17688 knot_2.0.1-4.debian.tar.xz <pkg>_<dver>.debian.tar.xz Files: ○ Debian directory (debian/) 96c1[...]22e5 958560 knot_2.0.1.orig.tar.xz 643a[...]4d6d 17688 knot_2.0.1-4.debian.tar.xz ○ Or other changes in older package formats Debian package versioning [epoch:]upstream_version[-debian_revision] # fixup the versioning (start over) # e.g. new package with same name or mistake # example: git (formelly GNU Interactive Tools) ● epoch – [0-9] dpkg --compare-versions ‘1:2.1.4-2.1’ gt ‘4.3.20-7’ ● upstream_version – [0-9a-z.+-:~] # repacked upstream tarball with non-free parts removed ● debian_version – [0-9a-z.+~] dpkg --compare-versions ‘1.0.3-3’ lt ‘1.0.3+dfsg-1’ ● Version comparison # botched upload without epoch bump dpkg --compare-versions ‘2.1.0+really+2.0.0-1’ gt \ ○ First numerically compare epoch ‘2.1.0-1’ ○ Then left to right upstream_version # final release trumps over alpha/beta/rc ■ Split to non-numeric and numeric parts dpkg --compare-versions ‘2.1.0-1’ gt ‘2.1.0~rc1-1’ All letters goes before numbers ■ # upload to Debian security orproposed-updates ■ ~ loses even to an empty string dpkg --compare-versions ‘5.6.14-0+deb9u1’ lt ‘5.6.14-1’ ○ Finally compare debian_version # ~ vs “” examples ■ Same algorithm as upstream_version dpkg --compare-versions '0:0' lt '1' #true 0 is default Dpkg --compare-versions '1:0' gt '1000' #true dpkg --compare-versions '0~' lt '0' #true dpkg --compare-versions '~~' lt '~~a' #true dpkg --compare-versions '~~' lt '~' #true Source package — debian/ directory ● d/changelog $ ls -1d debian/* debian/changelog ○ List of packaging changes debian/compat debian/control ○ aka Debian changelog debian/copyright ● d/control debian/docs debian/init-d-script ○ Metadata for source and binary packages debian/knot-resolver.default debian/knot-resolver.init ● d/rules debian/knot-resolver.install Rules (Makefile) for building the package debian/knot-resolver.lintian-overrides ○ debian/knot-resolvers.dirs ● d/copyright debian/knot-resolver.service debian/knot-resolver.tmpfile ○ (Machine readable) list of source licenses debian/kresd.conf debian/patches/ ● d/source/format debian/rules ○ 3.0 (quilt) or 3.0 (native) debian/source/ ● d/patches/ ○ Patches for upstream sources ○ Managed by quilt or gbp pq ● And more... Basic (barebone) toolchain ● is Makefile #!/usr/bin/make -f d/rules # -*- makefile -*- ● d/rules <target>, where <target> is: build: ○ clean gcc -o helloworld helloworld.c Return the pristine state ■ clean: ○ build (build-arch, build-indep) rm -f helloworld ■ Compile the sources install: build ○ install (install-indep, install-arch) mkdir debian/tmp/usr/bin cp -a helloworld debian/tmp/usr/bin/ ■ Install into the temporary path ○ binary (binary-arch, binary-indep) binary-indep: build ■ Assemble the file pages binary-arch: build mkdir debian/tmp/DEBIAN/ ● Minimal d/rules → dpkg-gencontrol ○ But don’t do this at home, kids dpkg-deb -b debian/tmp/ <package>_<dver>.deb binary: binary-arch binary-indep .PHONY: build clean install binary binary-arch \ binary-indep dh_make – the first simple step dh_make $ tar -xJf knot-resolver-1.0-beta.tar.xz $ cd knot-resolver-1.0-beta/ ● Creates a basic skeleton in debian/ knot-resolver-1.0-beta$ dh_make -s -f \ ../knot-resolver-1.0-beta.tar.xz ● It’s not magic, you need to modify it Maintainer name : Ondřej Surý Email-Address : [email protected] ● Package types: Date : Fri, 09 Oct 2015 16:08:12 +0200 Package Name : knot-resolver ○ Single (--single) Version : 1.0-beta ■ Just single binary package License : blank Type of Package : Single ○ Multi Hit <enter> to confirm: ■ Multiple binary packages Done. Please edit the files in the debian/ subdirectory now. You should also ○ Library check that the knot-resolver Makefiles install into Shared library (libfoo0 + libfoo-dev) $DESTDIR and not in / . ■ ○ … dh_make_perl, gem2deb ● Simple CPAN or GEM packaging Recommended toolchain – debhelper ● Rich set of dh_* commands for packaging #!/usr/bin/make -f # See debhelper(7) (uncomment to enable) ○ dh_auto_configure, dh_auto_build, # output every command that modifies files on the build system. dh_auto_install, dh_auto_test #DH_VERBOSE = 1 ■ Automatic detection of build system # see EXAMPLES in dpkg-buildflags(1) and read (autotool, cmake, and others) /usr/share/dpkg/* ○ dh_install, dh_strip, dh_installcron, … DPKG_EXPORT_BUILDFLAGS = 1 include /usr/share/dpkg/default.mk ○ Compatibility levels (more features added) v9 (d/compat) # see FEATURE AREAS in dpkg-buildflags(1) ■ export DEB_BUILD_MAINT_OPTIONS = hardening=+all ● Multi-arch support # see ENVIRONMENT in dpkg-buildflags(1) ● Hardening (dpkg-buildflags) # package maintainers to append CFLAGS ■ v10 export DEB_CFLAGS_MAINT_APPEND = -Wall -pedantic # package maintainers to append LDFLAGS ● Excensible framework (… in Perl) export DEB_LDFLAGS_MAINT_APPEND = -Wl,--as-needed ● dh7 added dh command → Runs a sequence of dh_* commands # main packaging script based on dh7 syntax ○ %: dh $@ #DH7_ADDON# Standard debhelper sequence $ dh binary --no-act dh_installlogrotate dh_testdir dh_installpam dh_auto_configure dh_installppp dh_auto_build dh_installudev dh_auto_test dh_installwm dh_testroot dh_installgsettings dh_prep dh_bugfiles dh_installdirs dh_ucf dh_auto_install dh_lintian dh_install dh_gconf dh_installdocs dh_icons dh_installchangelogs dh_perl dh_installexamples dh_usrlocal dh_installman dh_link dh_installcatalogs dh_installxfonts dh_installcron dh_compress dh_installdebconf dh_fixperms dh_installemacsen dh_strip dh_installifupdown dh_makeshlibs dh_installinfo dh_shlibdeps dh_installinit dh_installdeb dh_installmenu dh_gencontrol dh_installmime dh_md5sums dh_installmodules dh_builddeb dh_installlogcheck Files $ cat debian/knot-resolvers.dirs dh_installdirs /etc/knot-resolver ● Helper script to create directories /var/lib/knot-resolver described in d/<package>.dirs $ cat debian/libknot1.install usr/lib/*/libknot.so.* $ cat debian/libknot-dev.install dh_install usr/include/ ● Helper script to copy d/tmp/ contents into usr/lib/*/*.a usr/lib/*/*.so d/<package>/ directories usr/lib/*/pkgconfig/* ● Described d/<package>.install $ cat debian/knot-host.install usr/bin/khost ● It can’t rename files (doh!) usr/share/man/man1/khost.1 ● Since compat level 9 the .install files can be #! /usr/bin/dh-exec src/libfoo-*.so.* executable (#!/usr/bin/dh-exec) debian/foo-plugins/usr/lib/foo/${DEB_HOST_MULTIARCH}/ ○ It can rename files etc/example.conf => debian/foo/etc/foo/foo.conf [linux-any kfreebsd-any] arch-specific /usr/lib/foo/ ○ It can use env variables (f.e. Multi-Arch) Documentation dh_installdocs # příklad ze staršího balíčku override_dh_installdocs: ● Helper script to install files from dh_installdocs -p cyrus-common -p cyrus-doc-2.4 dh_installdocs -p cyrus-doc \ d/<package>.docs + d/copyright, --link-doc=cyrus-doc-2.4