mGAR Producing complex packages for Solaris the easy way OpenCSW Technical Wintercamp Dublin 2011 Dagobert Michelsen [email protected] Samstag, 19. Februar 2011 1 Once upon a time find /usr/local/pkg -print > /tmp/files cat /tmp/files | pkgproto > /tmp/Prototype vim /tmp/Prototype PKG=FOOtree NAME=tree - A recursive directory listing program ARCH=sparc VERSION=1.5.3,REV=2010.02.02 CATEGORY=application VENDOR=http://mama.indstate.edu/users/ice/tree/ [email protected] PSTAMP=joe@darkstar-20100202140539 CLASSES=none HOTLINE=http://www.example.com/ pkgmk -o -r / -d /tmp -f Prototype pkgtrans . tree.pkg FOOtree gzip tree.pkg Samstag, 19. Februar 2011 2 What is missing? What‘s wrong? Compilation Packaging standards Checking for errors Hardcoded prototypes Hardcoded pkginfo Samstag, 19. Februar 2011 3 Enter mGAR gmake package Samstag, 19. Februar 2011 4 What is mGAR? A system to build packages for Solaris Origins are „GAR“ LNX-BBC and GARNOME m = modulated ➜ multiple similar builds Files written in GNU Make 5 Samstag, 19. Februar 2011 5 The simplest case The minimal information to build a package: 1. What should the package be named 2. Where to get the sources 3. What the dependencies are Samstag, 19. Februar 2011 6 Example: gperf NAME = gperf VERSION = 3.0.3 CATEGORIES = devel DESCRIPTION = A perfect hash function generator MASTER_SITES = $(GNU_MIRROR) DISTFILES = $(NAME)-$(VERSION).tar.gz CONFIGURE_ARGS = $(DIRPATHS) TEST_TARGET = check include gar/category.mk Samstag, 19. Februar 2011 7 Categories include gar/category.mk Should be really named 'profiles' Contains settings common to a class of builds Generic Perl Modules Ruby Gems Python Eggs Samstag, 19. Februar 2011 8 Typical customization needs Code needs to be patched to compile or run on Solaris Code doesn‘t honor information passed via ./configure Files need to placed at specific locations in the package Samstag, 19. Februar 2011 9 Phases prerequisites fetch checksum exract patch configure build test install merge package checkpkg ...and custom targets: pre-fetch post-fetch pre-extract (...) Samstag, 19. Februar 2011 10 Modulations „A Modulation is the process of varying one waveform in relation to another waveform“ For software builds that means: „A Modulation is the process of varying a build in relation to certain parameters“ Samstag, 19. Februar 2011 11 Phases and modulations Prerequisites One Executed Fetch modulation globally Checksum Extract Patch Configure Executed inside Build Te s t modulation Install Merge Package Executed globally Samstag, 19. Februar 2011 12 What can be used as modulator? ISA for build architecture For 32/64 bit versions and optimized binaries. See lame Package version Useful when multiple versions of a library should be included See automake ...anything that controls the build Static vs. dynamic linked versions See wget Parameter size See nrpe Wide vs. narrow characters See ncurses Samstag, 19. Februar 2011 13 Modulation over ISA Allows easy building of multiple ISAs Built-in special case of modulation No merge needed also Simplest case BUILD64 = 1 Advanced EXTRA_BUILD_ISAS = sparcv8plus+vis pentium_pro Samstag, 19. Februar 2011 14 Modulation over ISA requirements Certain ISAs require specific platforms x64 requires Solaris 10, but integrated package for Solaris 8-10 Parallelizing modulation builds May require builds on multiple hosts Build platform for each modulation can be specified in the package manifest How does ISAEXEC relate to multi-ISA? Samstag, 19. Februar 2011 15 Modulation over static/dynamic linking Adds modulator with GARNAME = wget this name ... Cool trick: EXTRA_MODULATORS = STATIC Use dynamically expanded variables to set per-modulation MODULATIONS_STATIC = disable enable EXTRA_CONFIGURE_STATIC-disable = --with-ssl EXTRA_CONFIGURE_STATIC-enable = --disable-shared EXTRA_CONFIGURE_STATIC-enable += --enable-static EXTRA_CONFIGURE_STATIC-enable += --without-ssl ... CONFIGURE_ARGS += $(EXTRA_CONFIGURE_STATIC-$(STATIC)) MERGE_SCRIPTS_isa-sparcv8-static-disable = copy-all MERGE_SCRIPTS_isa-sparcv8-static-enable = rename-wget copy-only MERGE_DIRS_isa-sparcv8-static-enable = $(bindir) MERGE_SCRIPTS_isa-i386-static-disable = copy-all MERGE_SCRIPTS_isa-i386-static-enable = rename-wget copy-only MERGE_DIRS_isa-i386-static-enable = $(bindir) merge-rename-wget: @mv $(DESTDIR)$(bindir)/wget $(DESTDIR)$(bindir)/wget.static ... include gar/category.mk Samstag, 19. Februar 2011 16 Modulation over parameter size Ok, I guess you understood GARNAME = nrpe how this works ... EXTRA_MODULATORS = SIZE Use patchfiles on a MODULATIONS_SIZE = 1k 8k per-modulation basis ... PATCHFILES_isa-sparcv8-size-8k = common.h.diff PATCHFILES_isa-i386-size-8k = common.h.diff ... MERGE_SCRIPTS_isa-sparcv8-size-1k = copy-all MERGE_SCRIPTS_isa-sparcv8-size-8k = copy-nrpe-only MERGE_SCRIPTS_isa-i386-size-1k = copy-all MERGE_SCRIPTS_isa-i386-size-8k = copy-nrpe-only include gar/category.mk ... merge-copy-nrpe-only: cp $(INSTALLISADIR)$(bindir)/nrpe_1k $(PKGROOT)$(bindir)/nrpe_8k cp $(INSTALLISADIR)$(LIBEXECDIR)/check_nrpe $(PKGROOT)$(LIBEXECDIR)/check_nrpe_8k @$(MAKECOOKIE) Samstag, 19. Februar 2011 17 Modulations and their names The name of a modulation is <modulator1>-<value1>-<modulator2>-<value2>-... Call each phase from a modulation manually with <phase>-<modulation> Example: gmake build-isa-sparcv8 This results in a sub-make-invocation <phase>-modulated MODULATION=<MODULATION> <MODULATOR1>=<value1> ... Example: gmake build-modulated MODULATION=isa-sparcv8 ISA=sparcv8 Samstag, 19. Februar 2011 18 Merging Why is this needed? Optimized binaries with isaexec Libraries may be located elsewhere in the package than they were configured …/lib/32 -> . …/lib/64 -> sparcv9 | amd64 Pathes stored in pkgconfig-files Samstag, 19. Februar 2011 19 Merging Why is this needed? 32 and 64 bit need different install locations to have sane pkgconfig-files build-isa-sparcv8/opt/csw/bin/myapp Install build-isa-sparcv9/opt/csw/bin/64/myapp pkgroot/opt/csw/bin/myapp Merge pkgroot/opt/csw/bin/sparcv9/myapp /opt/csw/bin/myapp -> isaexec Package /opt/csw/bin/sparcv8/myapp /opt/csw/bin/sparcv9/myapp Samstag, 19. Februar 2011 20 Merging Builtin rules Copy all Copy only some MERGE_DIRS_<mod> Do not relocate copy-all copy-only Do relocate copy-relocate copy-relocated-only Samstag, 19. Februar 2011 21 Merging with custom rules Only merge specific files This is the default build GARNAME = nrpe Just copy everything over ... MERGE_SCRIPTS_isa-sparcv8-size-1k = copy-all MERGE_SCRIPTS_isa-sparcv8-size-8k = copy-nrpe-only We want to control exactly what is used from the additional modulation MERGE_SCRIPTS_isa-i386-size-1k = copy-all MERGE_SCRIPTS_isa-i386-size-8k = copy-nrpe-only Merging copies from the install include gar/category.mk location to PKGROOT ... merge-copy-nrpe-only: cp $(INSTALLISADIR)$(bindir)/nrpe_1k $(PKGROOT)$(bindir)/nrpe_8k cp $(INSTALLISADIR)$(LIBEXECDIR)/check_nrpe $(PKGROOT)$(LIBEXECDIR)/check_nrpe_8k @$(MAKECOOKIE) Samstag, 19. Februar 2011 22 Class Action Scripts They mean more or less 'hooks' f cswinitsmf /opt/csw/etc/init.d/cswpuppetd i.<class> scripts handle the files i.cswinitsmf r.cswinitsmf GAR integration: INITSMF = $(prefix)/init\.d/cswpuppetd Fully automated: Texinfo registration Configuration file preservation Placement of README.CSW in package docs ... Samstag, 19. Februar 2011 23 checkpkg and overrides Examines a set of .pkg files Looks for common issues Written mostly in Python, with lintian in mind allows to override checks Samstag, 19. Februar 2011 24 Your package is ready! Samstag, 19. Februar 2011 25 Questions? Bueller? Anybody? Samstag, 19. Februar 2011 26.