Buildroot Using embedded tools to build container images

Jérémy Rosen A couple of disclaimers

I come from the embedded world More precisely the embedded, industrial world. Not the phone/tablet world Quite dierent to the consumer product world I don’t really know anything about datacenter problems This is not about container conguration Nothing about namespaces Nothing about CGroups Nothing about network conguration

We will only talk about rootfs generation.

2/10 Containers in the embedded world

Containers have a limited but real use in the embedded world No thin provisionning, no dynamic deployment Applications are highly hardware dependant Limited use-case for multiversionning of libraries (and old/closed software) Limited use-case as “package manager” The DevOps approch is a hard sell... Rapid deployment doesn’t work on embedded device Exponential number of versions oating around All source-code need to be archived and traced for licence compliance Rollback/failed upgrades are much more frightening in the embedded world Containers as a security feature is...complicated

3/10 Building an image from source

What we want Complete tracability to the line of code Everything is checksumed Complete archival of the source code All source tarballs are archived All patches are archived All overlays are archived All scripts run during build are archived Indepence from the host/Reproducible build Build/archive the build-tools

4/10 Buildroot : An image building tool

Buildroot is an “automated Linux From Scratch” based on advanced makeles Automated build process

Handle dependencies Patch sources Install Download Congure Check SHA Build

Image generation (Fakeroot based)

Tweak permissions Creates disk images Add les and content License collection Creates FS images Source-code collection

Cross-compilation friendly Since 2001

We love our “single command from source to image” approch 5/10 A complete buildroot container build

Minimal image based on buildroot’s default conguration git clone git ://git.buildroot.net/buildroot -b 2019.05.1 make menuconfig Target ⇒ target architecture ⇒ x86_64 make sudo tar xvf output/images/rootfs.tar sudo systemd-nspawn -D -b What we have here busybox, µClibc, no kernel cross-gcc, host-autotools, host-makedevs, host-fakeroot... “heavy” container (it has busybox init) Built in 12min (excluding download), 1.6M total 1 This image is very similar to an alpine-linux built from scratch.

6/10 1. 1min and 6.3M with a glibc pre-compiled toolchain A more realistic example : Light container with Apache

Changing a couple of options in the KCong Pre-built GCC from bootlin (glibc based) Add Apache Remove busybox, init-system support, shell support After rebuild, we have a functionnal light container No shell, no init No customisation Built in 3m, 11M image apachectl doesn’t work...because apachectl is a shell script sudo systemd-nspawn -D --as-pid2 httpd -D FOREGROUND

7/10 Customising the image : overlay

As an example of customisation, let’s transform our container into a portable service To do that, we need an apache.service le added in the rootfs. System Configuration ⇒ Root filesystem overlay directory $(TOPDIR)/overlay mkdir -p overlay/etc/systemd/system add apache.service in that directory [Service] ExecStart=/usr/bin/httpd -D FOREGROUND make etc. portablectl attach -p trusted apache systemctl start apache

The overlay directory makes it trivial to see what has changed in your image (and git it) You can have multiple overlay directories 8/10 other customisation tools

patch directory : easily patch any package from upstream. overlay directory : most of the customisation go here You can have multiple overlay directories post-rootfs script : adjust permission in fakeroot. post-image script : adjust things at image/disk generation time

These tools are here to allow you to never manually tweak the image. All changes should be done through buildroot to keep changes trackable and image-builds reproducible

9/10 Conclusion : why use buildroot

What it brings Source based, signatures included in the recipes No need to be root to build images Trivial to understand, easy to master. Archival/tracability/license-compliance Highly independant from the host system All patches are visible and easy to manage Reduced attack surface Reproducible builds (experimental) Easy to build portable services/light-containers Easy to debug (all sources, debug symbols, build artifacts are available) Drawbacks Need to understand Linux. No hand-holding by the distro. Hard to debug (no shell, remote gdb)

10/10 Build time.