Roll Your Own? a Neer Klaus Knopper

Build your own Distro Intro COVER STORY ROLLTools and techniques forYOUR creating custom Linux systemsOWN

Tools such as Linux From Scratch, NimbleX Live CD Generator, Kiwi image system, and the Ubuntu Custom-

ization Kit can help you create your own custom Linux system. BY KLAUS KNOPPER

f you can’t find a Linux that strikes by Knoppix founder and Live-distro pio- these CDs were merely intended as huge your fancy, why not roll your own? A neer Klaus Knopper. bootable floppies with a command shell Inew generation of Linux tools helps and some troubleshooting tools, I started you build a custom Linux system. You Why Create a Live CD? thinking about the possibility of creating can use these distro-building tools to In 1999, collections of system repair a CD that was not just a “set of tools,” create Live CDs – bootable discs suitable tools on bootable business-card-sized but a full Linux desktop system on a CD. for troubleshooting, mass installation, CDs became a If I could put a and even ordinary end-user tasks. Add a popular mar- whole Linux sys- missing driver to the base system and re- keting give- tem on a portable master a modified DVD, then start your away at CD, I wouldn’t familiar home environment from any- Linux-related have to carry a where – without disturbing the contents expos. laptop with me to of the hard drive. Although foreign IT environ- Today’s Live systems aren’t confined ments. I could just to the CD tray. Live-Linux users also boot from USB sticks and other portable media. And not all custom distros fall in the Live niche. If you’re really ambi- tious, you can build a conventional hard-disk system file by file. This month, we examine tools for creating Linux systems. We start with a look at the doc- uments and utilities of the Linux From Scratch project. We also take you to the NimbleX website, where you can create a custom bootable CD online. And we examine a pair of competing distribution tools from a pair of major vendors: SUSE’s Kiwi and the Ubuntu Customization Kit (UCK). But first, we start with an intro- duction to the challenges of Live Linux

COVER STORY

Linux From Scratch ...... 31

NimbleX Live CD Generator ...... 36

Kiwi ...... 40

Ubuntu Customization Kit ...... 43

MARCH 2008 ISSUE 88 21

021-025_coverintro.indd 21 17.01.2008 18:54:23 Uhr COVER STORY Build your own Distro Intro

isolinux.bin is a small program with the same purpose as lilo or grub. This mini-bootloader loads an operating system’s kernel and accepts options typed 3. Read file again from within an interactive boot shell. User Process 2. Write/modify The isolinux.bin boot loader also allows same file loading of additional menus. Some dis- tributions offer a fancy interface with graphics, interactive elements, and mouse support, but so far, the plain isolinux bootloader has shown the best 2. Reading file /ramdisk (read-write) compatibility with the full range of BIOS NOT in ramdisk yet types. With isolinux.bin, cases in which the CD won’t boot because of problems with the BIOS/bootloader combination are very rare. /KNOPPIX on CD (read-write) isolinux allows boot options and kernel choices, but you have to type them on the boot command line. If you just hit mount - t aufs -o br:/ramdisk=rw:/KNOPPIX=ro none /UNIONFS Return without entering the options, isolinux continues to load the kernel with Figure 1: A union filesystem invisibly integrates a writable ramdisk with a read-only medium. default options set in isolinux.cfg (List- ing 2). Because isolinux is capable of slip my portable system into any avail- some of which you’ll learn about in this reading files directly from the ISO file- able computer and boot my own familiar issue. Behind the scenes, though, the de- system structure of a CD (hence the work environment. The concept of boot- velopers of these tools face the same name), no additional drivers or mounts ing from an easily transportable medium problems I faced when creating the first are necessary in order to access the ker- was a very promising idea. versions of Knoppix. No matter what nel and the initial ramdisk. Some Linux distributors offered Live special features you put inside, a good isolinux accesses the CD using BIOS demos and semi-Live bootable CDs (CDs Live system has to boot Linux, support calls. As soon as the operating system that offered some basic tools as a front write access to important files, and suc- kernel starts and switches to protected end for a hard-disk installation), but cessfully detect the system hardware. mode, everything about the CD – includ- these tools were intended for demonstra- ing its location – is “forgotten.” At that tion purposes – to get people to buy the Booting from a CD or DVD point, the operating system must find “real” product in a shrink-wrapped box. Although hard-disk-like devices (includ- where the CD is located, mount its file- I wanted a Live system for practical ing USB sticks, Zip media, and even work: software tests, presentations, and floppy disks) read a boot record at de- Listing 2: isolinux.cfg play. I did not spend much time on find- vice or partition start, the El Torito BIOS 01 DEFAULT linux ing a fancy name for this experiment; I standard for CD-ROMs and DVDs takes a 02 APPEND initrd=minirt.gz just called it Knoppix [1]. different approach to loading an OS. The Affordable USB memory sticks and developer has three options for design- 03 TIMEOUT 300 USB hard disks were not available then, ing a bootable CD or DVD: 04 PROMPT 1 so I decided to take a look at the techni- • Put a bootable floppy disk image 05 cal aspects of booting and running soft- (which cannot be larger than 2.88MB) 06 LABEL knoppix ware directly from a read-only CD. This on the CD and set floppy emulation 07 KERNEL linux task posed a unique set of problems, mode. which I will describe in this article. • Use a (very small) hard-disk image 08 APPEND initrd=minirt.gz Live systems have grown much more (a seldom-used option). sophisticated since the early days. • Start a bootloader program present on SysLinux Knoppix now comes as a full DVD the CD in no emulation mode. (See An equivalent to isolinux for DOS- image. Several tools are available for Listing 1 for a command that creates a formatted hard-disk-like devices (such creating custom Live CDs and DVDs – bootable CD ISO image.) as USB sticks and flash memory) is called SysLinux. It is quite easy to create Listing 1: Creating a Bootable CD ISO Image a bootable memory stick from a CD by just installing SysLinux and renaming 01 mkisofs -pad -l -r -J \ boot/isolinux/isolinux.cfg to syslinux. 02 -no-emul-boot -boot-load-size 4 -boot-info-table \ cfg. Knoppix versions from 5.0 and up 03 -b boot/isolinux/isolinux.bin -c boot/isolinux/boot.cat \ include a small utility called mkbootdev 04 -o cd-image.iso cd-data-directory for this task.

22 ISSUE 88 MARCH 2008

021-025_coverintro.indd 22 17.01.2008 18:54:59 Uhr Upcoming Conferences

2008 LINUX STORAGE & FILESYSTEM WORKSHOP THE SIXTH INTERNATIONAL CONFERENCE ON Co-located with FAST ’08 MOBILE SYSTEMS, APPLICATIONS, AND SERVICES FEBRUARY 25–26, 2008, SAN JOSE, CA, USA (MOBISYS 2008) http://www.usenix.org/lsf08 Jointly sponsored by ACM SIGMOBILE and USENIX JUNE 10–13, 2008, BRECKENRIDGE, CO, USA http://www.sigmobile.org/mobisys/2008/ 6TH USENIX CONFERENCE ON FILE AND STORAGE TECHNOLOGIES (FAST ’08) Sponsored by USENIX in cooperation with ACM SIGOPS, IEEE Mass 2008 USENIX ANNUAL TECHNICAL CONFERENCE Storage Systems Technical Committee (MSSTC), and IEEE TCOS JUNE 22–27, 2008, BOSTON, MA, USA FEBRUARY 26–29, 2008, SAN JOSE, CA, USA http://www.usenix.org/usenix08 http://www.usenix.org/fast08

2008 USENIX/ACCURATE ELECTRONIC 2008 ACM INTERNATIONAL CONFERENCE ON VIRTUAL VOTING TECHNOLOGY WORKSHOP (EVT ’08) EXECUTION ENVIRONMENTS (VEE ’08) Co-located with USENIX Security ’08 Sponsored by ACM SIGPLAN in cooperation with USENIX JULY 28–29, 2008, SAN JOSE, CA, USA MARCH 5–7, 2008, SEATTLE, WA, USA http://vee08.cs.tcd.ie 3RD USENIX WORKSHOP ON HOT TOPICS IN SECURITY (HOTSEC ’08) USABILITY, PSYCHOLOGY, AND SECURITY 2008 Co-located with USENIX Security ’08 Co-located with NSDI ’08 JULY 29, 2008, SAN JOSE, CA, USA APRIL 14, 2008, SAN FRANCISCO, CA, USA http://www.usenix.org/upsec08 17TH USENIX SECURITY SYM POSIUM JULY 28–AUGUST 1, 2008, SAN JOSE, CA, USA 1ST USENIX WORKSHOP ON LARGE-SCALE http://www.usenix.org/sec08 EXPLOITS AND EMERGENT THREATS (LEET ’08) BOTNETS, SPYWARE, WORMS, AND MORE Co-located with NSDI ’08 22ND LARGE INSTALLATION SYSTEM ADMINISTRATION APRIL 15, 2008, SAN FRANCISCO, CA, USA CONFERENCE (LISA ’08) http://www.usenix.org/leet08 Sponsored by USENIX and SAGE Paper submissions due: February 11, 2008 NOVEMBER 9–14, 2008, SAN DIEGO, CA, USA

5TH USENIX SYMPOSIUM ON NETWORKED 8TH USENIX SYMPOSIUM ON OPERATING SYSTEMS SYSTEMS DESIGN AND IMPLEMENTATION DESIGN AND IMPLEMENTATION (OSDI ’08) (NSDI ’08) DECEMBER 8–10, 2008, SAN DIEGO, CA, USA Sponsored by USENIX in cooperation with ACM SIGCOMM http://www.usenix.org/osdi08 and ACM SIGOPS Paper submissions due: May 8, 2008 APRIL 16–18, 2008, SAN FRANCISCO, CA, USA http://www.usenix.org/nsdi08

USENIX: THE ADVANCED COMPUTING SYSTEMS ASSOCIATION

TECHNICAL SESSIONS AND TRAINING PROGRAM INFORMATION AND HOW TO REGISTER ARE AVAILABLE ONLINE AND FROM THE USENIX OFFICE: http://www.usenix.org/events | Email: [email protected] | Tel: +1.510.528.8649 | Fax: +1.510.548.5738

021-025_coverintro.indd 23 17.01.2008 18:55:17 Uhr COVER STORY Build your own Distro Intro

system, and continue to start the system. If the kernel does not have a driver for the controller the CD is plugged into, continuing to boot is not possible and you will most likely end up with a “kernel panic” because the kernel won’t be able to mount the root filesystem. In the case of Knoppix, you’ll receive a message that indicates the CD cannot be found. (This message will sound odd to someone who just put the CD in the drive and booted from it.) The initial ramdisk, which is loaded by the bootloader when the system is still in “BIOS mode,” must contain all drivers and commands necessary to access the CD again and mount it successfully. The Read-Only Problem The next big question faced by a Live CD designer is how to create a writable space for the system. Luckily for the de- Figure 2: The shell scripts at the Linux Live website help you create a Live system based on velopers of Linux Live CDs, Linux does your current Linux installation. not need a lot of writable space in nor- mal operation. programs such as web browsers that use rary files. Just mounting /home and /tmp The minimum is a writable home di- a local cache – and /tmp as a global tem- as a ramdisk will provide a nearly com- rectory – necessary for applications that porary space, which also is used by shell plete working system. write personal configuration files and scripts for inline redirection and tempo- Of course, the requirements greatly depend on what you plan to do with the Speed and Disk Space Live system. For a demo, it is fine to use Because a Live system is limited in size, data streams, with no gaps in between, simple default settings that work on vir- and because it must operate from the since a gap would cause another seek. tually every computer (such as a VESA graphics driver and fixed settings for the relatively slow CD tray, Live CD develop- Another problem associated with Live ers try to make the best possible use of systems is the lack of available space. desktop). available resources. Many of the steps For the first versions of Knoppix, 700MB If you want a fully configurable sys- for optimizing a Live system are quite of available space seemed like a lot, es- tem, you will need write permissions for different from the steps for optimizing pecially compared with the business- at least /etc and /var. Just copying them a system that runs from the hard drive. card-sized Live CDs, which had to fit in to the ramdisk quickly eats up all avail- One example of a problem associated only about 20MB. However, a modern able memory, and most of the space with writing for the CD is latency of read Linux distro with decent desktop soft- would be wasted by files that might not access. CD-ROMs have comparably ware requires about 2GB, and desktops ever change. heavy mechanics – the laser head is very like KDE and Gnome always tend to slow in positioning to the requested sec- grow in size in every new release. Suggested Solutions tor. Compared with devices with no Iptables author Paul “Rusty” Russell moving parts (like USB memory sticks), One possibility is to copy only the files quickly hacked up a kernel module CDs and DVDs have dramatically slower that are likely to change to the ramdisk. based on the file-backed loop module for read access just because of seek times. This option is inexact, however, requir- a repair toolkit CD. This module lets you ing a lot of time and guesswork to deter- A solution to the read latency issue is to store a filesystem with blockwise gzip defragment the filesystem so that the compression, which reduces the data mine which files will be modified. system requires only one seek per file. size to 1/3 of the original, tripling the Another possibility is to make certain Another time-saving step is to place files available space for an average installa- directories writable and create symlinks that are frequently accessed together, tion. I ported and later rewrote this mod- to the files within them. A simple com- right behind each other in the access ule for newer kernels, extending it for mand like cp -aus /read-only-dir/etc order so they can be read in a continu- 64-bit file access. In addition to saving /ramdisk/etc does this. /ramdisk/etc and ous sweep. space, the cloop module [5] improves all subdirectories are now writable, but ISO9660 is a good filesytem for read- read speed because the CD reader only the files contained within are just sym- only data because it stores directory in- reads about 1/3 of the data that an un- links to the mounted read-only medium. formation right at the beginning of the compressed file would require, which re- This surprisingly stable technique was filesystem (not scattered between files). sults in a net time savings even though used in Knoppix up to version 3.7. You ISO9660 also stores files as continuous the data must be uncompressed. would think that this method would

24 ISSUE 88 MARCH 2008

021-025_coverintro.indd 24 17.01.2008 18:55:24 Uhr Build your own Distro Intro COVER STORY

cause lot of “permission denied” mes- tain a kernel that is not too CPU specific, rary files and manage private settings – sages when programs try to overwrite with as many drivers as possible com- such as passwords and secret keys – but files symlinked to the read-only area, but piled as modules. Knoppix uses a kernel tasks related to customizing and adding in fact, most programs are implemented compiled in 386-compatibility mode, new software have become significantly in a way such that the original file is first which should run on all CPUs since the easier. moved away, a modified copy is created, first 80386. Although it is still my own preference and then the moved original file (in this For detection speed, most IDE and to build a Live CD from scratch, some case, the old symlink) is deleted upon SATA drivers needed for CD-ROMs are developers have created and published success. The result is that the symlink directly compiled into the kernel, easy-to-use scripts with menus that let gets replaced with a real file when the whereas others (SCSI, USB, sound) are you create Live CDs from a master file is modified. loaded as modules. installation very quickly; for example, For programs that use a direct write on Udev (the standard dynamic hard- check out the Linux Live project [6]. the symlink, the files should be manu- ware detection and device manager in Read on for more articles about other ally copied into the ramdisk. In my expe- Debian) and hwsetup (a Knoppix- distro-building tools. I rience, some CUPS configuration files, specific tool written in C that loads some Samba databases, and /var/log/ drivers and sets symlinks independently {wtmp,utmp} need this extra attention. of Udev) take care of activating all INFO The problem with this option is that you peripheral devices. [1] Knoppix: http://www.knoppix.net/ still do not have the full writing capa- Another important feature is auto- [2] UnionFS: http:// www. filesystems. org/ bilities needed to try out new software matic creation of configuration files. project-unionfs. html at arbitrary places in the filesystem. Manual configuration is very time [3] AUFS: http:// aufs. sourceforge. net/ Just copying everything to the ramdisk consuming – especially for the Xorg as symlinks eats up too much RAM. graphical subsystem. The X server has [4] Comparing UnionFS and AUFS: http:// www. unionfs. org/ The third option is to overlay a writ- an auto-creation routine for /etc/X11/ able ramdisk with read-only data. This xorg.conf, but unfortunately it is not very [5] Cloop module: “Cloop” technique, which has gained popularity reliable. by Klaus Knopper, Linux Magazine, January 2008, pg. 32 with the appearance of the UnionFS file- Knoppix uses scripts to create a work- system (Figure 1), really is like magic. ing xorg.cnf file from templates and chip- [6] Linux Live Project: set-specific information obtained from http:// www. linux-live. org/ UnionFS and AUFS DDC probes and Had UnionFS existed in 1999, many the PCI subsys- problems in Knoppix would have been tem. solved much more elegantly – without Currently, each a lot of symlinks and scripts as work- distribution N8EKKFBEFN arounds for making read-only data writ- comes with its able on demand. UnionFS [2] was devel- own hardware oped at Stony Brook University and was detection utility, N?8KÊJLGE8Q@E<GI8Q@E<%:FD&E

MARCH 2008 ISSUE 88 25

021-025_coverintro.indd 25 17.01.2008 18:55:28 Uhr