USENIX Association Proceedings of BSDCon ’03 San Mateo, CA, USA September 8–12, 2003 THE ADVANCED COMPUTING SYSTEMS ASSOCIATION © 2003 by The USENIX Association All Rights Reserved For more information about the USENIX Association: Phone: 1 510 528 8649 FAX: 1 510 548 5738 Email: [email protected] WWW: http://www.usenix.org Rights to individual papers remain with the author or the author's employer. Permission is granted for noncommercial reproduction of the work for educational or research purposes. This copyright notice must be included in the reproduced paper. USENIX acknowledges all trademarks herein. build.sh: Cross-building NetBSD Luke Mewburn, Matthew Green The NetBSD Foundation [email protected], [email protected] Abstract 2. Background NetBSD has a cross build infrastructure which allows cross-building of an entire NetBSD release including 2.1. NetBSD bootable distribution media. The build process does not Since the NetBSD project was started in 1993 it has had require root privileges or writable source directories. a goal of being portable [3] to many target platforms. The build process works on many POSIX compatible There has been significant effort in designing, imple- operating systems. This paper explains the changes menting, and improving NetBSD to make it easier to made to NetBSD to enable this build process, enumer- “port” to a new target platform [4]. Device drivers are ates benefits of the work, and introduces future work written in a way that permits easy sharing between plat- enabling cross building of any software for NetBSD. forms without unnecessary code replication [5]. 1. Introduction The source code portability of NetBSD did not equate to “ease of use” when building the system on a host other NetBSD [1] is the most portable Unix operating system than the target platform, or indeed, natively. in common use. It is freely available and redistributable, and runs on a broad variety of platforms from modern Prior to build.sh a NetBSD release for a given platform desktop systems and high end servers that can build an was built “natively” on that platform on a version of the entire release in less than an hour, to embedded systems operating system that was “close” to the target release. and older machines that may take several days to build a There were exceptions, but these alternate processes release. were not simple to use nor easily automated, and had a variety of other limitations which build.sh addresses. In late 2001, work began on changes to improve the ability of NetBSD to be cross built, especially an entire build.sh offers a level of flexibility in building NetBSD release. This system is referred to as “build.sh”, because that has not been addressed by other open source operat- that is the name of the script that is the user-visible ing systems. front-end to the infrastructure. NetBSD 1.6 was the first release to be shipped with 2.2. Cross compiling Unix build.sh, and the Release Engineering group of the Net- Unix was cross-compiled from the beginning, but when BSD Project took advantage of it to cross-build binary native hosting was available, that became the main releases for 39 platforms on a near daily basis during the development methodology and has remained so. release cycle for NetBSD 1.6 [2]. Previous releases required access to each of the various platforms by Cross-compilation is the technique of running programs release engineers, or co-ordination with developers with on a “host” system to generate object code for a differ- that hardware. While that method works for a moderate ent “target” system. This has not been an easy task for number of platforms (NetBSD 1.5 released binaries for most system builders and has generally not been inte- 20 platforms), it does not scale, especially as the number grated into operating system build processes as used by of platforms in NetBSD is growing (54 as of June 2003). open source operating systems. Freely available software projects such as GCC [6] have object files. Shared libraries are created by calling supported being cross-compiled for a long time, and gcc with the -shared option, which calls ld with GCC is part of the GNU toolchain which NetBSD uses various options to create a shared library. and is heavily dependent upon for cross-compiling. gcc’s -v flag may be used to see exactly what external 2.2.1. An introduction to cross-compiling programs are called. For example, cross-compiling a simple NetBSD/sparc “hello world” C program on a There are many parts to a full cross compiler environ- NetBSD/macppc box gives: ment. Besides the compiler itself, many others tools and files are required to create functional programs. Every- what-time-is-love~>/tools/bin/sparc--netbsde thing that a normal compiler needs must be present. For lf-gcc-I/dest/usr/include-L/dest/usr/lib-B/ the GNU toolchain, this includes: dest/usr/lib/-v-save-temps-ohello.xhello. c • The compiler - gcc. Readingspecsfrom/tools/lib/gcc-lib/sparc--n etbsdelf/2.95.3/specs gccversion2.95.320010315(release)(NetBSD • The assembler - as. nb4) /tools/lib/gcc-lib/sparc--netbsdelf/2.95.3/cp • The linker - ld. p0-lang-c-v-I/dest/usr/include-isystem/de st/usr/lib/include-D__GNUC__=2-D__GNUC_MINOR • The “binutils”; size, nm, strip, ar, etc. __=95-D__sparc__-D__NetBSD__-D__ELF__-D__s parc__-D__NetBSD__-D__ELF__-Asystem(unix)- Asystem(NetBSD)-D__GCC_NEW_VARARGS__-Acpu(sp • Header files (provided by NetBSD). arc)-Amachine(sparc)-D__sparchello.chello. i • Libraries (provided by NetBSD and the GNU tool- GNUCPPversion2.95.320010315(release)(Net chain). BSDnb4)(sparc-netbsdelf) #include"..."searchstartshere: #include<...>searchstartshere: Following is a quick overview of how it all works. This /dest/usr/include is basically the same for any compiler; in this example /tools/lib/gcc-lib/sparc--netbsdelf/2.95.3/in the details are from the GCC C compiler: clude /tools/lib/gcc-lib/sparc--netbsdelf/2.95.3/.. /../../../sparc--netbsdelf/include 1. The C compiler front-end gcc calls the C pre- Endofsearchlist. processor cpp on an input source file, usually a Thefollowingdefaultdirectorieshavebeenom “.c” file, producing a “.i” file. This is still valid C ittedfromthesearchpath: code but will now be devoid of C pre-processor /tools/lib/gcc-lib/sparc--netbsdelf/2.95.3/.. directives. (Actually in modern GCC, the cpp pass /../../../include/g++-3 /tools/lib/gcc-lib/sparc--netbsdelf/2.95.3/.. is done inside the cc1 pass to speed up the process /../../../sparc--netbsdelf/sys-include and provide better error reporting. This process is Endofomittedlist. largely invisible to the user.) /tools/lib/gcc-lib/sparc--netbsdelf/2.95.3/cc 1hello.i-quiet-dumpbasehello.c-version-o 2. gcc calls the back-end cc1 with the output of cpp hello.s GNUCversion2.95.320010315(release)(NetBS producing a “.s” file. This is a assembler source file Dnb4)(sparc-netbsdelf)compiledbyGNUCver corresponding to the input C file. sion2.95.320010315(release)(NetBSDnb4). /tools/sparc--netbsdelf/bin/as-32-ohello.o 3. gcc calls the assembler as with the output of cc1 hello.s producing a “.o” file. This is an object file /tools/lib/gcc-lib/sparc--netbsdelf/2.95.3/co llect2-melf32_sparc-dy-dc-dp-e__start- corresponding to the input assembler file. dynamic-linker/usr/libexec/ld.elf_so-ohello .x/dest/usr/lib/crt0.o/dest/usr/lib/crti.o/ 4. gcc calls the linker ld with the output of as plus dest/usr/lib/crtbegin.o-L/dest/usr/lib-L/des several other files (sometimes collectively called t/usr/lib-L/tools/lib/gcc-lib/sparc--netbsdel the “crtstuff”), to produce an executable. f/2.95.3-L/tools/sparc--netbsdelf/libhello.o -lgcc-lc-lgcc/dest/usr/lib/crtend.o/dest/ usr/lib/crtn.o In addition to creating executables, archive and shared libraries are built. Archive libraries are usu- ally created with the ar binutils program from The output shows the cpp0, cc1, as and ld programs 3. A “Canadian cross compile” is when “build host”, being called in succession. An object file is created and “host”, and “target” are all different: the sundry “crtstuff” is added to the final ld line. ./configure --build=i386-netbsdelf \ If gcc is passed the -save-temps option the output of --host=i686-pc-cygwin \ each program will be saved rather than deleted after it --target=powerpc-eabi && make has been processed. In the above example, the following This strange beast is called a “Canadian cross com- files were created: piler”, because when a name was needed there were three political parties in Canada. It is most often what-time-is-love~>ls-lhello.? found when dealing with pre-built toolchains. The -rw-rw-r--1mrgmrg57Jun52001hello.c -rw-rw-r--1mrgmrg7820Jul715:01hello.i above example would produce a toolchain that will -rw-r--r--1mrgmrg872Jul715:01hello.o run on a i686-pc-cygwin host (i.e., a Windows sys- -rw-rw-r--1mrgmrg408Jul715:01hello.s tem with Cygwin installed) and will target embed- -rwxr-xr-x1mrgmrg68638Jul715:01hello.x ded PowerPC platforms. The toolchain would be built on a NetBSD/i386 machine. These are the source file and the cpp, cc1, as and ld outputs, respectively. A normal cross-compiler is required to build a Canadian cross-compiler. For a cross compiler the assembler, linker and binutils must also be “cross” tools. For the GNU toolchain these Most people who build software use a native build in all are normally called “$target-$tool”. For example cases. Usually people using cross compilers use the powerpc-eabi-ld is the linker for the “powerpc-eabi” same “build host” and “host”.