USENIX Association
Proceedings of the XFree86 Technical Conference
Oakland, California, USA November 8Ð9, 2001
THE ADVANCED COMPUTING SYSTEMS ASSOCIATION
© 2001 by The USENIX Association All Rights Reserved For more information about the USENIX Association: Phone: 1 510 528 8649 FAX: 1 510 548 5738 Email: [email protected] WWW: http://www.usenix.org Rights to individual papers remain with the author or the author's employer. Permission is granted for noncommercial reproduction of the work for educational or research purposes. This copyright notice must be included in the reproduced paper. USENIX acknowledges all trademarks herein. A Cross Compile Environment for X
Egbert Eich, XFree86 Core Team, SuSE GmbH Mergenthaleralle 45-47, D-65760 Eschborn, Germany, [email protected]
Abstract Inc.3 The purpose of this port was to “stress test” both the compiler and the operating system with its Cross compiling the X Window System has been system libraries. However, at the time this port was a complicated task. imake macros that differed done, the new CPU was not yet available in hard-
between target and build system had to be identi- ware. £ fied and redefined. It was not possible to use an The SimICS software emulator[MDHG 98] which existing configuration for the target platform. This emulates an entire system consisting of the CPU, document describes the design and implementation buses, IO and network interfaces, disks and graph- of a comfortable cross compile scheme for XFree86 ics hardware was available for testing purposes. which makes use of an existing target configuration. However, it was clear from the beginning that this emulator was not suitable for building the entire X source tree consisting of the X Window System 1 Introduction server as well as several applications.
While a comfortable cross compile environment ex- 1 Advanced Micro Devices, Inc. (AMD) has de- isted to build the GNU compiler gcc the GNU bi- signed a new microprocessor architecture which ex- nary utilities, the Linux kernel and the GNU C Li- tends the instruction set of the ia32 processors to brary, the cross compile environment that existed 64-bit[AMD00] . This new architecture also known ¢¡ for the X Window System was largely unsuitable under the name x86-64 will provide the advan- for our purposes. tages of a 64-bit platform while maintaining full backwards compatibility for existing 32-bit ia32 ap- The XFree86 source code is configured using plications without the sacrifice of performance. imake[DuB93], a tool written during develop-
2 ment of X version 11. While not as automated as Early on, SuSE GmbH (a major Linux distributor) autoconf[VETT00], imake has the advantage of envisioned the new AMD architecture as an inter- providing significantly more control to the devel- esting target platform for the Linux operating sys- oper over the build process. tem. At this very early stage, SuSE began to port the Linux kernel, the system libraries as well as ma- The existing cross compilation mechanism requires jor applications to the x86-64 architecture. hand editing of the imake configuration of the build system to meet the needs of the target system. One of the first major applications to be ported to It is mainly suitable for compiling X for target sys- the new platform after the kernel and a primitive tems that will never host an X build themselves. At shell environment were operational was the X Win- the same time it was heavily dependent not only on dow System [SG92]. This port was to be based on the target system but also on the system that served the source tree maintained by the XFree86 Project, as the build platform. 1http://www.amd.com 2http://www.suse.de 3http://xfree86.org The design goals of the cross compile environment macros from the Imakefile. This Imakefile to be presented here were different: itself contains information about the targets to be built and macros for the rules for the corresponding 1. It had to be independent of the system the Makefile. build was going to take place on. When imake is built, preprocessor macros prede- 2. It had to be able to make use of an existing fined by the C compiler determine the values of imakemdep.h configuration of the target system “in place” macros defined in the header file . imake.c i.e. it should not require additional configura- This header file is itself included in –the imake tion files. source file. The macros defined in imakemdep.h control: (2) was of special importance as the cross com- pile environment for the x86-64 environment was ¤ which flags are needed to build the final ver- supposed to serve as a test base for the port of the sion of imake.
X Window System and should provide roughly the ¤ which preprocessor imake should call to gen- same environment as if the build took place on the erate a Makefile. Depending on the plat- target system itself. form this may either be the C preprocessor This article is divided into 3 parts: First, the build itself or the C compiler invoked with the ap- environment of the X Window System and the cur- propriate flag run only the preprocessing stage rent cross compile environment will be reviewed. (usually -E). The next section will describe the internals of the ¤ whether output files generated by the system’s new cross compile environment. Finally, the re- preprocessor need to reprocessed. Certain sults we obtained by testing the x86-64 port with preprocessors may collapse tabs to a single the cross compile environment will be presented whitespace or replaces escaped newline char- acters with a single space.
2 The build environment of the X Window ¤ flags that should be passed to the preproces- System sor when a Makefile is generated. These flags may be macro definitions which uniquely 2.1 How the build environment obtains infor- characterize the target platform. This is espe- mations about the system cially important if imake is to use the C pre- processor instead of the preprocessing stage of The X Window System obtains its informations the C compiler to generate the Makefile, as about the underlying system almost exclusively the predefined compiler macros are not known from the predefined preprocessor macros of the sys- to the preprocessor. Switches which control tem’s C compiler. This information is gathered the operation of imake may also be specified when compiling the program imake which will here. It should be noted that imake may ex- later control the generation of the Makefiles. tend this list by arguments passed to it on the The creation of Makefiles itself is for the most command line or by values which need to be part done by the C preprocessor which selects rule obtained by imake at run time. and variable definitions from rule and definition files. Which rules and definitions are to be selected ¤ how the default name, version numbers of the is governed by a set of macros listed in an imake- operating system can be obtained from the generated file the preprocessor gets passed on invo- utsname structure or by some other system cation. dependent way.
The preprocessor performs variable substitutions ¤ any system specific symbols the compiler in Makefile templates and expands build rule may define. These symbols are required for makedepend which will parse the source files DEFAULT_OS_NAME they may depend on. DEFAULT_OS_MAJOR_REV DEFAULT_OS_MINOR_REV DEFAULT_OS_TEENY_REV The predefined preprocessor macros also enable DEFAULT_MACHINE_ARCITECTURE platform specific functionalities in imake. imake is build in a two step process. When the to extract values from the operating sys- build is bootstrapped by “make World” a simple tem utsname structure. For each such imake version is build. This will be used to gener- definition found, the related line from ate a Makefile from a simple Imakefile. This this list: Makefile contains the rules to build the final ver- #define DefaultOSName \\ sion of imake which will be used to build X.
macros files adminmacros and ¤ localmacros if either one is present the name of the distribution used. in the current directory: On FreeBSD imake attempts to discover whether #define IMAKE_ADMIN_MACROS \\ the system’s default binary format is ELF. "adminmacros" #include IMAKE_ADMIN_MACROS OpenBSD systems further require a version defini- tion for the preprocessor which needs to be obtained or by imake from the uname() syscall at runtime. #define IMAKE_LOCAL_MACROS \\ "localmacros" #include IMAKE_LOCAL_MACROS 3 How the old cross compile environment works 5. start up the preprocessor passing it Imakefile.c (or the file specified The old cross compile environment did not modify with the -C option on the command the way imake obtained its information about the line) along with any preprocessor flag host system. It pretended to generate a Makefile specified in imakemdep.h or obtained for the build system until all configuration files had by imake. Imakefile.c will include been processed. At that stage all system dependent the IMAKE_TEMPLATE file. Controlled defines had been set. by macros defined in Imakefile.c and Before any templates were filled in and any build on the preprocessor’s command line this rule macros were expanded a file cross.def was template file itself will include the required included. This file redefined all those macros that definition and rule files. Finally the template differed from the build system to target system val- file will include the file specified in the ues. Additionally it defined some simple make INCLUDE_IMAKEFILE macro. rule macros for tools which were needed during the 6. gather the output from the preprocessor, build process - like makedepend and imake itself. do any necessary cleanup (as specified by These needed to run on the host system and there- imakemdep.h) expanding any @@ charac- fore required to be compiled by the host C compiler. ters to newlines and converting any strings XCOMM to #, the comment sign found in 3.1 Disadvantages of the old scheme Makefiles. While the old cross compile environment required very little modifications to the imake build scheme, imake will place the result in Makefile or an- it had a number of disadvantages: other file specified on the command line.
Depending on the platform imake, may add addi- ¤ The redefinitions required to do the cross com- tional macros to Imakefile.c. On systems that pile depended not only on the target system but have the GNU C compiler installed the definitions also on the host system. for the following are added: ¤ If build host and target differed from
¤ each other considerably the redefinitions in the version of the C compiler gcc cross.def became difficult to handle.
¤ the default gcc include directory ¤ It was not possible to use an already existing configuration for the target system. On a Linux system it adds additional macros for:
¤ Only platforms for which a configuration al- ¤ the version of the C library glibc ready existed could be used as build host. To overcome these issues a scheme needed to be 4.3 Internals found which allowed to use the an existing config- uration in place. In other words the macros defined When the make variable CROSSCOMPILEDIR in Imakefile.c needed to expand to values of is set the C cross compiler will be invoked target system, not the build host. with the proper defines to preprocess the header file imakemdep.h. During this preprocessing step, those macros which concern the target plat- 4 Design of a new cross compile environ- form are expanded. Their values are assigned ment to variables defined in the resulting header file imakemdep_cpp.h. When building imake As described above, imake obtains the majority of for cross compiling, this header file will get in- it’s information about the system from predefined cluded. The variables are evaluated at runtime. macros in the C compiler. Additionally, it obtains imakemdep.h macros which control the build of some system dependent information at runtime. If imake itself are evaluated by including the header the target system is different from the build host, file imakemdep.h itself. The variables control: the C compiler of the target system should be used to preprocess imakemdep.h. A cross compile ¤ how the target preprocessor should be invoked scheme was designed which does exactly this. to generate the Makefile.
4.1 Prerequisites ¤ if the preprocessor output needs to be repro- cessed to be suitable for make.
The new cross compile environment makes the fol- ¤ which flags should be passed to the prepro- lowing assumptions: cessor. Since the target C compiler is used to preprocess this header file, these flags are ¤ The C cross compiler and all tools required to identical as if X was built on the target sys- create binary files are located in one directory tem. Therefore the configuration, definition under their normal names (e.g. if the compiler and rules files used to build the Makefile for the target system is gcc it should reside will see exactly the same macros as if the
under the same name in this directory. Makefiles were built on the target platform. ¤ If the default name C cross compiler is not ¤ information about the operating system name, “cc” a link exists from cc to the C cross com- version numbers and architecture. Since piler in this directory. imake can no longer obtain this information
at runtime by calling the uname() system ¤ The cross compile tools directory is not part of call, it must be obtained from the target sys- the binary search path. tem’s header files. This is very system de- pendent. Corresponding to the elements of the 4.2 Invoking a cross compile build utsname structure returned by uname() the macros: If the above prerequisites are met the cross compile build can simply be invoked by assigning the name – CROSS_UTS_SYSNAME, of the cross compile tools directory to the make – CROSS_UTS_RELEASE, variable CROSSCOMPILEDIR. Assuming the – CROSS_UTS_VERSION and name of this directory is
#define CROSS_UTS_SYSNAME "Linux" ¤ DefaultOsName, #include
For other platforms default values are assigned ¤ DefaultOSTeenyVersion and to these variables which may produce incor- rect results. ¤ DefaultToElfFormat.
¤ makedepend any defines required by . It should be noted that since imake contains in- formation about version numbers it may need re- Other information which imake usually compiling whenever a component of the cross com- collects at runtime is now obtained when pile suite is updated. Tools like makefontsdir imakemdep_cpp.h is created: or bdftopcf that process files during the build or installation are built for the target system. As those ¤ When using the GNU C compiler, the won’t run on the build system, they are not executed predefined version macros __GNUC__ and during the build or install step. This means that the __GNUC_MINOR__ are assigned to variables installation process will not be complete. Alterna- when imakemdep_cpp.h is created. tively, the user may add
¤ imake will execute the cross compiler to ob- #define UseInstalledOnCrossCompile YES tain the default include directories of the GNU C compiler. to hosts.def. This will cause the build system ¤ On a Linux system, the version of the C library to use the tools installed on the build host. These is collected from the system header files when tools need to be in the search path of the shell. imakemdep_cpp.h is created.
4.4 Limitations 5 Conclusions
On systems other than Linux, defini- The new cross compile environment was tested tions need to be added to imakemdep.c when building the x86-64 port of X. The operating for the macros CROSS_UTS_SYSNAME, system used on the build and the target machines CROSS_UTS_RELEASE, was Linux in both cases while the binary format CROSS_UTS_VERSION and differed—the build platform was an ia32 system. The macro UseInstalledOnCrossCompile was set to “YES” in hosts.def and the location of the cross compile tools were specified by the make variable CROSSCOMPILEDIR. Apart from this, no other variables were set. The system was installed into a separate directory tree. From there it was copied into the emulator’s file system. A test run of the Xserver as well as several clients was performed on the emulated target system. The Xserver and the clients worked flawlessly and all fonts were found. This indicated that the compo- nents were built and installed correctly, demonstrat- ing that the port, as well as the new cross compile suite, were working correctly.
References ¥¡ [AMD00] AMD Corporation. x86-64 Tech- nology White Paper. Technical re- port, AMD Corporation, One AMD Place, Sunnyvale, CA 94088, USA, August 2000.
[DuB93] Paul DuBois. Software Portability with imake. Nutshell Handbook Se- ries. O’Reilly & Associates Inc, July
1993. ISBN 1-56592-055-4. £ [MDHG 98] Peter S. Magnusson, Fredrik Dahlgren, Magnus Karlsson Hakan˚ Grahn, Fredrik Larsson, Fredrik Lundholm, Andreas Moest- edt, Jim Nilsson, Per Stenstrom,¨ and Bengt Werner. SimICS/sun4m: A Virtual Workstation. In Technical Conference Proceedings, New Orleans, LA, June 1998. USENIX.
[SG92] Robert W. Scheifler and James Get- tys. X Window System. Digital Press, third edition, 1992.
[VETT00] Gary V. Vaughan, Ben Elliston, Tom Tromey, and Ian Lance Taylor. GNU Autoconf, Automake and Libtool. New Riders, 2000. ISBN 1-57870- 190-2.