Native POSIX Threads Library (NPTL) Support for uClibc. Steven J. Hill Reality Diluted, Inc. [email protected] Abstract calls for applications to execute. Linux is for- tunate in that there are a number of C libraries available for varying platforms and environ- Linux continues to gain market share in embed- ments. Whether an embedded system, high- ded systems. As embedded processing power performance computing, or a home PC, there increases and more demanding applications in is a C library to fit each need. need of multi-threading capabilities are devel- oped, Native POSIX Threads Library (NPTL) The GNU C library, known also as glibc [1], support becomes crucial. The GNU C library and uClibc [2] are the most common Linux C li- [1] has had NPTL support for a number of years braries in use today. There are other C libraries on multiple processor architectures. However, like Newlib [3], diet libc [4], and klibc [5] used the GNU C library is more suited for work- in embedded systems and small root file sys- station and server platforms and not embed- tems. We list them only for completeness, yet ded systems due to its size. uClibc [2] is a they are not considered in this paper. Our focus POSIX-compliant C library designed for size will be solely on uClibc and glibc. and speed, but currently lacking NPTL sup- port. This paper will present the design and implementation of NPTL support in uClibc. In 2 Comparing C Libraries addition to the design overview, benchmarks, limitations and comparisons between glibc and uClibc will be discussed. NPTL for uClibc is To understand the need for NPTL in uClibc, we currently only supported for the MIPS proces- first examine the goals of both the uClibc and sor architecture. glibc projects. We will quickly examine the strengths and weaknesses of both C implemen- tations. It will then become evident why NPTL is needed in uClibc. 1 The Contenders 2.1 GNU C Library Project Goals Every usable Linux system has applications built atop a C library run-time environment. To quote from the main GNU C Library web The C library is at the core of user space and page [1], “The GNU C library is primarily de- provides all the necessary functions and system signed to be a portable and high performance C 410 • Native POSIX Threads Library (NPTL) Support for uClibc. library. It follows all relevant standards (ISO C The last two features in the table warrant ad- 99, POSIX.1c, POSIX.1j, POSIX.1d, Unix98, ditional explanation. glibc recently removed Single Unix Specification). It is also interna- linuxthreads support from its main de- tionalized and has one of the most complete in- velopment tree. It was moved into a sepa- ternationalization interfaces known.” In short, rate ports tree. It is maintained on a volun- glibc, aims to be the most complete C library teer basis only. uClibc will maintain both the implementation available. It succeeds, but at linuxthreads and nptl thread models ac- the cost of size and complexity. tively. Secondly, glibc only supports a couple of primary processor architectures. The rest of the architectures were also recently moved into 2.2 uClibc Project Goals the ports tree. uClibc continues to actively sup- port many more architectures by default. For embedded systems, uClibc is clearly the win- Let us see what uClibc has to offer. Again, ner. quoting from the main page for uClibc [2], “uClibc (a.k.a. µClibc, pronounced yew-see- lib-see) is a C library for developing embed- ded Linux systems. It is much smaller than 3 Why NPTL for Embedded Sys- the GNU C Library, but nearly all applica- tems? tions supported by glibc also work perfectly with uClibc. Porting applications from glibc to uClibc typically involves just recompiling uClibc supports multiple thread library the source code. uClibc even supports shared models. What are the shortcomings of libraries and threading. It currently runs on linuxthreads? Why is nptl better, or standard Linux and MMU-less (also known as worse? The answer lies in the requirements, µClinux) systems. ” Sounds great for embed- software and hardware, of the embedded ded systems development. Obviously, uClibc is platform being developed. We need to first going to be missing some features since its goal compare linuxthreads and nptl to is to be small in size. However, uClibc has its choose the thread library that best meets the own strengths as well. needs of our platform. Table 2 lists the key features the two thread libraries have to offer. 2.3 Comparing Features Using the table above, the nptl model is use- ful in systems that do not have severe memory constraints, but need threads to respond quickly Table 1 shows the important differentiating fea- and efficiently. The linuxthreads model is tures between glibc and uClibc. useful mostly for resource constrained systems still needing basic thread support. As men- It should be obvious from the table above that tioned at the beginning of this paper, embed- glibc certainly has better POSIX compliance, ded systems with faster processors and greater backwards binary compatibility, and network- memory resources are being required to do ing services support. uClibc shines in that it is more, with less. Using NPTL in conjunction much smaller (how much smaller will be cov- with the already small C library provided by ered later), more configurable, supports more uClibc, creates a perfectly balanced embedded processor architectures and is easier to build Linux system that has size, speed and an high- and maintain. performance multi-threading. 2006 Linux Symposium, Volume One • 411 Feature glibc uClibc LGPL Y Y Complete POSIX compliance Y N Binary compatibility across releases Y N NSS Support Y N NIS Support Y N Locale support Y Y Small disk storage footprint N Y Small runtime memory footprint N Y Supports MMU-less systems N Y Highly configurable N Y Simple build system N Y Built-in configuration system N Y Easily maintained N Y NPTL Support Y Y Linuxthreads Support N (See below) Y Support many processor architectures N (See below) Y Table 1: Library Feature Comparison Feature Description LinuxThreads NPTL Storage Size The actual amount of storage space consumed Smallest Largest in the file system by the libraries. The actual amount of RAM consumed at run- Memory Usage time by the thread library code and data. In- Smallest Largest cludes both kernel and user space memory us- age. Number of Threads The maximum number of threads available in a Hard Coded Value Dynamic process. Thread Efficiency Rate at which threads are created, destroyed, Slowest Fastest managed, and run. Per-Thread Signals Signals are handled on a per-thread basis and No Yes not the process. Inter-Thread Threads can share synchronization primitives Synchronization like mutexes and semaphores. No Yes POSIX.1 Compliance Thread library is compliant No Table 2: Thread Library Features 412 • Native POSIX Threads Library (NPTL) Support for uClibc. 4 uClibc NPTL Implementation loader must detect any TLS sections, allocate initial memory blocks for any needed TLS data, perform initial TLS relocations, and later per- The following sections outline the major tech- form additional TLS symbol look-ups and re- nical components of the NPTL implementation locations during the execution of the process. for uClibc. For the most part, they should apply It must also deal with the loading of shared equally to glibc’s implementation except where objects containing TLS data during program noted. References to additional papers and in- execution and properly allocate and relocate formation are provided should the reader wish its data. The TLS paper [6] provides ample to delve deeper into the inner workings of vari- overview of how these mechanisms work. The ous components. document does not, however, currently cover the specifics of MIPS-specific TLS storage and 4.1 TLS—The Foundation of NPTL implementation. MIPS TLS information is available from the Linux/MIPS website [9]. The first major component needed for NPTL on Adding TLS relocation support into uClibc’s Linux systems is Thread Local Storage (TLS). dynamic loader required close to 2200 lines Threads in a process share the same virtual ad- of code to change. The only functionality not dress space. Usually, any static or global data available in the uClibc loader is the handling of declared in the process is visible to all threads TLS variables in the dynamic loader itself. It within that process. TLS allows threads to have should also be noted that statically linked bi- their own local static and global data. An ex- naries using TLS/NPTL are not currently sup- cellent paper, written by Ulrich Drepper, cov- ported by uClibc. Static binaries will not be ers the technical details for implementing TLS supported until all processor architectures ca- for the ELF binary format [6]. Supporting TLS pable of supporting NPTL have working shared required extensive changes to binutils [7], GCC library support. In reality, shared library sup- [8] and glibc [1]. We cover the changes made port of NPTL is a prerequisite for debugging in the C library necessary to support TLS data. static NPTL support. Thank you to Daniel Ja- cobowitz for pointing this out. 4.1.1 The Dynamic Loader The dynamic loader, also affectionately known as ld.so, is responsible for the run-time link- 4.1.2 TLS Variables in uClibc ing of dynamically linked applications. The loader is the first piece of code to execute be- fore the main function of the application is There are four TLS variables currently used in called. It is responsible for loading and map- uClibc. The noticeable difference that can be ping in all required shared objects for the appli- observed in the source code is that they have cation.