Prelinker Usage for MIPS® Cores CELF Jamboree #11 Tokyo, Japan, Oct 27, 2006

Home , Loader (computing), Prelink

At the core of the user experience.®

Arvind Kumar, MIPS Technologies Kazu Hirata, CodeSourcery

Î What is the prelinker?

Î Object formats, loaders & issues

Î Driving the prelinker

Î Under the hood

Î Speedup

Î Prelinker is a tool to make programs start faster

Î Processes libraries & executable Î A new step in system building

Î Best when whole system is prelinked

Î Supports MIPS Î Linker, prelinker & dynamic linker changes

Î Works in cross environment Î Original prelinker only ran on the system being prelinked Î Modified to allow prelinking of a cross built root file system Î No longer need a live target system

Î Unspecified load address Î requires relocations

Î Symbolic resolution Î requires symbol table management

Î Allocate space in process's address space

Î Load library

Î Relocate the Global Offset Table

Î Apply relocations from .rel.dyn

Î Allocate address at link time Î Requires universal table of library addresses

Î No dynamic relocations Î No comdat or weak symbols Î No overriding by executable

Î Symbol resolution takes time

Î Relocation causes pages to be unshared Î GOT is conceptually constant

Î Many more shared libraries used now

Î Global table of library addresses is unmanageable

Î The flexibility of ELF

Î The speed of a.out

Î Fails gracefully

Î Postprocesses linked executables & libraries Î Preprocesses dynamic loaded images

Î Prelink all the libraries & executables in these directories

Î Prelink this set of libraries to be coresident

Î Prelink this executable against these libraries

Î Checks consistency of dependent libraries Î ensures none have changed since executable was prelinked

Î If all is well then loads the libraries & executables without needing dynamic linking

Î Otherwise falls back to old behaviour Î Graceful degradation

Î Cannot automatically prelink dlopen'ed libraries Î can manually prelink them

Î Requires updated linker and dynamic loader Î Libraries produced by older tools will force full linking

Î Scans libraries and executables

Î Builds dependency tree

Î Determines which libraries are coresident

Î Allocates base addresses

Î Prelinks the libraries & executables

Î Adjust every local GOT entry

Î Resolve global GOT entries Î Some entries cannot be prelinked

Î Apply relocations according to assigned base addresses Î Relocations to other libraries Î Relocations within library

Î Insert some ident information

Î Apply relocations Î Relocations to libraries

Î Resolve global GOT entries for externaly defined symbols

Î Insert ident information

Î Create conflict list Î Executable can override symbols in libraries Î Global GOT entries that couldn't be prelinked Î TLS relocations

Î DT_GNU_PRELINKED provides a timestamp

Î DT_CHECKSUM provides a checksum

Î .gnu.liblist list of libraries needed

Î .gnu.libstr names of libraries

Î dynamic loader checks Î each library's DT_GNU_PRELINKED & DT_CHECKSUM against .gnu.liblist

Î Sometimes SHT_REL must be converted to SHT_RELA Î Need correct addend when reprelinking the library or when prelink checks fail Î Some other (changed) library may override or stop overriding the base symbol Î So we can't 'undo' the REL relocation to extract the addend Î We must explicitly store the addend somewhere Î Or use a relocation type that has no addend

Î An executable can override a symbol in a shared library Î Must update shared library when loaded Î Prelinker generates such a list for each executable Î Places it in .gnu.conflicts section Î Dynamic loader iterates over the list Î Each conflict causes that page to be duplicated (copy on write) Î If there are no conflicts, the are no page copies

Î Improperly linked shared libraries Î prelinker will warn

Î Library (or executable) overriding a symbol Î eg libpthread.so overrides libc.so

Î COMDAT Î symbol occurs in multiple places, must pick one

Î Library's GOT can point to a lazy binding stub

Î Thread Local Storage

Î C++ code can have many COMDAT instances

Î Some are generated by compiler Î vtable, rtti info

Î These do not have to be commonized

Î Prelinker will not emit conflicts for these

Î Prelinker has an undo operation

Î .gnu.prelink_undo section contains Î original ELF, program & section headers

Î Can unprelink specific executables & libraries Î or unprelink everything found in a search path

Î Prelinking increases the on-disk image Î undo info, REL->RELA conversion, conflict list

Î A test system of 422 executables and 38 libraries went from 86.8MB to 88.5MB (2%) Î For libc only, 1749K to 1752K (0.2%) Î For libstdc++ only, 1413K to 1426K (0.9%) Î libc avoided the REL->RELA conversion, but libstdc++ did not.

Î Speedup is noticeable in two places Î large gui applications with lots of shared libraries Î shell scripts rapidly invoking many worker tools

Î Test machine MIPS64® 20Kc™ core @ 500MHz

Î Test machine MIPS64® 20Kc™ core @500MHz

Î /bin/bash -c /bin/true (300 iterations) Î 7.4s to 4.6s ~38% improvement

Î /bin/bash -c /bin/ls >/dev/null (300 iterations) Î 10.5s to 7.0s ~33% improvement

Î firefox (100 iterations, no X server) Î 30.9 to 16.7 ~45% improvement

Î Test machine MIPS32® 24Kc™ core @266MHz

Î /bin/bash -c /bin/true (300 iterations) Î 8.9s to 4.9s ~45% improvement

Î /bin/bash -c /bin/ls >/dev/null (300 iterations) Î 12.3s to 7.2s ~42% improvement

Î firefox (100 iterations, no X server) Î 43.9 to 24.4 ~44% improvement

Î Binutils FSF mainline in progress Î sourceware.org

Î GLIBC FSF mainline in progress Î sourceware.org/glibc

Î Prelinker in progress Î svn://sourceware.org/svn/prelink

Î None are yet in official releases