Outline • Static linking CS 4120 – Object files – Libraries Introduction to Compilers – Shared libraries Ross Tate – Relocatable code Cornell University • Dynamic linking – explicit vs. implicit linking Lecture 37: Linking and Loading – dynamically linked libraries/dynamic shared objects 2 Object files Unresolved references • Output of compiler is an object file source extern int abs( int x ); – not executable source code source code code … y = y + abs(x); – may refer to external compiler symbols (variables, functions, etc.) whose object code object code assembly PUSH ecx definition is not known. code CALL _abs • Linker joins together linker ADD ebx, eax object files, resolves 51 external references executable object to be filled in 9A 00 00 00 00 image code by linker 03 D8 3 4 Object file structure Action of Linker object files executable image memory layout • Object file contains text1 file header various sections init1 uninitialized text section: • text section contains the sym1 data compiled code with some data unresolved machine code rel1 text3 patching needed init3 segment initialized data • For uninitialized data, init3 init2 only need to know total sym3 init1 symbol table size of data segment (maps identifiers to text2 rel3 text3 code • Describes structure of text machine-code locations) text2 segment and data sections init2 text1 relocation info • Points to places in text sym2 and data section that need rel2 fix-up 5 6 1 Executable file structure Executing programs • Same as object file, but • Multiple copies of program share code (text), code is ready to be have own data executed as-is file header • Data appears at same virtual address in every text section: execution- • Pages of code and data process brought in lazily from ready machine code physical virtual text and data section as initialized data notepad data 3 notepad data 3 needed: rapid start-up notepad code optional: symbol table heap data • Text section shared notepad data 2 notepad data 2 static data across processes notepad code code notepad data 1 • Symbols allow debugging stack data notepad data 1 notepad code notepad code 7 8 Libraries Shared libraries • Library : collection of object files • Problem: libraries take up a lot of memory when • Linker adds all object files necessary to linked into many running applications resolve undefined references in explicitly • Solution: shared libraries (e.g. DLLs) named files Physical memory • Object files, libraries searched in user- ls libc specified order for external references cat libc libc X11 Unix: ld main.o foo.o /usr/lib/X11.a /usr/lib/libc.a emacs libc NT: link main.obj foo.obj kernel32.lib user32.lib … X11 • Index over all object files in library for xterm libc rapid searching X11 9 10 Step 1: Jump tables Global tables • Executable file refers to, does not contain library • Problem: shared libraries may depend on code; library code loaded dynamically external symbols (even symbols within the • Library code found in separate shared library shared library); different applications may have file (similar to DLL); linking done against different linkage: import library that does not contain code ld -o prog1 main.o /usr/lib/libc.a • Library compiled at fixed address, starts with ld -o prog2 main.o mymalloc.o /usr/lib/libc.a jump table to allow new versions; client code • If routine in libc.a calls malloc(), for prog1 jumps to jump table (indirection). should get standard version; for prog2, version program: library: in mymalloc.o scanf: jmp real_scanf • Calls to external symbols are made through call printf printf: jmp real_printf global tables unique to each program putc: jmp real_putc 11 12 2 Global tables Using global tables prog1 prog2 Data segment: • Global table contains entries for all external references main.o Global table main.o malloc(n) push [ebp + n] … mov eax, [malloc_entry] malloc() malloc_entry: malloc() call eax Shared lib (libc) • Non-shared application code unaffected printf: jmp … • Same-object references can still be used directly malloc: jmp … mymalloc.o: • Global-table entries (malloc_entry) placed in non-shared memory locations so each program has different linkage real_printf: malloc() • Initialized by dynamic loader when program begins: reads symbol tables, relocation info real_malloc: 13 14 Relocation Dynamic shared objects • Before widespread support for virtual memory, • Unix systems: Code is typically compiled code had to be relocatable (could not contain as a dynamic shared object (DSO): fixed memory addresses) location-independent shared library • With virtual memory, all programs could start at same address, could contain fixed addresses • Shared libraries can be mapped to any address in virtual memory—no copying! • Problem with shared libraries (e.g., DLLs): if allocated at fixed addresses, can collide in • Questions: virtual memory (code, data, global tables, …) – how to make code completely relocatable? – Collision code copied and explicitly relocated – what is the performance impact? • Back to relocatable code! 15 16 Location Independence Global tables • Can’t use absolute addresses (directly named • Can put address of all globals into global table memory locations) anywhere: • But…can’t put the global table at a fixed address: not location independent! – Not in calls to external functions – Not for global variables in data segment • Three solutions: – Not even for global table entries 1. Pass global-table address as an extra argument push [ebp + n] (possibly in a register) : affects first-class functions mov eax, [malloc_entry] ; Oops! (next global-table address stored in current GT) call eax 2. Use address arithmetic on current program counter (eip register) to find global table. Use link-time • Not a problem: branch instructions, local calls. constant offset between eip and global table. Use relative addressing 3. Stick global-table entries into the current object’s vtable : V-Table is the global table (only works for methods, but otherwise the best) 17 18 3 Cost of DSOs Link-time optimization • Assume esi contains global table pointer (set-up • When linking object files, linker provides code at beginning of function) flags to allow peephole optimization of • Call to function f: inter-module references call [esi + f_offset] • Global variable accesses: • Unix: –non_shared link option means mov eax, [esi + v_offset] application to get its own copy of library mov ebx, [eax] code • Calling extern functions calling methods – calls and global variables performed directly • Accessing extern variables is more expensive (peephole opt.) than accessing local variables call [esi + malloc_addr] call malloc • Most computer benchmarks run w/o DSOs! • Allows performance/functionality trade-off 19 20 Dynamic linking Conclusions • Shared libraries (DLLs) and DSOs can be linked • Shared libraries and DSOs allow efficient dynamically into a running program memory use on a machine running many • Normal case: implicit linking. When setting up global different programs that share code tables, shared libraries are automatically loaded if necessary (even lazily), symbols looked up & global • Improves cache, TLB performance overall tables created. • Hurts individual program performance by • Explicit dynamic linking: application can choose how adding indirections through global tables, to extend its own functionality bloating code with extra instructions – Unix: h = dlopen(filename) loads an object file • Important new functionality: dynamic extension into some free memory (if necessary), allows query of program of globals: p = dlsym(h, name) – Windows: h = LoadLibrary(filename), • Peephole linker optimization can restore p = GetProcAddress(h, name) performance, but with loss of functionality 21 22 4 .