DOCSLIB.ORG

Outline Object Files Unresolved References Object File Structure

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Outline Object Files Unresolved References Object File Structure Outline • Static linking CS 4120 – Object files – Libraries Introduction to Compilers – Shared libraries Ross Tate – Relocatable code Cornell University • Dynamic linking – explicit vs. implicit linking Lecture 37: Linking and Loading – dynamically linked libraries/dynamic shared objects 2 Object files Unresolved references • Output of compiler is an object file source extern int abs( int x ); – not executable source code source code code … y = y + abs(x); – may refer to external compiler symbols (variables, functions, etc.) whose object code object code assembly PUSH ecx definition is not known. code CALL _abs • Linker joins together linker ADD ebx, eax object files, resolves 51 external references executable object to be filled in 9A 00 00 00 00 image code by linker 03 D8 3 4 Object file structure Action of Linker object files executable image memory layout • Object file contains text1 file header various sections init1 uninitialized text section: • text section contains the sym1 data compiled code with some data unresolved machine code rel1 text3 patching needed init3 segment initialized data • For uninitialized data, init3 init2 only need to know total sym3 init1 symbol table size of data segment (maps identifiers to text2 rel3 text3 code • Describes structure of text machine-code locations) text2 segment and data sections init2 text1 relocation info • Points to places in text sym2 and data section that need rel2 fix-up 5 6 1 Executable file structure Executing programs • Same as object file, but • Multiple copies of program share code (text), code is ready to be have own data executed as-is file header • Data appears at same virtual address in every text section: execution- • Pages of code and data process brought in lazily from ready machine code physical virtual text and data section as initialized data notepad data 3 notepad data 3 needed: rapid start-up notepad code optional: symbol table heap data • Text section shared notepad data 2 notepad data 2 static data across processes notepad code code notepad data 1 • Symbols allow debugging stack data notepad data 1 notepad code notepad code 7 8 Libraries Shared libraries • Library : collection of object files • Problem: libraries take up a lot of memory when • Linker adds all object files necessary to linked into many running applications resolve undefined references in explicitly • Solution: shared libraries (e.g. DLLs) named files Physical memory • Object files, libraries searched in user- ls libc specified order for external references cat libc libc X11 Unix: ld main.o foo.o /usr/lib/X11.a /usr/lib/libc.a emacs libc NT: link main.obj foo.obj kernel32.lib user32.lib … X11 • Index over all object files in library for xterm libc rapid searching X11 9 10 Step 1: Jump tables Global tables • Executable file refers to, does not contain library • Problem: shared libraries may depend on code; library code loaded dynamically external symbols (even symbols within the • Library code found in separate shared library shared library); different applications may have file (similar to DLL); linking done against different linkage: import library that does not contain code ld -o prog1 main.o /usr/lib/libc.a • Library compiled at fixed address, starts with ld -o prog2 main.o mymalloc.o /usr/lib/libc.a jump table to allow new versions; client code • If routine in libc.a calls malloc(), for prog1 jumps to jump table (indirection). should get standard version; for prog2, version program: library: in mymalloc.o scanf: jmp real_scanf • Calls to external symbols are made through call printf printf: jmp real_printf global tables unique to each program putc: jmp real_putc 11 12 2 Global tables Using global tables prog1 prog2 Data segment: • Global table contains entries for all external references main.o Global table main.o malloc(n) push [ebp + n] … mov eax, [malloc_entry] malloc() malloc_entry: malloc() call eax Shared lib (libc) • Non-shared application code unaffected printf: jmp … • Same-object references can still be used directly malloc: jmp … mymalloc.o: • Global-table entries (malloc_entry) placed in non-shared memory locations so each program has different linkage real_printf: malloc() • Initialized by dynamic loader when program begins: reads symbol tables, relocation info real_malloc: 13 14 Relocation Dynamic shared objects • Before widespread support for virtual memory, • Unix systems: Code is typically compiled code had to be relocatable (could not contain as a dynamic shared object (DSO): fixed memory addresses) location-independent shared library • With virtual memory, all programs could start at same address, could contain fixed addresses • Shared libraries can be mapped to any address in virtual memory—no copying! • Problem with shared libraries (e.g., DLLs): if allocated at fixed addresses, can collide in • Questions: virtual memory (code, data, global tables, …) – how to make code completely relocatable? – Collision code copied and explicitly relocated – what is the performance impact? • Back to relocatable code! 15 16 Location Independence Global tables • Can’t use absolute addresses (directly named • Can put address of all globals into global table memory locations) anywhere: • But…can’t put the global table at a fixed address: not location independent! – Not in calls to external functions – Not for global variables in data segment • Three solutions: – Not even for global table entries 1. Pass global-table address as an extra argument push [ebp + n] (possibly in a register) : affects first-class functions mov eax, [malloc_entry] ; Oops! (next global-table address stored in current GT) call eax 2. Use address arithmetic on current program counter (eip register) to find global table. Use link-time • Not a problem: branch instructions, local calls. constant offset between eip and global table. Use relative addressing 3. Stick global-table entries into the current object’s vtable : V-Table is the global table (only works for methods, but otherwise the best) 17 18 3 Cost of DSOs Link-time optimization • Assume esi contains global table pointer (set-up • When linking object files, linker provides code at beginning of function) flags to allow peephole optimization of • Call to function f: inter-module references call [esi + f_offset] • Global variable accesses: • Unix: –non_shared link option means mov eax, [esi + v_offset] application to get its own copy of library mov ebx, [eax] code • Calling extern functions calling methods – calls and global variables performed directly • Accessing extern variables is more expensive (peephole opt.) than accessing local variables call [esi + malloc_addr] call malloc • Most computer benchmarks run w/o DSOs! • Allows performance/functionality trade-off 19 20 Dynamic linking Conclusions • Shared libraries (DLLs) and DSOs can be linked • Shared libraries and DSOs allow efficient dynamically into a running program memory use on a machine running many • Normal case: implicit linking. When setting up global different programs that share code tables, shared libraries are automatically loaded if necessary (even lazily), symbols looked up & global • Improves cache, TLB performance overall tables created. • Hurts individual program performance by • Explicit dynamic linking: application can choose how adding indirections through global tables, to extend its own functionality bloating code with extra instructions – Unix: h = dlopen(filename) loads an object file • Important new functionality: dynamic extension into some free memory (if necessary), allows query of program of globals: p = dlsym(h, name) – Windows: h = LoadLibrary(filename), • Peephole linker optimization can restore p = GetProcAddress(h, name) performance, but with loss of functionality 21 22 4 .
Load more

Recommended publications
  • Executable Code Is Not the Proper Subject of Copyright Law a Retrospective Criticism of Technical and Legal Naivete in the Apple V
    Executable Code is Not the Proper Subject of Copyright Law A retrospective criticism of technical and legal naivete in the Apple V. Franklin case Matthew M. Swann, Clark S. Turner, Ph.D., Department of Computer Science Cal Poly State University November 18, 2004 Abstract: Copyright was created by government for a purpose. Its purpose was to be an incentive to produce and disseminate new and useful knowledge to society. Source code is written to express its underlying ideas and is clearly included as a copyrightable artifact. However, since Apple v. Franklin, copyright has been extended to protect an opaque software executable that does not express its underlying ideas. Common commercial practice involves keeping the source code secret, hiding any innovative ideas expressed there, while copyrighting the executable, where the underlying ideas are not exposed. By examining copyright’s historical heritage we can determine whether software copyright for an opaque artifact upholds the bargain between authors and society as intended by our Founding Fathers. This paper first describes the origins of copyright, the nature of software, and the unique problems involved. It then determines whether current copyright protection for the opaque executable realizes the economic model underpinning copyright law. Having found the current legal interpretation insufficient to protect software without compromising its principles, we suggest new legislation which would respect the philosophy on which copyright in this nation was founded. Table of Contents INTRODUCTION................................................................................................. 1 THE ORIGIN OF COPYRIGHT ........................................................................... 1 The Idea is Born 1 A New Beginning 2 The Social Bargain 3 Copyright and the Constitution 4 THE BASICS OF SOFTWARE ..........................................................................
    [Show full text]
  • The LLVM Instruction Set and Compilation Strategy
    The LLVM Instruction Set and Compilation Strategy Chris Lattner Vikram Adve University of Illinois at Urbana-Champaign lattner,vadve ¡ @cs.uiuc.edu Abstract This document introduces the LLVM compiler infrastructure and instruction set, a simple approach that enables sophisticated code transformations at link time, runtime, and in the field. It is a pragmatic approach to compilation, interfering with programmers and tools as little as possible, while still retaining extensive high-level information from source-level compilers for later stages of an application’s lifetime. We describe the LLVM instruction set, the design of the LLVM system, and some of its key components. 1 Introduction Modern programming languages and software practices aim to support more reliable, flexible, and powerful software applications, increase programmer productivity, and provide higher level semantic information to the compiler. Un- fortunately, traditional approaches to compilation either fail to extract sufficient performance from the program (by not using interprocedural analysis or profile information) or interfere with the build process substantially (by requiring build scripts to be modified for either profiling or interprocedural optimization). Furthermore, they do not support optimization either at runtime or after an application has been installed at an end-user’s site, when the most relevant information about actual usage patterns would be available. The LLVM Compilation Strategy is designed to enable effective multi-stage optimization (at compile-time, link-time, runtime, and offline) and more effective profile-driven optimization, and to do so without changes to the traditional build process or programmer intervention. LLVM (Low Level Virtual Machine) is a compilation strategy that uses a low-level virtual instruction set with rich type information as a common code representation for all phases of compilation.
    [Show full text]
  • The “Stabs” Debug Format
    The \stabs" debug format Julia Menapace, Jim Kingdon, David MacKenzie Cygnus Support Cygnus Support Revision TEXinfo 2017-08-23.19 Copyright c 1992{2021 Free Software Foundation, Inc. Contributed by Cygnus Support. Written by Julia Menapace, Jim Kingdon, and David MacKenzie. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled \GNU Free Documentation License". i Table of Contents 1 Overview of Stabs ::::::::::::::::::::::::::::::: 1 1.1 Overview of Debugging Information Flow ::::::::::::::::::::::: 1 1.2 Overview of Stab Format ::::::::::::::::::::::::::::::::::::::: 1 1.3 The String Field :::::::::::::::::::::::::::::::::::::::::::::::: 2 1.4 A Simple Example in C Source ::::::::::::::::::::::::::::::::: 3 1.5 The Simple Example at the Assembly Level ::::::::::::::::::::: 4 2 Encoding the Structure of the Program ::::::: 7 2.1 Main Program :::::::::::::::::::::::::::::::::::::::::::::::::: 7 2.2 Paths and Names of the Source Files :::::::::::::::::::::::::::: 7 2.3 Names of Include Files:::::::::::::::::::::::::::::::::::::::::: 8 2.4 Line Numbers :::::::::::::::::::::::::::::::::::::::::::::::::: 9 2.5 Procedures ::::::::::::::::::::::::::::::::::::::::::::::::::::: 9 2.6 Nested Procedures::::::::::::::::::::::::::::::::::::::::::::: 11 2.7 Block Structure
    [Show full text]
  • Architectural Support for Scripting Languages
    Architectural Support for Scripting Languages By Dibakar Gope A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy (Electrical and Computer Engineering) at the UNIVERSITY OF WISCONSIN–MADISON 2017 Date of final oral examination: 6/7/2017 The dissertation is approved by the following members of the Final Oral Committee: Mikko H. Lipasti, Professor, Electrical and Computer Engineering Gurindar S. Sohi, Professor, Computer Sciences Parameswaran Ramanathan, Professor, Electrical and Computer Engineering Jing Li, Assistant Professor, Electrical and Computer Engineering Aws Albarghouthi, Assistant Professor, Computer Sciences © Copyright by Dibakar Gope 2017 All Rights Reserved i This thesis is dedicated to my parents, Monoranjan Gope and Sati Gope. ii acknowledgments First and foremost, I would like to thank my parents, Sri Monoranjan Gope, and Smt. Sati Gope for their unwavering support and encouragement throughout my doctoral studies which I believe to be the single most important contribution towards achieving my goal of receiving a Ph.D. Second, I would like to express my deepest gratitude to my advisor Prof. Mikko Lipasti for his mentorship and continuous support throughout the course of my graduate studies. I am extremely grateful to him for guiding me with such dedication and consideration and never failing to pay attention to any details of my work. His insights, encouragement, and overall optimism have been instrumental in organizing my otherwise vague ideas into some meaningful contributions in this thesis. This thesis would never have been accomplished without his technical and editorial advice. I find myself fortunate to have met and had the opportunity to work with such an all-around nice person in addition to being a great professor.
    [Show full text]
  • Portable Executable File Format
    Chapter 11 Portable Executable File Format IN THIS CHAPTER + Understanding the structure of a PE file + Talking in terms of RVAs + Detailing the PE format + The importance of indices in the data directory + How the loader interprets a PE file MICROSOFT INTRODUCED A NEW executable file format with Windows NT. This for- mat is called the Portable Executable (PE) format because it is supposed to be portable across all 32-bit operating systems by Microsoft. The same PE format exe- cutable can be executed on any version of Windows NT, Windows 95, and Win32s. Also, the same format is used for executables for Windows NT running on proces- sors other than Intel x86, such as MIPS, Alpha, and Power PC. The 32-bit DLLs and Windows NT device drivers also follow the same PE format. It is helpful to understand the PE file format because PE files are almost identi- cal on disk and in RAM. Learning about the PE format is also helpful for under- standing many operating system concepts. For example, how operating system loader works to support dynamic linking of DLL functions, the data structures in- volved in dynamic linking such as import table, export table, and so on. The PE format is not really undocumented. The WINNT.H file has several struc- ture definitions representing the PE format. The Microsoft Developer's Network (MSDN) CD-ROMs contain several descriptions of the PE format. However, these descriptions are in bits and pieces, and are by no means complete. In this chapter, we try to give you a comprehensive picture of the PE format.
    [Show full text]
  • Common Object File Format (COFF)
    Application Report SPRAAO8–April 2009 Common Object File Format ..................................................................................................................................................... ABSTRACT The assembler and link step create object files in common object file format (COFF). COFF is an implementation of an object file format of the same name that was developed by AT&T for use on UNIX-based systems. This format encourages modular programming and provides powerful and flexible methods for managing code segments and target system memory. This appendix contains technical details about the Texas Instruments COFF object file structure. Much of this information pertains to the symbolic debugging information that is produced by the C compiler. The purpose of this application note is to provide supplementary information on the internal format of COFF object files. Topic .................................................................................................. Page 1 COFF File Structure .................................................................... 2 2 File Header Structure .................................................................. 4 3 Optional File Header Format ........................................................ 5 4 Section Header Structure............................................................. 5 5 Structuring Relocation Information ............................................... 7 6 Symbol Table Structure and Content........................................... 11 SPRAAO8–April 2009
    [Show full text]
  • CS429: Computer Organization and Architecture Linking I & II
    CS429: Computer Organization and Architecture Linking I & II Dr. Bill Young Department of Computer Sciences University of Texas at Austin Last updated: April 5, 2018 at 09:23 CS429 Slideset 23: 1 Linking I A Simplistic Translation Scheme m.c ASCII source file Problems: Efficiency: small change Compiler requires complete re-compilation. m.s Modularity: hard to share common functions (e.g., printf). Assembler Solution: Static linker (or Binary executable object file linker). p (memory image on disk) CS429 Slideset 23: 2 Linking I Better Scheme Using a Linker Linking is the process of m.c a.c ASCII source files combining various pieces of code and data into a Compiler Compiler single file that can be loaded (copied) into m.s a.s memory and executed. Linking could happen at: Assembler Assembler compile time; Separately compiled m.o a.o load time; relocatable object files run time. Linker (ld) Must somehow tell a Executable object file module about symbols p (code and data for all functions defined in m.c and a.c) from other modules. CS429 Slideset 23: 3 Linking I Linking A linker takes representations of separate program modules and combines them into a single executable. This involves two primary steps: 1 Symbol resolution: associate each symbol reference throughout the set of modules with a single symbol definition. 2 Relocation: associate a memory location with each symbol definition, and modify each reference to point to that location. CS429 Slideset 23: 4 Linking I Translating the Example Program A compiler driver coordinates all steps in the translation and linking process.
    [Show full text]
  • CDC Build System Guide
    CDC Build System Guide Java™ Platform, Micro Edition Connected Device Configuration, Version 1.1.2 Foundation Profile, Version 1.1.2 Optimized Implementation Sun Microsystems, Inc. www.sun.com December 2008 Copyright © 2008 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, U.S.A. All rights reserved. Sun Microsystems, Inc. has intellectual property rights relating to technology embodied in the product that is described in this document. In particular, and without limitation, these intellectual property rights may include one or more of the U.S. patents listed at http://www.sun.com/patents and one or more additional patents or pending patent applications in the U.S. and in other countries. U.S. Government Rights - Commercial software. Government users are subject to the Sun Microsystems, Inc. standard license agreement and applicable provisions of the FAR and its supplements. This distribution may include materials developed by third parties. Parts of the product may be derived from Berkeley BSD systems, licensed from the University of California. UNIX is a registered trademark in the U.S. and in other countries, exclusively licensed through X/Open Company, Ltd. Sun, Sun Microsystems, the Sun logo, Java, Solaris and HotSpot are trademarks or registered trademarks of Sun Microsystems, Inc. or its subsidiaries in the United States and other countries. The Adobe logo is a registered trademark of Adobe Systems, Incorporated. Products covered by and information contained in this service manual are controlled by U.S. Export Control laws and may be subject to the export or import laws in other countries. Nuclear, missile, chemical biological weapons or nuclear maritime end uses or end users, whether direct or indirect, are strictly prohibited.
    [Show full text]
  • Chapter 6: the Linker
    6. The Linker 6-1 Chapter 6: The Linker References: • Brian W. Kernighan / Dennis M. Ritchie: The C Programming Language, 2nd Ed. Prentice-Hall, 1988. • Samuel P. Harbison / Guy L. Steele Jr.: C — A Reference Manual, 4th Ed. Prentice-Hall, 1995. • Online Documentation of Microsoft Visual C++ 6.0 (Standard Edition): MSDN Library: Visual Studio 6.0 release. • Horst Wettstein: Assemblierer und Binder (in German). Carl Hanser Verlag, 1979. • Peter Rechenberg, Gustav Pomberger (Eds.): Informatik-Handbuch (in German). Carl Hanser Verlag, 1997. Kapitel 12: Systemsoftware (H. M¨ossenb¨ock). Stefan Brass: Computer Science III Universit¨atGiessen, 2001 6. The Linker 6-2 Overview ' $ 1. Introduction (Overview) & % 2. Object Files, Libraries, and the Linker 3. Make 4. Dynamic Linking Stefan Brass: Computer Science III Universit¨atGiessen, 2001 6. The Linker 6-3 Introduction (1) • Often, a program consists of several modules which are separately compiled. Reasons are: The program is large. Even with fast computers, editing and compiling a single file with a million lines leads to unnecessary delays. The program is developed by several people. Different programmers cannot easily edit the same file at the same time. (There is software for collaborative work that permits that, but it is still a research topic.) A large program is easier to understand if it is divided into natural units. E.g. each module defines one data type with its operations. Stefan Brass: Computer Science III Universit¨atGiessen, 2001 6. The Linker 6-4 Introduction (2) • Reasons for splitting a program into several source files (continued): The same module might be used in different pro- grams (e.g.
    [Show full text]
  • Using Ld the GNU Linker
    Using ld The GNU linker ld version 2 January 1994 Steve Chamberlain Cygnus Support Cygnus Support [email protected], [email protected] Using LD, the GNU linker Edited by Jeffrey Osier (jeff[email protected]) Copyright c 1991, 92, 93, 94, 95, 96, 97, 1998 Free Software Foundation, Inc. Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies. Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying, provided also that the entire resulting derived work is distributed under the terms of a permission notice identical to this one. Permission is granted to copy and distribute translations of this manual into another lan- guage, under the above conditions for modified versions. Chapter 1: Overview 1 1 Overview ld combines a number of object and archive files, relocates their data and ties up symbol references. Usually the last step in compiling a program is to run ld. ld accepts Linker Command Language files written in a superset of AT&T’s Link Editor Command Language syntax, to provide explicit and total control over the linking process. This version of ld uses the general purpose BFD libraries to operate on object files. This allows ld to read, combine, and write object files in many different formats—for example, COFF or a.out. Different formats may be linked together to produce any available kind of object file. See Chapter 5 [BFD], page 47, for more information. Aside from its flexibility, the gnu linker is more helpful than other linkers in providing diagnostic information.
    [Show full text]
  • Linking Basics.Docx Page 1 of 35 Initial Creation: March, 24, 2015 Date Updated: October 4, 2015
    An Overview of Object Files And Linking Harry H. Porter III Portland State University cs.pdx.edu/~harry April 9, 2015 Goals of this Paper Audience: CS-201 students Coverage: Compiling Linking Object files External Symbols Relocatable Code Segments (.text, .data., .bss) Filename: Linking Basics.docx Page 1 of 35 Initial Creation: March, 24, 2015 Date Updated: October 4, 2015 Overview In order to create an executable file, a program must be compiled and linked. In this paper, I’ll use the “C” programming language as an example, but this discussion applies to other compiled languages like C++. Languages that are interpreted (like Java or Python) do things differently and linking does not apply to them. We’ll also discuss the Linux/Unix system, but other OSes are similar. A program begins as a human readable source text file, such as “hello.c”. The program must first be compiled and this step produces a human readable text file in assembly code. Then the assembly code version is assembled and this produces an object file. Finally, the object file is linked and the executable file is produced. At some later time, the OS will load the executable file into memory run it. Many programs are large and these programs are broken into several “.c” files. We’ll look at an example involving two files, “hello.c” and “there.c”. Each of the .c files must be compiled and assembled, but these steps can be done independently. In other words, we can compile and assemble hello.c before we even create the there.c file.
    [Show full text]
  • Chapter 13 ARM Image Format
    Chapter 13 ARM Image Format This chapter describes the ARM Image Format (AIF). It contains the following sections: • Overview of the ARM Image Format on page 13-2 • AIF variants on page 13-3 • The layout of AIF on page 13-4. ARM DUI 0041C Copyright © 1997 and 1998 ARM Limited. All rights reserved. 13-1 ARM Image Format 13.1 Overview of the ARM Image Format ARM Image Format (AIF) is a simple format for ARM executable images, consisting of: • a 128-byte header • the image code • the image initialized static data. An AIF image is capable of self-relocation if it is created with the appropriate linker options. The image can be loaded anywhere and it will execute where it is loaded. After an AIF image has been relocated, it can create its own zero-initialized area. Finally, the image is entered at the unique entry point. 13-2 Copyright © 1997 and 1998 ARM Limited. All rights reserved. ARM DUI 0041C ARM Image Format 13.2 AIF variants There are three variants of AIF: Executable AIF Executable AIF can be loaded at its load address and entered at the same point (at the first word of the AIF header). It prepares itself for execution by relocating itself if required and setting to zero its own zero-initialized data. The header is part of the image itself. Code in the header ensures that the image is properly prepared for execution before being entered at its entry address. The fourth word of an executable AIF header is: BL entrypoint The most significant byte of this word (in the target byte order) is 0xeb.
    [Show full text]