DOCSLIB.ORG

Outline Executable/Object File Formats Brief History of Binary File Formats

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Outline Executable/Object File Formats Brief History of Binary File Formats Outline CSci 5980/8980 ELF basics Manual and Automated Binary Reverse Engineering Slides 5: The ELF Binary File Format Stephen McCamant Static and dynamic linking University of Minnesota Executable/object file formats Brief history of binary file formats (Unix) Modern systems usually use a common format for Early Unix had a simple a.out format relocatable object files during compilation and final Lasted until early days of free Linux/BSD, now obsolete executables AT&T’s second try was named COFF Mostly binary data representing code and data Still limited, but widely adopted with changes Plus metadata allowing the data to be linked and AT&T’s third try was ELF, now used in almost all Unix loaded systems Brief history of binary file formats (non-Unix) Compile-time and run-time Early DOS and Windows had several limited formats Some file features are used during compilation Since the 32-bit era, Windows uses the PE (Portable Typically first created by assembler, then used/modified Executable) format by the linker Partially derived from COFF Other features are used when the program runs OS X era Apple (including iOS, etc) uses a format By the OS when the program starts named Mach-O And now also by runtime linking First developed for the Mach microkernel used on the NeXT Static and dynamic/shared linking ELF Traditional “static” linking happens all at compile time Executable (or Extensible) and Linking (or Linkable) Libraries become indistinguishable from the rest of the Format program First appeared in System V Release 4 Unix, c. 1989 For efficiency and flexibility, it is now more common to postpone library linking until runtime Linux switched to ELF c. 1995 At startup, or later in execution In part because they’d chosen a hard-to-use approach to Library code stays separate, so its memory can be a.out shared libraries shared BSD switched later, c. 1998-2000 Segments and sections High-level ELF file structure A fixed-size header with a magic number \x7fELF The run-time structure of an ELF executable divides it into segments and basic information The table describing segments is the program headers The program headers The compile-time structure of an ELF file divides it Code and data that are loaded when the program into sections runs The table describing segments is the section headers Data that isn’t normally loaded, like debugging Commonly several sections make up a segment symbols The section headers Kinds of ELF files The two main segments The code segment contains executable code, and .o .obj Relocatable object files (Windows: ) read-only data .exe Executables (Windows: ) The data segment contains writeable data .so .dll Dynamic/shared object files (Windows: ) Both are type LOAD Core dumps The segments are arranged this way so only two memory mappings are needed Other common segments The main sections INTERP: holds pathname of dynamic loader .text: most code DYNAMIC: information used by dynamic linking .rodata: read-only data like string constants (GNU )STACK: specifies stack permissions .data: initialized data (values stored in file) NOTE: miscellaneous data; in core dumps, register .bss: zero-initialized data (zeros not stored) values Other common sections Outline .init/.fini: startup/cleanup code .rel.*/.rela.*: relocation information ELF basics .comment: compiler version number .eh frame: exception-handling metadata Static and dynamic linking .debug *: debugging information Static linking Static linking vs. PIE Standard fixed addresses: At compile time, combining .o files into an x86-32: Starting at 0x08480000 executable binary x86-64: Code at 0x400000, data at 0x600000 The Unix linker is traditionally called ld Recent systems default to making even the main Generally, content from the same section of each executable position independent object file is grouped together PIE = position-independent executable PIE binaries look like shared libraries, and look on Traditionally, the linker chooses a fixed address for disk like they start at address 0 the binary to be loaded at At run-time, these offsets added to a random base Static libraries Relocation Unix static libraries end in .a, and are just archives Content in .o files must be fixed up when final of .o files locations chosen Program ar was once a relative of tar The relocation table tells the linker how The .o file are the unit of code inclusion Gives location, target symbol, machine-specific type So, often one per API function An additional offset (“addend”) may be stored in the Transitive requirements and ordering are not original bytes or in the table automatic Relocations are always size-preserving Symbol table Static program startup Static programs are loaded just by the kernel, and ELF files that define symbols list them in a symbol fairly simple table section .symtab Code and data regions are mapped as if by mmap Can examine with nm as well as objdump (demand paged) By default, finished executables include the symbol Stack is initialized with arguments, environment table variables, and auxiliary vector auxv But it is removed by strip Execution starts at the entry point Kinds of dynamic linking Dynamic program startup Automatic: libraries chosen at compile time, loaded The kernel loads both the program and the dynamic before main loader ld.so See list with ldd Full name, e.g., /lib64/ld-linux-x86-64.so.2 Linking process transparent to code, like static ld.so On-demand: requested by program runs first and performs linking Then returns control to the main program Open library with dlopen, lookup symbol with dlsym (in library libdl) You can also invoke ld.so manually Used for things like plugins Dynamic linking structures The PLT and GOT Dynamic linking uses ELF file structures separate but The Procedure Linkage Table (PLT) contains a code analogous to static linking, e.g.: stub for each called function from an external library printf@plt .dynamic section is DYNAMIC segment The static linker makes calls to, e.g., , in place Dynamic symbols .dynsym (c.f. nm -D) of a static function address Dynamic relocations .rela.dyn PLT stubs reference function pointers stored in the Calls to dynamically-linked functions use code stubs Global Offset Table (GOT) E.g. a pointer holding the location of printf in the C in the PLT referencing pointers in the GOT library Lazy resolution vs. RELRO To save startup time, symbol lookup for a function is often delayed until the first call, “lazy” On the other hand, dynamic linker structures are useful for attackers Writeable function pointers with a standard layout RELRO (relocation read-only) configurations make more dynamic linking state read-only after startup Full RELRO disables lazy resolution.
Load more

Recommended publications
  • Theendokernel: Fast, Secure
    The Endokernel: Fast, Secure, and Programmable Subprocess Virtualization Bumjin Im Fangfei Yang Chia-Che Tsai Michael LeMay Rice University Rice University Texas A&M University Intel Labs Anjo Vahldiek-Oberwagner Nathan Dautenhahn Intel Labs Rice University Abstract Intra-Process Sandbox Multi-Process Commodity applications contain more and more combina- ld/st tions of interacting components (user, application, library, and Process Process Trusted Unsafe system) and exhibit increasingly diverse tradeoffs between iso- Unsafe ld/st lation, performance, and programmability. We argue that the Unsafe challenge of future runtime isolation is best met by embracing syscall()ld/st the multi-principle nature of applications, rethinking process Trusted Trusted read/ architecture for fast and extensible intra-process isolation. We IPC IPC write present, the Endokernel, a new process model and security Operating System architecture that nests an extensible monitor into the standard process for building efficient least-authority abstractions. The Endokernel introduces a new virtual machine abstraction for Figure 1: Problem: intra-process is bypassable because do- representing subprocess authority, which is enforced by an main is opaque to OS, sandbox limits functionality, and inter- efficient self-isolating monitor that maps the abstraction to process is slow and costly to apply. Red indicates limitations. system level objects (processes, threads, files, and signals). We show how the Endokernel Architecture can be used to develop enforces subprocess access control to memory and CPU specialized separation abstractions using an exokernel-like state [10, 17, 24, 25, 75].Unfortunately, these approaches only organization to provide virtual privilege rings, which we use virtualize minimal parts of the CPU and neglect tying their to reorganize and secure NGINX.
    [Show full text]
  • DC Console Using DC Console Application Design Software
    DC Console Using DC Console Application Design Software DC Console is easy-to-use, application design software developed specifically to work in conjunction with AML’s DC Suite. Create. Distribute. Collect. Every LDX10 handheld computer comes with DC Suite, which includes seven (7) pre-developed applications for common data collection tasks. Now LDX10 users can use DC Console to modify these applications, or create their own from scratch. AML 800.648.4452 Made in USA www.amltd.com Introduction This document briefly covers how to use DC Console and the features and settings. Be sure to read this document in its entirety before attempting to use AML’s DC Console with a DC Suite compatible device. What is the difference between an “App” and a “Suite”? “Apps” are single applications running on the device used to collect and store data. In most cases, multiple apps would be utilized to handle various operations. For example, the ‘Item_Quantity’ app is one of the most widely used apps and the most direct means to take a basic inventory count, it produces a data file showing what items are in stock, the relative quantities, and requires minimal input from the mobile worker(s). Other operations will require additional input, for example, if you also need to know the specific location for each item in inventory, the ‘Item_Lot_Quantity’ app would be a better fit. Apps can be used in a variety of ways and provide the LDX10 the flexibility to handle virtually any data collection operation. “Suite” files are simply collections of individual apps. Suite files allow you to easily manage and edit multiple apps from within a single ‘store-house’ file and provide an effortless means for device deployment.
    [Show full text]
  • Memory Layout and Access Chapter Four
    Memory Layout and Access Chapter Four Chapter One discussed the basic format for data in memory. Chapter Three covered how a computer system physically organizes that data. This chapter discusses how the 80x86 CPUs access data in memory. 4.0 Chapter Overview This chapter forms an important bridge between sections one and two (Machine Organization and Basic Assembly Language, respectively). From the point of view of machine organization, this chapter discusses memory addressing, memory organization, CPU addressing modes, and data representation in memory. From the assembly language programming point of view, this chapter discusses the 80x86 register sets, the 80x86 mem- ory addressing modes, and composite data types. This is a pivotal chapter. If you do not understand the material in this chapter, you will have difficulty understanding the chap- ters that follow. Therefore, you should study this chapter carefully before proceeding. This chapter begins by discussing the registers on the 80x86 processors. These proces- sors provide a set of general purpose registers, segment registers, and some special pur- pose registers. Certain members of the family provide additional registers, although typical application do not use them. After presenting the registers, this chapter describes memory organization and seg- mentation on the 80x86. Segmentation is a difficult concept to many beginning 80x86 assembly language programmers. Indeed, this text tends to avoid using segmented addressing throughout the introductory chapters. Nevertheless, segmentation is a power- ful concept that you must become comfortable with if you intend to write non-trivial 80x86 programs. 80x86 memory addressing modes are, perhaps, the most important topic in this chap- ter.
    [Show full text]
  • Types and Programming Languages by Benjamin C
    < Free Open Study > . .Types and Programming Languages by Benjamin C. Pierce ISBN:0262162091 The MIT Press © 2002 (623 pages) This thorough type-systems reference examines theory, pragmatics, implementation, and more Table of Contents Types and Programming Languages Preface Chapter 1 - Introduction Chapter 2 - Mathematical Preliminaries Part I - Untyped Systems Chapter 3 - Untyped Arithmetic Expressions Chapter 4 - An ML Implementation of Arithmetic Expressions Chapter 5 - The Untyped Lambda-Calculus Chapter 6 - Nameless Representation of Terms Chapter 7 - An ML Implementation of the Lambda-Calculus Part II - Simple Types Chapter 8 - Typed Arithmetic Expressions Chapter 9 - Simply Typed Lambda-Calculus Chapter 10 - An ML Implementation of Simple Types Chapter 11 - Simple Extensions Chapter 12 - Normalization Chapter 13 - References Chapter 14 - Exceptions Part III - Subtyping Chapter 15 - Subtyping Chapter 16 - Metatheory of Subtyping Chapter 17 - An ML Implementation of Subtyping Chapter 18 - Case Study: Imperative Objects Chapter 19 - Case Study: Featherweight Java Part IV - Recursive Types Chapter 20 - Recursive Types Chapter 21 - Metatheory of Recursive Types Part V - Polymorphism Chapter 22 - Type Reconstruction Chapter 23 - Universal Types Chapter 24 - Existential Types Chapter 25 - An ML Implementation of System F Chapter 26 - Bounded Quantification Chapter 27 - Case Study: Imperative Objects, Redux Chapter 28 - Metatheory of Bounded Quantification Part VI - Higher-Order Systems Chapter 29 - Type Operators and Kinding Chapter 30 - Higher-Order Polymorphism Chapter 31 - Higher-Order Subtyping Chapter 32 - Case Study: Purely Functional Objects Part VII - Appendices Appendix A - Solutions to Selected Exercises Appendix B - Notational Conventions References Index List of Figures < Free Open Study > < Free Open Study > Back Cover A type system is a syntactic method for automatically checking the absence of certain erroneous behaviors by classifying program phrases according to the kinds of values they compute.
    [Show full text]
  • Use of Seek When Writing Or Reading Binary Files
    Title stata.com file — Read and write text and binary files Description Syntax Options Remarks and examples Stored results Reference Also see Description file is a programmer’s command and should not be confused with import delimited (see [D] import delimited), infile (see[ D] infile (free format) or[ D] infile (fixed format)), and infix (see[ D] infix (fixed format)), which are the usual ways that data are brought into Stata. file allows programmers to read and write both text and binary files, so file could be used to write a program to input data in some complicated situation, but that would be an arduous undertaking. Files are referred to by a file handle. When you open a file, you specify the file handle that you want to use; for example, in . file open myfile using example.txt, write myfile is the file handle for the file named example.txt. From that point on, you refer to the file by its handle. Thus . file write myfile "this is a test" _n would write the line “this is a test” (without the quotes) followed by a new line into the file, and . file close myfile would then close the file. You may have multiple files open at the same time, and you may access them in any order. 1 2 file — Read and write text and binary files Syntax Open file file open handle using filename , read j write j read write text j binary replace j append all Read file file read handle specs Write to file file write handle specs Change current location in file file seek handle query j tof j eof j # Set byte order of binary file file set handle byteorder hilo j lohi j 1 j 2 Close
    [Show full text]
  • Introduction to Computer Systems 15-213/18-243, Spring 2009 1St
    M4-L3: Executables CSE351, Winter 2021 Executables CSE 351 Winter 2021 Instructor: Teaching Assistants: Mark Wyse Kyrie Dowling Catherine Guevara Ian Hsiao Jim Limprasert Armin Magness Allie Pfleger Cosmo Wang Ronald Widjaja http://xkcd.com/1790/ M4-L3: Executables CSE351, Winter 2021 Administrivia ❖ Lab 2 due Monday (2/8) ❖ hw12 due Friday ❖ hw13 due next Wednesday (2/10) ▪ Based on the next two lectures, longer than normal ❖ Remember: HW and readings due before lecture, at 11am PST on due date 2 M4-L3: Executables CSE351, Winter 2021 Roadmap C: Java: Memory & data car *c = malloc(sizeof(car)); Car c = new Car(); Integers & floats c->miles = 100; c.setMiles(100); x86 assembly c->gals = 17; c.setGals(17); Procedures & stacks float mpg = get_mpg(c); float mpg = Executables free(c); c.getMPG(); Arrays & structs Memory & caches Assembly get_mpg: Processes pushq %rbp language: movq %rsp, %rbp Virtual memory ... Memory allocation popq %rbp Java vs. C ret OS: Machine 0111010000011000 100011010000010000000010 code: 1000100111000010 110000011111101000011111 Computer system: 3 M4-L3: Executables CSE351, Winter 2021 Reading Review ❖ Terminology: ▪ CALL: compiler, assembler, linker, loader ▪ Object file: symbol table, relocation table ▪ Disassembly ▪ Multidimensional arrays, row-major ordering ▪ Multilevel arrays ❖ Questions from the Reading? ▪ also post to Ed post! 4 M4-L3: Executables CSE351, Winter 2021 Building an Executable from a C File ❖ Code in files p1.c p2.c ❖ Compile with command: gcc -Og p1.c p2.c -o p ▪ Put resulting machine code in
    [Show full text]
  • The Ifplatform Package
    The ifplatform package Original code by Johannes Große Package by Will Robertson http://github.com/wspr/ifplatform v0.4a∗ 2017/10/13 1 Main features and usage This package provides the three following conditionals to test which operating system is being used to run TEX: \ifwindows \iflinux \ifmacosx \ifcygwin If you only wish to detect \ifwindows, then it does not matter how you load this package. Note then that use of (Linux or Mac OS X or Cygwin) can then be detected with \ifwindows\else. If you also wish to determine the difference between which Unix-variant you are using (i.e., also detect \iflinux, \ifmacosx, and \ifcygwin) then shell escape must be enabled. This is achieved by using the -shell-escape command line option when executing LATEX. If shell escape is not enabled, \iflinux, \ifmacosx, and \ifcygwin will all return false. A warning will be printed in the console output to remind you in this case. ∗Thanks to Ken Brown, Joseph Wright, Zebb Prime, and others for testing this package. 1 2 Auxiliary features \ifshellescape is provided as a conditional to test whether shell escape is active or not. (Note: new versions of pdfTEX allow you to query shell escape with \ifnum\pdfshellescape>0 , and the pdftexcmds package provides the wrapper \pdf@shellescape which works with X TE EX, pdfTEX, and LuaTEX.) Also, the \platformname command is defined to expand to a macro that represents the operating system. Default definitions are (respectively): \windowsname ! ‘Windows’ \notwindowsname ! ‘*NIX’ (when shell escape is disabled) \linuxname ! ‘Linux’ \macosxname ! ‘Mac OS X’ \cygwinname ! ‘Cygwin’ \unknownplatform ! whatever is returned by uname E.g., if \ifwindows is true then \platformname expands to \windowsname, which expands to ‘Windows’.
    [Show full text]
  • The Gnu Binary Utilities
    The gnu Binary Utilities Version cygnus-2.7.1-96q4 May 1993 Roland H. Pesch Jeffrey M. Osier Cygnus Support Cygnus Support TEXinfo 2.122 (Cygnus+WRS) Copyright c 1991, 92, 93, 94, 95, 1996 Free Software Foundation, Inc. Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies. Permission is granted to copy and distribute modi®ed versions of this manual under the conditions for verbatim copying, provided also that the entire resulting derived work is distributed under the terms of a permission notice identical to this one. Permission is granted to copy and distribute translations of this manual into another language, under the above conditions for modi®ed versions. The GNU Binary Utilities Introduction ..................................... 467 1ar.............................................. 469 1.1 Controlling ar on the command line ................... 470 1.2 Controlling ar with a script ............................ 472 2ld.............................................. 477 3nm............................................ 479 4 objcopy ....................................... 483 5 objdump ...................................... 489 6 ranlib ......................................... 493 7 size............................................ 495 8 strings ........................................ 497 9 strip........................................... 499 Utilities 10 c++®lt ........................................ 501 11 nlmconv ....................................
    [Show full text]
  • CERES Software Bulletin 95-12
    CERES Software Bulletin 95-12 Fortran 90 Linking Experiences, September 5, 1995 1.0 Purpose: To disseminate experience gained in the process of linking Fortran 90 software with library routines compiled under Fortran 77 compiler. 2.0 Originator: Lyle Ziegelmiller ([email protected]) 3.0 Description: One of the summer students, Julia Barsie, was working with a plot program which was written in f77. This program called routines from a graphics package known as NCAR, which is also written in f77. Everything was fine. The plot program was converted to f90, and a new version of the NCAR graphical package was released, which was written in f77. A problem arose when trying to link the new f90 version of the plot program with the new f77 release of NCAR; many undefined references were reported by the linker. This bulletin is intended to convey what was learned in the effort to accomplish this linking. The first step I took was to issue the "-dryrun" directive to the f77 compiler when using it to compile the original f77 plot program and the original NCAR graphics library. "- dryrun" causes the linker to produce an output detailing all the various libraries that it links with. Note that these libaries are in addition to the libaries you would select on the command line. For example, you might compile a program with erbelib, but the linker would have to link with librarie(s) that contain the definitions of sine or cosine. Anyway, it was my hypothesis that if everything compiled and linked with f77, then all the libraries must be contained in the output from the f77's "-dryrun" command.
    [Show full text]
  • Also Includes Slides and Contents From
    The Compilation Toolchain Cross-Compilation for Embedded Systems Prof. Andrea Marongiu ([email protected]) Toolchain The toolchain is a set of development tools used in association with source code or binaries generated from the source code • Enables development in a programming language (e.g., C/C++) • It is used for a lot of operations such as a) Compilation b) Preparing Libraries Most common toolchain is the c) Reading a binary file (or part of it) GNU toolchain which is part of d) Debugging the GNU project • Normally it contains a) Compiler : Generate object files from source code files b) Linker: Link object files together to build a binary file c) Library Archiver: To group a set of object files into a library file d) Debugger: To debug the binary file while running e) And other tools The GNU Toolchain GNU (GNU’s Not Unix) The GNU toolchain has played a vital role in the development of the Linux kernel, BSD, and software for embedded systems. The GNU project produced a set of programming tools. Parts of the toolchain we will use are: -gcc: (GNU Compiler Collection): suite of compilers for many programming languages -binutils: Suite of tools including linker (ld), assembler (gas) -gdb: Code debugging tool -libc: Subset of standard C library (assuming a C compiler). -bash: free Unix shell (Bourne-again shell). Default shell on GNU/Linux systems and Mac OSX. Also ported to Microsoft Windows. -make: automation tool for compilation and build Program development tools The process of converting source code to an executable binary image requires several steps, each with its own tool.
    [Show full text]
  • The “Stabs” Debug Format
    The \stabs" debug format Julia Menapace, Jim Kingdon, David MacKenzie Cygnus Support Cygnus Support Revision TEXinfo 2017-08-23.19 Copyright c 1992{2021 Free Software Foundation, Inc. Contributed by Cygnus Support. Written by Julia Menapace, Jim Kingdon, and David MacKenzie. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled \GNU Free Documentation License". i Table of Contents 1 Overview of Stabs ::::::::::::::::::::::::::::::: 1 1.1 Overview of Debugging Information Flow ::::::::::::::::::::::: 1 1.2 Overview of Stab Format ::::::::::::::::::::::::::::::::::::::: 1 1.3 The String Field :::::::::::::::::::::::::::::::::::::::::::::::: 2 1.4 A Simple Example in C Source ::::::::::::::::::::::::::::::::: 3 1.5 The Simple Example at the Assembly Level ::::::::::::::::::::: 4 2 Encoding the Structure of the Program ::::::: 7 2.1 Main Program :::::::::::::::::::::::::::::::::::::::::::::::::: 7 2.2 Paths and Names of the Source Files :::::::::::::::::::::::::::: 7 2.3 Names of Include Files:::::::::::::::::::::::::::::::::::::::::: 8 2.4 Line Numbers :::::::::::::::::::::::::::::::::::::::::::::::::: 9 2.5 Procedures ::::::::::::::::::::::::::::::::::::::::::::::::::::: 9 2.6 Nested Procedures::::::::::::::::::::::::::::::::::::::::::::: 11 2.7 Block Structure
    [Show full text]
  • Download Article (PDF)
    Proceedings of the 2nd International Conference on Computer Science and Electronics Engineering (ICCSEE 2013) SecGOT: Secure Global Offset Tables in ELF Executables Chao Zhang, Lei Duan, Tao Wei, Wei Zou Beijing Key Laboratory of Internet Security Technology Institute of Computer Science and Technology, Peking University Beijing, China {chao.zhang, lei_duan, wei_tao, zou_wei}@pku.edu.cn Abstract—Global Offset Table (GOT) is an important feature library code for these two processes are different). This to support library sharing in Executable and Linkable Format problem also restrains the code sharing feature of libraries. (ELF) applications. The addresses of external modules’ global A solution called PIC (Position Independent Code [3]) is variables and functions are runtime resolved and stored in the proposed for the ELF (Executable and Linkable Format [4]) GOT and then are used by the program. If attackers tamper executable binaries which are common in Linux. with the function pointers in the GOT, they can hijack the In libraries or main modules supporting PIC, the code program’s control flow and execute arbitrary malicious code. section does not reference any absolute addresses in order to Current research pays few attentions on this threat (i.e. GOT support code sharing between processes. However, absolute hijacking attack). In this paper, we proposed and implemented addresses are unavoidable in programs. As a result, a GOT a protection mechanism SecGOT to randomize the GOT at table (Global Offset Table [4]) is introduced in the library. load time, and thus prevent attackers from guessing the GOT’s position and tampering with the function pointers. SecGOT is This table resides in the data section and is not shared evaluated against 101 binaries in the /bin directory for Linux.
    [Show full text]