Programming from the Ground Up

Total Page:16

File Type:pdf, Size:1020Kb

Programming from the Ground Up Programming from the Ground Up Jonathan Bartlett Edited by Dominick Bruno, Jr. Programming from the Ground Up by Jonathan Bartlett Edited by Dominick Bruno, Jr. Copyright © 2003 by Jonathan Bartlett Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in Appendix H. In addition, you are granted full rights to use the code examples for any purpose without even having to credit the authors. All trademarks are property of their respective owners. This book can be purchased at http://www.bartlettpublishing.com/ This book is not a reference book, it is an introductory book. It is therefore not suitable by itself to learn how to professionally program in x86 assembly language, as some details have been left out to make the learning process smoother. The point of the book is to help the student understand how assembly language and computer programming works, not to be a reference to the subject. Reference information about a particular processor can be obtained by contacting the company which makes it. To receive a copy of this book in electronic form, please visit the website http://savannah.nongnu.org/projects/pgubook/ This site contains the instructions for downloading a transparent copy of this book as defined by the GNU Free Documentation License. Table of Contents 1. Introduction........................................................................................................1 Welcome to Programming ...............................................................................1 Your Tools........................................................................................................3 2. Computer Architecture .....................................................................................7 Structure of Computer Memory.......................................................................7 The CPU...........................................................................................................9 Some Terms ...................................................................................................11 Interpreting Memory......................................................................................13 Data Accessing Methods................................................................................14 Review ...........................................................................................................16 3. Your First Programs ........................................................................................19 Entering in the Program.................................................................................19 Outline of an Assembly Language Program..................................................22 Planning the Program.....................................................................................28 Finding a Maximum Value.............................................................................31 Addressing Modes .........................................................................................41 Review ...........................................................................................................45 4. All About Functions.........................................................................................49 Dealing with Complexity...............................................................................49 How Functions Work .....................................................................................50 Assembly-Language Functions using the C Calling Convention ..................52 A Function Example ......................................................................................59 Recursive Functions.......................................................................................64 Review ...........................................................................................................71 5. Dealing with Files.............................................................................................75 The UNIX File Concept.................................................................................75 Buffers and .bss ...........................................................................................76 Standard and Special Files.............................................................................78 Using Files in a Program................................................................................79 iii Review ...........................................................................................................93 6. Reading and Writing Simple Records ...........................................................95 Writing Records...........................................................................................100 Reading Records ..........................................................................................104 Modifying the Records ................................................................................111 Review .........................................................................................................114 7. Developing Robust Programs........................................................................117 Where Does the Time Go?...........................................................................117 Some Tips for Developing Robust Programs...............................................118 Handling Errors Effectively .........................................................................121 Making Our Program More Robust .............................................................123 Review .........................................................................................................126 8. Sharing Functions with Code Libraries.......................................................129 Using a Shared Library ................................................................................130 How Shared Libraries Work ........................................................................133 Finding Information about Libraries............................................................134 Useful Functions ..........................................................................................140 Building a Shared Library............................................................................141 Review .........................................................................................................143 9. Intermediate Memory Topics........................................................................147 How a Computer Views Memory ................................................................147 The Memory Layout of a Linux Program....................................................149 Every Memory Address is a Lie ..................................................................151 Getting More Memory .................................................................................155 A Simple Memory Manager ........................................................................157 Using our Allocator......................................................................................174 More Information.........................................................................................177 Review .........................................................................................................178 iv 10. Counting Like a Computer .........................................................................181 Counting.......................................................................................................181 Truth, Falsehood, and Binary Numbers .......................................................186 The Program Status Register .......................................................................195 Other Numbering Systems...........................................................................196 Octal and Hexadecimal Numbers ................................................................199 Order of Bytes in a Word .............................................................................201 Converting Numbers for Display.................................................................204 Review .........................................................................................................210 11. High-Level Languages .................................................................................213 Compiled and Interpreted Languages ..........................................................213 Your First C Program...................................................................................215 Perl ...............................................................................................................218 Python ..........................................................................................................219 Review .........................................................................................................220 12. Optimization.................................................................................................223 When to Optimize........................................................................................223 Where to Optimize.......................................................................................224 Local Optimizations.....................................................................................225
Recommended publications
  • Linux Boot Loaders Compared
    Linux Boot Loaders Compared L.C. Benschop May 29, 2003 Copyright c 2002, 2003, L.C. Benschop, Eindhoven, The Netherlands. Per- mission is granted to make verbatim copies of this document. This is version 1.1 which has some minor corrections. Contents 1 introduction 2 2 How Boot Loaders Work 3 2.1 What BIOS does for us . 3 2.2 Parts of a boot loader . 6 2.2.1 boot sector program . 6 2.2.2 second stage of boot loader . 7 2.2.3 Boot loader installer . 8 2.3 Loading the operating system . 8 2.3.1 Loading the Linux kernel . 8 2.3.2 Chain loading . 10 2.4 Configuring the boot loader . 10 3 Example Installations 11 3.1 Example root file system and kernel . 11 3.2 Linux Boot Sector . 11 3.3 LILO . 14 3.4 GNU GRUB . 15 3.5 SYSLINUX . 18 3.6 LOADLIN . 19 3.7 Where Can Boot Loaders Live . 21 1 4 RAM Disks 22 4.1 Living without a RAM disk . 22 4.2 RAM disk devices . 23 4.3 Loading a RAM disk at boot time . 24 4.4 The initial RAM disk . 24 5 Making Diskette Images without Diskettes 25 6 Hard Disk Installation 26 7 CD-ROM Installation 29 8 Conclusions 31 1 introduction If you use Linux on a production system, you will only see it a few times a year. If you are a hobbyist who compiles many kernels or who uses many operating systems, you may see it several times per day.
    [Show full text]
  • GNU MP the GNU Multiple Precision Arithmetic Library Edition 6.2.1 14 November 2020
    GNU MP The GNU Multiple Precision Arithmetic Library Edition 6.2.1 14 November 2020 by Torbj¨ornGranlund and the GMP development team This manual describes how to install and use the GNU multiple precision arithmetic library, version 6.2.1. Copyright 1991, 1993-2016, 2018-2020 Free Software Foundation, Inc. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with the Front-Cover Texts being \A GNU Manual", and with the Back-Cover Texts being \You have freedom to copy and modify this GNU Manual, like GNU software". A copy of the license is included in Appendix C [GNU Free Documentation License], page 132. i Table of Contents GNU MP Copying Conditions :::::::::::::::::::::::::::::::::::: 1 1 Introduction to GNU MP ::::::::::::::::::::::::::::::::::::: 2 1.1 How to use this Manual :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 2 2 Installing GMP ::::::::::::::::::::::::::::::::::::::::::::::::: 3 2.1 Build Options:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 3 2.2 ABI and ISA :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 8 2.3 Notes for Package Builds:::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 11 2.4 Notes for Particular Systems :::::::::::::::::::::::::::::::::::::::::::::::::::::: 12 2.5 Known Build Problems ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 14 2.6 Performance
    [Show full text]
  • Introduction to X86 Assembly
    CS342 Computer Security Handout # 5 Prof. Lyn Turbak Monday, Oct. 04, 2010 Wellesley College Lab 4: Introduction to x86 Assembly Reading: Hacking, 0x250, 0x270 Overview Today, we continue to cover low-level programming details that are essential for understanding software vulnerabilities like buffer overflow attacks and format string exploits. You will get exposure to the following: • Understanding conventions used by compiler to translate high-level programs to low-level assembly code (in our case, using Gnu C Compiler (gcc) to compile C programs). • The ability to read low-level assembly code (in our case, Intel x86). • Understanding how assembly code instructions are represented as machine code. • Being able to use gdb (the Gnu Debugger) to read the low-level code produced by gcc and understand its execution. In tutorials based on this handout, we will learn about all of the above in the context of some simple examples. Intel x86 Assembly Language Since Intel x86 processors are ubiquitous, it is helpful to know how to read assembly code for these processors. We will use the following terms: byte refers to 8-bit quantities; short word refers to 16-bit quantities; word refers to 32-bit quantities; and long word refers to 64-bit quantities. There are many registers, but we mostly care about the following: • EAX, EBX, ECX, EDX are 32-bit registers used for general storage. • ESI and EDI are 32-bit indexing registers that are sometimes used for general storage. • ESP is the 32-bit register for the stack pointer, which holds the address of the element currently at the top of the stack.
    [Show full text]
  • ECE 598 – Advanced Operating Systems Lecture 2
    ECE 598 { Advanced Operating Systems Lecture 2 Vince Weaver http://www.eece.maine.edu/~vweaver [email protected] 15 January 2015 Announcements • Update on room situation (shouldn't be locked anymore, but not much chance of getting a better room) 1 Building Linux by Hand • Check out with git or download tarball: git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git http://www.kernel.org/pub/linux/kernel/v3.x/ • Configure. make config or make menuconfig Also can copy existing .config and run make oldconfig • Compile. 2 make What does make -j 8 do differently? • Make the modules. make modules • sudo make modules install • copy bzImage to boot, update boot loader 3 Building Linux Automated • If in a distro there are other commands to building a package. • For example on Debian make-kpkg --initrd --rootcmd fakeroot kernel image • Then dpkg -i to install; easier to track 4 Overhead (i.e. why not to do it natively on a Pi) • Size { clean git source tree (x86) 1.8GB, compiled with kernel, 2.5GB, compiled kernel with debug features (x86), 12GB!!! Tarball version with compiled kernel (ARM) 1.5GB • Time to compile { minutes (on fast multicore x86 machine) to hours (18 hours or so on Pi) 5 Linux on the Pi • Mainline kernel, bcm2835 tree Missing some features • Raspberry-pi foundation bcm2708 tree More complete, not upstream 6 Compiling { how does it work? • Take C-code • Turn into assembler • Run assembler to make object (machine language files). Final addresses not necessarily put into place • Run linker (which uses linker script) • Makes some sort of executable.
    [Show full text]
  • IT Acronyms.Docx
    List of computing and IT abbreviations /.—Slashdot 1GL—First-Generation Programming Language 1NF—First Normal Form 10B2—10BASE-2 10B5—10BASE-5 10B-F—10BASE-F 10B-FB—10BASE-FB 10B-FL—10BASE-FL 10B-FP—10BASE-FP 10B-T—10BASE-T 100B-FX—100BASE-FX 100B-T—100BASE-T 100B-TX—100BASE-TX 100BVG—100BASE-VG 286—Intel 80286 processor 2B1Q—2 Binary 1 Quaternary 2GL—Second-Generation Programming Language 2NF—Second Normal Form 3GL—Third-Generation Programming Language 3NF—Third Normal Form 386—Intel 80386 processor 1 486—Intel 80486 processor 4B5BLF—4 Byte 5 Byte Local Fiber 4GL—Fourth-Generation Programming Language 4NF—Fourth Normal Form 5GL—Fifth-Generation Programming Language 5NF—Fifth Normal Form 6NF—Sixth Normal Form 8B10BLF—8 Byte 10 Byte Local Fiber A AAT—Average Access Time AA—Anti-Aliasing AAA—Authentication Authorization, Accounting AABB—Axis Aligned Bounding Box AAC—Advanced Audio Coding AAL—ATM Adaptation Layer AALC—ATM Adaptation Layer Connection AARP—AppleTalk Address Resolution Protocol ABCL—Actor-Based Concurrent Language ABI—Application Binary Interface ABM—Asynchronous Balanced Mode ABR—Area Border Router ABR—Auto Baud-Rate detection ABR—Available Bitrate 2 ABR—Average Bitrate AC—Acoustic Coupler AC—Alternating Current ACD—Automatic Call Distributor ACE—Advanced Computing Environment ACF NCP—Advanced Communications Function—Network Control Program ACID—Atomicity Consistency Isolation Durability ACK—ACKnowledgement ACK—Amsterdam Compiler Kit ACL—Access Control List ACL—Active Current
    [Show full text]
  • CS 331 GDB and X86-64 Assembly Language
    CS 331 Spring 2016 GDB and x86-64 Assembly Language I am not a big fan of debuggers for high-level languages. While they are undeniably useful at times, they are no substitute for careful coding. If code is well structured, I usually find it easier to debug analytically (i.e, by staring at it) than by running it through a debugger. In any event, monkey-coding with a debugger is still monkey-coding, and I don’t want to fly in an airplane being guided by monkey-code. The situation in assembly language is quite different. Assembly instructions can be opaque, especially if you aren’t used to programming in assembly language. In addition, I/O at the assembly level is quite complex; printing values at various places in your code is often not an option.. Debuggers are frequently the only way to discover what your code is actually doing. There are nicer assemblers than the gas (gnu assembler) that we will use, but the gas-gdb combination works quite well and is easy to use. In this document we’ll go briefly through the essential commands for gdb. There is more complete documentation available on any of the Linux systems; just type info gdb And, of course, there is no end of documentation online. To assemble your program with hooks for gdb use the hyphen g flag. For example, if your assembly language program is foobar.s you would assemble it wth gcc foobar.s –g –o foobar and then run gdp with gdb foobar You should always have the program visible in an editor that includes line numbers while you run gdb.
    [Show full text]
  • GNU Toolchain for Atmel ARM Embedded Processors
    RELEASE NOTES GNU Toolchain for Atmel ARM Embedded Processors Introduction The Atmel ARM GNU Toolchain (5.3.1.487) supports Atmel ARM® devices. The ARM toolchain is based on the free and open-source GCC. This toolchain is built from sources published by ARM's "GNU Tools for ARM Embedded Processors" project at launchpad.net (https://launchpad.net/gcc-arm-embedded). The toolchain includes compiler, assembler, linker, binutils (GCC and binutils), GNU Debugger (GDB with builtin simulator) and Standard C library (newlib, newlib nano). 42368A-MCU-06/2016 Table of Contents Introduction .................................................................................... 1 1. Supported Configuration ......................................................... 3 1.1. Supported Hosts ................................................................... 3 1.2. Supported Targets ................................................................. 3 2. Downloading, Installing, and Upgrading ................................. 4 2.1. Downloading/Installing on Windows .......................................... 4 2.2. Downloading/Installing on Linux ............................................... 4 2.3. Downloading/Installing on Mac OS ........................................... 4 2.4. Upgrading ............................................................................ 4 3. Layout and Components ......................................................... 5 3.1. Layout ................................................................................. 5 3.2. Components
    [Show full text]
  • An ECMA-55 Minimal BASIC Compiler for X86-64 Linux®
    Computers 2014, 3, 69-116; doi:10.3390/computers3030069 OPEN ACCESS computers ISSN 2073-431X www.mdpi.com/journal/computers Article An ECMA-55 Minimal BASIC Compiler for x86-64 Linux® John Gatewood Ham Burapha University, Faculty of Informatics, 169 Bangsaen Road, Tambon Saensuk, Amphur Muang, Changwat Chonburi 20131, Thailand; E-mail: [email protected] Received: 24 July 2014; in revised form: 17 September 2014 / Accepted: 1 October 2014 / Published: 1 October 2014 Abstract: This paper describes a new non-optimizing compiler for the ECMA-55 Minimal BASIC language that generates x86-64 assembler code for use on the x86-64 Linux® [1] 3.x platform. The compiler was implemented in C99 and the generated assembly language is in the AT&T style and is for the GNU assembler. The generated code is stand-alone and does not require any shared libraries to run, since it makes system calls to the Linux® kernel directly. The floating point math uses the Single Instruction Multiple Data (SIMD) instructions and the compiler fully implements all of the floating point exception handling required by the ECMA-55 standard. This compiler is designed to be small, simple, and easy to understand for people who want to study a compiler that actually implements full error checking on floating point on x86-64 CPUs even if those people have little programming experience. The generated assembly code is also designed to be simple to read. Keywords: BASIC; compiler; AMD64; INTEL64; EM64T; x86-64; assembly 1. Introduction The Beginner’s All-purpose Symbolic Instruction Code (BASIC) language was invented by John G.
    [Show full text]
  • Pipenightdreams Osgcal-Doc Mumudvb Mpg123-Alsa Tbb
    pipenightdreams osgcal-doc mumudvb mpg123-alsa tbb-examples libgammu4-dbg gcc-4.1-doc snort-rules-default davical cutmp3 libevolution5.0-cil aspell-am python-gobject-doc openoffice.org-l10n-mn libc6-xen xserver-xorg trophy-data t38modem pioneers-console libnb-platform10-java libgtkglext1-ruby libboost-wave1.39-dev drgenius bfbtester libchromexvmcpro1 isdnutils-xtools ubuntuone-client openoffice.org2-math openoffice.org-l10n-lt lsb-cxx-ia32 kdeartwork-emoticons-kde4 wmpuzzle trafshow python-plplot lx-gdb link-monitor-applet libscm-dev liblog-agent-logger-perl libccrtp-doc libclass-throwable-perl kde-i18n-csb jack-jconv hamradio-menus coinor-libvol-doc msx-emulator bitbake nabi language-pack-gnome-zh libpaperg popularity-contest xracer-tools xfont-nexus opendrim-lmp-baseserver libvorbisfile-ruby liblinebreak-doc libgfcui-2.0-0c2a-dbg libblacs-mpi-dev dict-freedict-spa-eng blender-ogrexml aspell-da x11-apps openoffice.org-l10n-lv openoffice.org-l10n-nl pnmtopng libodbcinstq1 libhsqldb-java-doc libmono-addins-gui0.2-cil sg3-utils linux-backports-modules-alsa-2.6.31-19-generic yorick-yeti-gsl python-pymssql plasma-widget-cpuload mcpp gpsim-lcd cl-csv libhtml-clean-perl asterisk-dbg apt-dater-dbg libgnome-mag1-dev language-pack-gnome-yo python-crypto svn-autoreleasedeb sugar-terminal-activity mii-diag maria-doc libplexus-component-api-java-doc libhugs-hgl-bundled libchipcard-libgwenhywfar47-plugins libghc6-random-dev freefem3d ezmlm cakephp-scripts aspell-ar ara-byte not+sparc openoffice.org-l10n-nn linux-backports-modules-karmic-generic-pae
    [Show full text]
  • GNU Toolchain for Atmel AVR 32-Bit Embedded Processors
    RELEASE NOTES GNU Toolchain for Atmel AVR 32-bit Embedded Processors Introduction The Atmel AVR® 32-bit GNU Toolchain (3.4.3.820) supports all AVR 32-bit devices. The AVR 32-bit Toolchain is based on the free and open-source GCC compiler. The toolchain includes compiler, assembler, linker, and binutils (GCC and Binutils), Standard C library (Newlib). 32215A-MCU-08/2015 Table of Contents Introduction .................................................................................... 1 1. Installation Instructions .......................................................... 3 1.1. System Requirements ............................................................ 3 1.1.1. Hardware Requirements ............................................. 3 1.1.2. Software Requirements .............................................. 3 1.2. Downloading, Installing, and Upgrading ..................................... 3 1.2.1. Downloading/Installing on Windows .............................. 3 1.2.2. Downloading/Installing on Linux ................................... 3 1.2.3. Upgrading from previous versions ................................ 3 1.2.4. Manifest .................................................................. 3 1.3. Layout ................................................................................. 4 2. Toolset Background ................................................................ 5 2.1. Compiler .............................................................................. 5 2.2. Assembler, Linker, and Librarian .............................................
    [Show full text]
  • Using As, the GNU Assembler
    Using as The gnu Assembler Version 2.14 The Free Software Foundation Inc. thanks The Nice Computer Company of Australia for loaning Dean Elsner to write the first (Vax) version of as for Project gnu. The proprietors, management and staff of TNCCA thank FSF for distracting the boss while they got some work done. Dean Elsner, Jay Fenlason & friends Using as Edited by Cygnus Support Copyright c 1991, 92, 93, 94, 95, 96, 97, 98, 99, 2000, 2001, 2002 Free Software Foundation, Inc. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled “GNU Free Documentation License”. i Table of Contents 1 Overview .................................. 1 1.1 Structure of this Manual ................................ 12 1.2 The GNU Assembler ................................... 12 1.3 Object File Formats .................................... 12 1.4 Command Line ........................................ 12 1.5 Input Files............................................. 13 1.6 Output (Object) File ................................... 13 1.7 Error and Warning Messages ............................ 14 2 Command-Line Options ................... 15 2.1 Enable Listings: ‘-a[cdhlns]’........................... 15 2.2 ‘-D’ ................................................... 15 2.3 Work Faster: ‘-f’ ...................................... 16 2.4 .include Search Path: ‘-I’ path ........................ 16 2.5 Difference Tables: ‘-K’ .................................. 16 2.6 Include Local Labels: ‘-L’............................... 16 2.7 Configuring listing output: ‘--listing’ .................. 16 2.8 Assemble in MRI Compatibility Mode: ‘-M’ .............. 17 2.9 Dependency Tracking: ‘--MD’............................ 19 2.10 Name the Object File: ‘-o’............................
    [Show full text]
  • The ARM Instruction Set
    APPENDIX A The ARM Instruction Set This appendix lists the ARM 64-bit instruction in two sections: first, the core instruction set, then the NEON and FPU instructions. There is a brief description of each instruction: {S} after an instruction indicates you can optionally set the condition flags. † means the instruction is an alias. ARM 64-Bit Core Instructions Instruction Description ADC{S} Add with carry ADD{S} Add ADDG Add with tag ADR Form PC relative address ADRP Form PC relative address to 4KB page AND{S} Bitwise AND (continued) © Stephen Smith 2020 367 S. Smith, Programming with 64-Bit ARM Assembly Language, https://doi.org/10.1007/978-1-4842-5881-1 AppEndix A THE ARM InstructiOn SEt Instruction Description ASR† Arithmetic shift right ASRV Arithmetic shift right variable AT† Address translate AUTDA, AUTDZA Authenticate data address, using key A AUTDB, AUTDZB Authenticate data address, using key B AUTIA, AUTIA1716 Authenticate instruction address, using key A AUTIASP, AUTIAZ Authenticate instruction address, using key A AUTIZA Authenticate instruction address, using key A AUTIB, AUTIB1716 Authenticate instruction address, using key B AUTIBSP, AUTIBZ Authenticate instruction address, using key B AUTIZB Authenticate instruction address, using key B AXFlag Convert floating-point condition flags B Branch B.cond Branch conditionally BFC† Bitfield clear BFI† Bitfield insert BFM Bitfield move BFXIL† Bitfield extract and insert at low end BIC{S} Bitwise bit clear BL Branch with link BLR Branch with link to register BLRAA, BLRAAZ Branch with
    [Show full text]