Introduction to 64 Bit Intel Assembly Language Programming for Linux Introduction to 64 Bit Intel Assembly Language Programming for Linux Ray Seyfarth October 27, 2011 Ray Seyfarth School of Computing University of Southern Mississippi Hattiesburg, MS 39406 USA Seyfarth, Ray Introduction to 64 Bit Intel Assembly Language Programming Includes index. ISBN-13: 978-1466470033 ISBN-10: 1466470038 @2011 Ray Seyfarth All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the copyright holder, except for brief excerpts in connection with reviews or scholarly analyses. Preface The Intel CPU architecture has evolved over 3 decades fr om a 16 bit CPU with no memory protection, through a period with 32 bit processors with sophisticated architectures into the current series of processors which support all the old modes of operation in addition to a greatly expanded 64 bit mode of operation. Assembly textbooks tend to focus on the history and generally conclude with a discussion on the 32 bit mode. Students are introduced to the concepts of 16 bit CPUs with segment registers allowing access to 1 megabyte of internal memory. This is an unnecessary fo cus on the past. With the x86-64 architecture there is almost a complete departure from the past. Segment registers are essentially obsolete and more reg­ ister usage is completely general purpose, with the glaring exception of the repeat-string loops which use specific registers and have no operands. Both these changes contribute to simpler assembly language program­ mmg. There are now 16 general purpose integer registers with a few spe­ cialized instructions. The archaic register stack of the 8087 has been superseded by a well-organized model providing 16 floating point regis­ ters with the floating point instructions for the SSE and AVX extensions. In fact the AV X extensions even allow a three operand syntax which can simplify coding even more. Overall the x86-64 assembly language programming is simpler than its predecessors. The dominant mode of operation will be 64 bits within a few short years. Together these trends indicate that it is time to teach 64 bit assembly language. The fo cus in this textbook is on early hands-on use of 64 bit assembly prograrnmmg. There is no 16 or 32 bit programming and the discussion Ill IV PREFACE of the history is fo cused on explaining the origin of the old register names and the few non-orthogonal features of the instruction set. The intention is to get students involved with using the yasm assembler and the gdb debugger fr om the start. There are assignments using the computer from the very first chapter. Not every statement will be fully understood at this time, but the assignments are still possible. The primary target for this book is beginning assembly language pro­ grammers and for a gentle introduction to assembly programming, stu­ dents should study chapters 1, 2, 3, 5, 6, 7, 8, 9, 10 and 11. Chapter 4 on memory mapping is not critical to the rest of the book and can be skipped if desired. Chapters 12 through 15 are significantly more in depth. Chapter 15 is about data structures in assembly and is an excellent adjunct to studying data structures in C/C++. The subject will be much clearer after exposure in assembly language. The final four chapters focus on high performance programming, in­ cluding discussion of SSE and AVX programming. The author provides PDF slides for classroom instruction along with sample code and errata at http://rayseyfa rth.com/asm. If you find errors in the book or have suggestions for improvement, please email the author as [email protected]. Thank you for buying the book and I hope you find something inter­ esting and worthwhile inside. Acknow ledgernents No book is created in isolation. This book is certainly no exception. I am indebted to numerous sources for information and assistance with this book. Dr. Paul Carter's PC assembly language book was used by this author to study 32 bit assembly language programming. His book is a free PDF file downloadable from his web site. This is a 195 page book which covers the basics of assembly language and is a great start at 32 bit assembly language. While working on this book, I discovered a treatise by Drs. Bryant and O'Hallaron of Carnegie Mellon about how gee takes advantage of the features of the x86-64 architecture to produce efficient code. Some of their observations have helped me understand the CPU better which assists with writing better assembly code. Programmers interested in efficiency should study their work. I found the Intel manuals to be an invaluable resource. They provide details on all the instructions of the CPU. Unfortunately the documents cover 32 bit and 64 bit instructions together which, along with the huge number of instructions, makes it difficult to learn assembly programming from these manuals. I hope that reading this book will make a good starting point, but a short book cannot cover many instructions. I have selected what I consider the most important instructions for general use, but an assembly programmer will need to study the Intel manuals (or equivalent manuals from AMD). I thank my friends Maggie and Tim Hampton for their editing con­ tributions to the book. I am indebted to my CSC 203 - Assembly Language class at the University of Southern Mississippi for their contributions to this book. v Vl ACKNOWLEDGEMENTS Te aching 64 bit assembly language has uncovered a few mistakes and errors in the original Create Space book from July 2011. In particular I wish to thank Chris Greene, Evan Stuart and Brandon Wolfe for locating errors in the book. Last I thank my wife, Phyllis, and my sons, David and Adam, for their encouragement and assistance. Phyllis and Adam are responsible for the cover design for both this and the Create Space book. Contents Preface iii Acknowledgements v 1 Introduction 1 1.1 Why study assembly language? 2 1.2 What is a computer? . 4 1.2.1 Bytes ...... 4 1.2.2 Program execution 4 1.3 Machine language .... 5 1.4 Assembly language ... 6 1.5 Assembling and linking 8 2 Numbers 11 2.1 Binary numbers . 11 2.2 Hexadecimal numbers 13 2.3 Integers ....... 16 2.3.1 Binary addition . 18 2.3.2 Binary multiplication 19 2.4 Floating point numbers ... 20 2.4.1 Converting decimal numbers to floats 23 2.4.2 Converting floats to decimal . 24 2.4.3 Floating point addition . 24 2.4.4 Floating point multiplication 25 3 Computer memory 27 3.1 Memory mapping . 27 Vll Vlll CONTENTS 3.2 Process memory model in Linux 28 3.3 Memory example ...... 30 3.4 Examining memory with gdb 32 3.4.1 Printing with gdb 32 3.4.2 Examining memory 34 4 Memory mapping in 64 bit mode 37 4.1 The memory mapping register . 37 4.2 Page Map Level 4 . 38 4.3 Page Directory Pointer Table 39 4.4 Page Directory Table . 39 4.5 Page Table . 39 4.6 Large pages . 40 4. 7 CPU Support for Fast Lookups 40 5 Registers 43 5.1 Moving a constant into a register . 45 5.2 Moving values from memory into registers . 46 5.3 Moving values from a register into memory 49 5.4 Moving data from one register to another 49 6 A little bit of math 51 6.1 Negation .. 51 6.2 Addition . ... 52 6.3 Subtraction . 54 6.4 Multiplication . 55 6.5 Division .... 57 6.6 Conditional move instructions . 57 6.7 Why move to a register? .... 58 7 Bit operations 61 7.1 Not operation . 61 7.2 And operation 62 7.3 Or operation 63 7.4 Exclusive or operation 64 7.5 Shift operations . 65 7.6 Bit testing and setting 67 7. 7 Extracting and filling a bit field . 68 CONTENTS IX 8 Branching and looping 71 8.1 Unconditional jump .... 71 8.2 Conditional jump . 73 8.2.1 Simple if statement . 74 8.2.2 If/else statement . 75 8.2.3 If/else-if/else statement 75 8.3 Looping with conditional jumps . 76 8.3.1 While loops .. 76 8.3.2 Do-while loops 80 8.3.3 Counting loops 82 8.4 Loop instructions . 82 8.5 Repeat string (array) instructions . 83 8.5.1 String instructions ..... 83 9 Functions 89 9.1 The stack 89 9.2 Call instruction .. 90 9.3 Return instruction 91 9.4 Function parameters and return value 91 9.5 Stack frames 92 9.6 Recursion 94 10 Arrays 99 10.1 Array address computation . 99 10.2 General pattern for memory references 101 10.3 Allocating arrays .... 103 10.4 Processing arrays ........... 104 10.4.1 Creating the array ...... 104 10.4.2 Filling the array with random numbers 105 10.4.3 Printing the array .......... 106 10.4.4 Finding the minimum value ...... 107 10.4.5 Main program for the array minimum 107 10.5 Command line parameter array ....... 109 11 Floating point instructions 115 11.1 Floating point registers . 115 11.2 Moving data to/from floating point registers . 116 X CONTENTS 11.2.1 Moving scalars ... 116 11.2.2 Moving packed data 117 11.3 Addition .......... 117 11.4 Subtraction . 118 11.5 Multiplication and division 119 11.6 Conversion . 119 11.6.1 Converting to a different length floating point . 119 11.6.2 Converting floating point to/from integer 120 11.7 Floating point comparison .