Reverse Engineering II: the Basics

Home , Endianness

Reverse Engineering II: The Basics

Gergely Erdélyi – Senior Manager, Anti-malware Research

Protecting the irreplaceable | f-secure.com Binary Numbers

1 0 1 1 - Nibble B

1 0 1 1 1 1 0 1 - Byte B D

1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 - Word B D 3 9

2 Byte Order a.k.a. Endianness

00 01 = 0x3412 (Little Endian) 12 34 = 0x1234 (Big Endian)

= 0x1234 (Little Endian) 34 12 = 0x3412 (Big Endian) 00 01

3 Little Endian Dword

00 01 02 03 12 34 56 78 0x78563412

78 56 34 12 0x12345678 00 01 02 03

4 Endianness Matters

• Data exchange between computers • Networking protocols • File formats for disk storage • Mixing endinannes

5 System Endianness

Little Big Switchable Endian Endian Endianness

PowerPC Intel x86 ARM (exc. G5)

Sparc Intel 8051 Alpha (exc. v9)

Most System/370 Intel IA64 uControllers

6 ASCII Code

Control Backspace, 0x00 - 0x1F Characters Line feed

Digits and 0-9 <> 0x20 - 0x3F Punctuation = .,: *-()!

Upper-case ABCD... 0x40 - 0x5F Letters and @[]\^_ Special

Lower-case abcd... 0x60 - 0x7E Letters and `{}|~ Special

7 ASCII Example

H e l l o 1 2 3 4 48 65 6C 6C 6F 20 31 32 33 34

http://en.wikipedia.org/wiki/ASCII

8 Unicode Strings

BOM H e l l o ff fe 48 00 65 00 6c 00 6c 00 6f 00

UTF-16 / UCS-2

http://en.wikipedia.org/wiki/UTF-16/UCS-2 http://en.wikipedia.org/wiki/Category:Unicode

9 String Storage

• ASCIIZ: Zero-terminated ASCII • Pascal: Size byte + ASCII string • Delphi: Size Dword + ASCII or Unicode string

H e l l o ASCIIZ: 48 65 6C 6C 6F 00 Pascal: 05 48 65 6C 6C 6F

10 Intel x86 Architecture

11 Introduction to Intel x86

• Started with 8086 in 1978 • Continued with 8088, 80186, 80286, 386, 486, Pentium, 686 ... • CISC architecture • 32-bit is called x86-32 or IA-32 • 64-bit is called x86-64, AMD64, EMT64T

• 80386 introduced in 1986 • Has a 32-bit word length • Has eight general-purpose registers • Supports paging and virtual memory • Addresses up to 4GiB of memory

12 Data Register Layout

13 Data Registers

AL / AH / AX Accumulator Arithmetic operations EAX

BL / BH / BX General data storage, Data index EBX index

CL / CH / CX Loop counter Loop constructs ECX

DL / DH / DX Data register Arithmetics EDX

14 Address Registers

IP / EIP Instruction Pointer Program execution

SP / ESP Stack Pointer Stack operation

BP / EBP Base Pointer Stack frame

SI / ESI Source Index String operation

DI / EDI Destination Index String operation

15 Segment Registers

CS Code Segment Program code

DS Data Segment Program data

ES / FS / GS Other Segments Other uses

16 EFLAGS Register

17 Mnemonic Examples

MOV EAX, 1 Move 1 to EAX

ADD EDX, 5 Add 5 to EDX

SUB EBX, 2 Subtract 2 from EBX

AND ECX, 0 Bit-wise AND 0 to ECX

XOR EDX, 4 Bit-wise eXclusive OR 4 to EDX

SHL ECX, 6 Shift ECX left by six

ROR EBX, 3 Bit-wise rotate EBX right by 3

INC ECX Increment ECX

18 More Mnemonics

JNZ label Jump if not zero (equal)

JMP label Unconditional jump to label

CALL func Call function

RET Return from function

LOOP label ECX--, Jump to label if not zero

PUSH EAX Push EAX to stack

POP EDI Pop EDI from stack

LODSB Load byte from DS:ESI to AL

19 Reversing C Code

20 Basic Data Types

char - 1 byte short - 2 bytes int - 4 bytes (platform word) long - 4 bytes float - 4 bytes floating point double - 8 bytes floating point

21 Arrays and Pointers

• Pointers can point to any memory location • One-dimensional arrays are flat memory • Multi-dimensional arrays use pointers

char a[4]; A A A A char *b, c; c = a[2]; c = *(b+2);

22 Structures and Unions

Structure Union

union foo { struct { unsigned int id; int one; unsigned short age; char two; char name[16]; } record; };

Memory is allocated for all Memory is allocated for the members combined. largest member only.

sizeof(record) = 24 sizeof(foo) = 4

23 Structure Alignment

• Data structures are aligned to word size by default • #pragma pack(n) directive can change it • #pragma pack(1) removes alignment • Important when reconstructing structures

24 Structure Storage

Aligned Packed DWORD id DWORD id

WORD age WORD age

2-byte padding 16 BYTES name

16 BYTES name

sizeof(record) = 24 sizeof(record) = 22

25 Simple C Program

int foobar(int x, int y) { int z = x+y; return z; }

int main(void) { int z = foobar(1, 2); }

26 Function Calls

• Calling conventions are important to know • Mixing them will crash the program • __stdcall - Standard calls on Windows • __cdecl - Most common C calling convention • __fastcall - Uses registers for arguments • __thiscall - Pass ‘this’ pointer in ECX in C++

27 __cdecl Calls

Stack PUSH arg2 PUSH arg1 ARG2 CALL function ARG1 ADD ESP,8 RET Addr. PUSH EBP Saved EBP MOV EBP, ESP LOC1 SUB ESP, 4 MOV EAX, [EBP+8] arg1: EBP+8 MOV ESP, EBP arg2: EBP+12 POP EBP loc1: EBP-4 RET

28 __stdcall Calls

PUSH arg1 ARG1 PUSH arg2 ARG2 CALL function RET Addr. Saved EBP PUSH EBP MOV EBP, ESP LOC1 SUB ESP, 4 MOV EAX, [EBP+8] arg1: EBP+8 MOV ESP, EBP arg1: EBP+12 POP EBP loc1: EBP-4 RETN 8

29 Further Reading

Intel Processor Documentation http://www.intel.com/products/processor/ manuals/index.htm

Netwide Assembler Mnemonic Documentation http://sourceforge.net/docman/display_doc.php? docid=47259&group_id=6208

The Art of Assembly Language Programming Windows 32-bit Edition http://webster.cs.ucr.edu/AoA/index.html