Binghamton CS-220 University Spring 2019

X86/64 Calling Conventions Computer Systems Section 3.7 Binghamton CS-220 University Spring 2019 ISA “Application Binary Interface” (ABI)

• How Binary interacts with other machine code • Think of a single program made up of code in two different languages • Architecture standard for: • Size, layout, endian-ness, and alignment of data types and structures • Calling Conventions – how functions are called, parameters passed, etc. • Object File and Debug Formats • ABI Often includes Conventions for: • Register Usage (integer, floating point, vector) • Flag Usage • Interrupt and Exception Handling • (for ++, etc.) • Code Relocation Binghamton CS-220 University Spring 2019 ABI vs. API

By Shmuel Csaba Otto Traian, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=33816823 Binghamton CS-220 University Spring 2019 Calling Conventions

• When invoking a lower level function • How are parameters provided • How is return address specified • How is return value accessed • What registers can the function change? • When invoked by a higher level function • How are arguments accessed? • What registers can I use? • How should I use the stack? • How do I return values? Binghamton CS-220 University Spring 2019 Terminology

Caller Callee int main(int argc, char **argv) { int add(int x, int y) { Return Address* … Arguments int op1=x; ans=add(ans,4); int op2=y; term=subtract(x,2); Parameters int res=op1+op2; … return res; return 0; } } Return Value Binghamton CS-220 University Spring 2019 Control Flow Conventions pushq %rip • Caller will invoke using “callq” instruction jmp add • callq add ; Invoke add function • %rip points to instruction AFTER the callq instruction!

• Upon return, callee will invoke “retq” instruction popq %rip • retq ; Return to caller • Next instruction will be the instruction after the callq instruction • Assumes stack conventions have been followed! Binghamton CS-220 University Spring 2019 Parameter Passing Conventions

• Parameters are put into the following registers by the caller: • (Parameters which don’t fit are pushed on the stack)

Parm 1 Parm 2 Parm 3 Parm 4 Parm 5 Parm 6 %rdi %rsi %rdx %rcx %r8 %r9 Arg 1 Arg 2 Arg 3 Arg 4 Arg 5 Arg 6

• Arguments are read from these registers by the callee • (Parameters which don’t fit are read from caller’s stack frame) Binghamton CS-220 University Spring 2019 Return Value Convention

• Callee will put return value in %rax before return

• Caller can read return value from %rax after return Binghamton CS-220 University Spring 2019 Resource Sharing

• All functions use the same set of registers • If I use a register, does it clobber the value that my caller expects to be in that register? • If I call a lower level function, will the my callee clobber the values in my registers? • Resolve the problem by pushing register values on the stack before or at the beginning of a call, and popping the values from the stack after or at the end of a call • Who is responsible… caller or callee? • Do I need to save/restore all registers, or just the ones I’m using? Binghamton CS-220 University Spring 2019 Example Register Sharing

• On entry to a function, save all the register values on the stack • Requires copying 16*8=96 bytes of data to memory • Use any register you want • Very flexible… no need to think about register sharing • On exit, restore all registers from the stack • Requires copying 16*8=96 bytes of data from memory

• Result: Slow function calls, but flexible register sharing Binghamton CS-220 University Spring 2019 Register Modification Conventions • Caller may assume blue regs are not changed by the callee • If callee wants to use these registers, he must push the callers value in the callee invocation record on the stack on entry, and pop the callers value from the callee invocation record when returning • blue registers are callee saved (non-volatile) registers

• Caller may assume red regs may be changed by the callee • If caller wants to preserve these registers, he must push their value on the caller’s invocation record on the stack before the call, and pop their value from the caller’s invocation record after the call • Red registers are caller saved (volatile) registers

%rax %rbx %rcx %rdx %rsp %rbp %rsi %rdi %r8 %r9 %r10 %r11 %r12 %r13 %r14 %r15 Binghamton CS-220 University Spring 2019

Suppose main wants to preserveAddress %Valuerdi (64 bit) (OS) stack frame 0000 7FFF FFFF E870 0000 0000 0000 0000 %rbp 0000 7FFF FFFF E868 0000 7FFF F7A5 2B45 0000 7FFF FFFF E860 0000 7FFF FFFF E88C … main’s 0000 7FFF FFFF E858 0000 0004 0000 0000 4005D0 pushq %rdi stack frame 0000 7FFF FFFF E850 0000 0010 FFFF E940 4005D4 callq add ; at 400621 0000 7FFF FFFF E848 0000 0002 0040 0450 4005D9 mov %eax,-08x(%rbp) 4005DB popq %rdi 0000 7FFF FFFF E840 0000 7FFF FFFF E948 … %rsp 0000 7FFF FFFF E838 0000 0000 0040 05D9 0000 7FFF FFFF E830 (main’s %rdi value) 0000 7FFF FFFF E828 0000 0010 0000 0010 pushq %rip jmp add 0000 7FFF FFFF E820 0000 0000 0000 0000 0000 7FFF FFFF E818 0000 0010 0000 0010 0000 7FFF FFFF E810 0000 7FFF FFFF E940 …. Binghamton CS-220 University Spring 2019

Suppose main wants to preserveAddress %Valuerdi (64 bit) (OS) stack frame 0000 7FFF FFFF E870 0000 0000 0000 0000 pushq %rip %rbp 0000 7FFF FFFF E868 0000 7FFF F7A5 2B45 jmp add 0000 7FFF FFFF E860 0000 7FFF FFFF E88C … main’s 0000 7FFF FFFF E858 0000 0004 0000 0000 4005D0 pushq %rdi stack frame 0000 7FFF FFFF E850 0000 0010 FFFF E940 4005D4 callq addem ; at 400621 0000 7FFF FFFF E848 0000 0002 0040 0450 4005D9 mov %eax,-08x(%rbp) 4005DB popq %rdi 0000 7FFF FFFF E840 0000 7FFF FFFF E948 … %rsp 0000 7FFF FFFF E838 0000 0000 0040 05D9 0000 7FFF FFFF E830 (main’s %rdi value) 0000 7FFF FFFF E828 0000 0010 0000 0010 0000 7FFF FFFF E820 0000 0000 0000 0000 0000 7FFF FFFF E818 0000 0010 0000 0010 0000 7FFF FFFF E810 0000 7FFF FFFF E940 …. Binghamton CS-220 University Spring 2019 Suppose “addem” wants to use %rbx

pushq %rbp movq %rsp, %rbp pushq %rbx int addem(int x, int y) { movl %edi, -20(%rbp) … movl %esi, -24(%rbp) … }

popq %rbx popq %rbp ret Binghamton CS-220 University Spring 2019 (OS) addem’s preamblestack frame Address Value (64 bit) 0000 7FFF FFFF E870 0000 0000 0000 0000 0000 7FFF FFFF E868 0000 7FFF F7A5 2B45 0000 7FFF FFFF E860 0000 7FFF FFFF E88C pushq %rbp main’s 0000 7FFF FFFF E858 0000 0004 0000 0000 movq %rsp, %rbp stack frame 0000 7FFF FFFF E850 0000 0010 FFFF E940 pushq %rbx 0000 7FFF FFFF E848 0000 0002 0040 0450 … %rbp add can use %rbx here 0000 7FFF FFFF E840 0000 7FFF FFFF E948 addem’s 0000 7FFF FFFF E838 0000 0000 0040 05D9 stack frame 0000 7FFF FFFF E830 0000 7FFF FFFF E860 %rsp 0000 7FFF FFFF E828 (main’s %rbx value) 0000 7FFF FFFF E820 0000 0000 0000 0000 the 0000 7FFF FFFF E818 0000 0010 0000 0010 “red zone” 0000 7FFF FFFF E810 0000 7FFF FFFF E940 …. Binghamton CS-220 University Spring 2019 (OS) addem’s return stack frame Address Value (64 bit) 0000 7FFF FFFF E870 0000 0000 0000 0000 main’s 0000 7FFF FFFF E868 0000 7FFF F7A5 2B45 stack frame 0000 7FFF FFFF E860 0000 7FFF FFFF E88C addem can use %rbx here 0000 7FFF FFFF E858 0000 0004 0000 0000 %rbp … 0000 7FFF FFFF E850 0000 0010 FFFF E940 movl -12(%rbp), %eax; save return value 0000 7FFF FFFF E848 0000 0002 0040 0450 popq %rbx addem’s popq %rbp ; restore main’s stack frame 0000 7FFF FFFF E840 0000 7FFF FFFF E948 stack frame ret 0000 7FFF FFFF E838 0000 0000 0040 05D9

%rsp 0000 7FFF FFFF E830 0000 7FFF FFFF E860 0000 7FFF FFFF E828 (main’s %rbx value) 0000 7FFF FFFF E820 0000 0000 0000 0000 the 0000 7FFF FFFF E818 0000 0010 0000 0010 “red zone” 0000 7FFF FFFF E810 0000 7FFF FFFF E940 …. Binghamton CS-220 University Spring 2019 Register Conventions for %rsp/%rbp

• %rbp (stack base pointer) treated just like %rbx… • Callee pushes caller’s %rbp value onto callee’s stack on entry • Callee pops caller’s %rbp value from callee’s stack on return

• %rsp (stack pointer) is a special case • Caller’s %rsp value is callee’s %rbp+8 • To restore caller’s %rsp, just copy %rbp to %rsp before popping %rbp • If %rbp==%rsp, don’t even need to copy • The x86 “leaveq” instruction restores both callers %rbp and %rsp Binghamton CS-220 University Spring 2019 Redundancy?

• If you know the length of each function’s stack frame, keeping a pointer to both the top and the bottom of the stack frame is redundant • gcc parameter “-fomit-frame-pointer” removes use of %rbp, but then all references are relative to %rsp, which can change, so the code works, but becomes confusing Binghamton CS-220 University Spring 2019 Register Usage Summary

%rax Return value %r8 Fifth parameter %rbx %r9 Sixth parameter %rcx Fourth parameter %r10 %rdx Third parameter %r11 %rsp stack & frame low address ptr %r12 %rbp frame high address ptr %r13 %rdi First parameter %r14 %rsi Second parameter %r15

Callee saved/restored Caller saved/restored Binghamton CS-220 University Spring 2019 Caller invocation of Callee

• If you need to maintain value in any of the caller saved (volatile, red) registers, push their values on the stack • Copy parameter values into parameter registers • Push parameters that don’t fit in registers onto stack • Invoke callee with the “callq” instruction • Pushes return address onto stack • Puts address of callee’s first instruction into %rip Binghamton CS-220 University Spring 2019 (OS) In main’s code stack frame Address Value (64 bit) 0000 7FFF FFFF E870 0000 0000 0000 0000 pushq %rip %rbp 0000 7FFF FFFF E868 0000 7FFF F7A5 2B45 jmp addem 0000 7FFF FFFF E860 0000 7FFF FFFF E88C … main’s 0000 7FFF FFFF E858 0000 0004 0000 0000 4005D4 callq addem ; at 400621 stack frame 0000 7FFF FFFF E850 0000 0010 FFFF E940 4005D9 mov %eax,-08x(%rbp) 0000 7FFF FFFF E848 0000 0002 0040 0450 … 0000 7FFF FFFF E840 0000 7FFF FFFF E948 %rsp 0000 7FFF FFFF E838 0000 0000 0040 05D9 0000 7FFF FFFF E830 0000 7FFF FFFF E860 0000 7FFF FFFF E828 0000 0010 0000 0010 0000 7FFF FFFF E820 0000 0000 0000 0000 0000 7FFF FFFF E818 0000 0010 0000 0010 0000 7FFF FFFF E810 0000 7FFF FFFF E940 …. Binghamton CS-220 University Spring 2019 Callee Preamble

• Push caller’s %rbp on stack • Copy %rsp to %rbp – • Creates single entry stack frame that contains only callers %rbp • If not a leaf function, subtract from %rsp (enlarge frame) • If you need to use callee saved (non-volatile, blue) registers other than %rsp and %rbp, push their value on the stack • Copy register argument values from parameter registers • Copy stacked argument values from caller’s stack frame (%ebp+8, 16, 24, etc.) as required Binghamton CS-220 University Spring 2019 (OS) addem’s preamblestack frame Address Value (64 bit) 0000 7FFF FFFF E870 0000 0000 0000 0000 %rbp 0000 7FFF FFFF E868 0000 7FFF F7A5 2B45 0000 7FFF FFFF E860 0000 7FFF FFFF E88C pushq %rbp main’s 0000 7FFF FFFF E858 0000 0004 0000 0000 movq %rsp, %rbp stack frame 0000 7FFF FFFF E850 0000 0010 FFFF E940 … 0000 7FFF FFFF E848 0000 0002 0040 0450 0000 7FFF FFFF E840 0000 7FFF FFFF E948 %rsp 0000 7FFF FFFF E838 0000 0000 0040 05D9 0000 7FFF FFFF E830 0000 7FFF FFFF E860 0000 7FFF FFFF E828 0000 0010 0000 0010 0000 7FFF FFFF E820 0000 0000 0000 0000 0000 7FFF FFFF E818 0000 0010 0000 0010 0000 7FFF FFFF E810 0000 7FFF FFFF E940 …. Binghamton CS-220 University Spring 2019 (OS) addem’s preamblestack frame Address Value (64 bit) 0000 7FFF FFFF E870 0000 0000 0000 0000 0000 7FFF FFFF E868 0000 7FFF F7A5 2B45 0000 7FFF FFFF E860 0000 7FFF FFFF E88C pushq %rbp main’s 0000 7FFF FFFF E858 0000 0004 0000 0000 movq %rsp, %rbp stack frame 0000 7FFF FFFF E850 0000 0010 FFFF E940 … 0000 7FFF FFFF E848 0000 0002 0040 0450 %rbp 0000 7FFF FFFF E840 0000 7FFF FFFF E948 %rsp 0000 7FFF FFFF E838 0000 0000 0040 05D9 add’s 0000 7FFF FFFF E830 0000 7FFF FFFF E860 stack frame 0000 7FFF FFFF E828 0000 0010 0000 0010 0000 7FFF FFFF E820 0000 0000 0000 0000 the 0000 7FFF FFFF E818 0000 0010 0000 0010 “red zone” 0000 7FFF FFFF E810 0000 7FFF FFFF E940 …. Binghamton CS-220 University Spring 2019 Callee’s return

• Put return value in %rax • Pop (restore) all callee saved (non-volatile, blue) registers that were pushed by preamble • Restore caller’s stack frame with either “leave” (or “popq %ebp” if %rsp==%rbp already) • Invoke “ret” instruction to pop return address off of stack, and put it in %rip Binghamton CS-220 University Spring 2019 (OS) return from addemstack frame Address Value (64 bit) 0000 7FFF FFFF E870 0000 0000 0000 0000 %rbp 0000 7FFF FFFF E868 0000 7FFF F7A5 2B45 0000 7FFF FFFF E860 0000 7FFF FFFF E88C movl -12(%rbp), %eax; save return value 0000 7FFF FFFF E858 0000 0004 0000 0000 popq %rbp ; restore main’s stack frame 0000 7FFF FFFF E850 0000 0010 FFFF E940 ret main’s 0000 7FFF FFFF E848 0000 0002 0040 0450 stack frame 0000 7FFF FFFF E840 0000 7FFF FFFF E948 %rsp 0000 7FFF FFFF E838 0000 0000 0040 05D9 0000 7FFF FFFF E830 0000 7FFF FFFF E860 was addem’s 0000 7FFF FFFF E828 0000 0000 0000 0004 stack frame 0000 7FFF FFFF E820 0000 0004 0000 0000 the 0000 7FFF FFFF E818 0000 0000 0000 0004 “red zone” 0000 7FFF FFFF E810 0000 7FFF FFFF E940 …. Binghamton CS-220 University Spring 2019 (OS) return from adddmstack frame Address Value (64 bit) 0000 7FFF FFFF E870 0000 0000 0000 0000 0000 7FFF FFFF E868 0000 7FFF F7A5 2B45 %rbp 0000 7FFF FFFF E860 0000 7FFF FFFF E88C movl -12(%rbp), %eax; save return value 0000 7FFF FFFF E858 0000 0004 0000 0000 popq %rbp ; restore main’s stack frame 0000 7FFF FFFF E850 0000 0010 FFFF E940 ret ; return to main main’s 0000 7FFF FFFF E848 0000 0002 0040 0450 stack frame 0000 7FFF FFFF E840 0000 7FFF FFFF E948 0000 7FFF FFFF E838 0000 0000 0040 05D9 %rsp 0000 7FFF FFFF E830 0000 7FFF FFFF E860 popq %rip 0000 7FFF FFFF E828 0000 0000 0000 0004 0000 7FFF FFFF E820 0000 0004 0000 0000 0000 7FFF FFFF E818 0000 0000 0000 0004 0000 7FFF FFFF E810 0000 7FFF FFFF E940 …. Binghamton CS-220 University Spring 2019 Caller after return

• Save/use return value in %rax • Pop any caller saved (volatile, red) registers pushed before the call Binghamton CS-220 University Spring 2019 Stack Frame Layout Address Value (64 bit) caller’s caller’s %rbp caller’s caller’s stack frame 0000 7FFF FFFF E870 0000 0000 0000 0000 0000 7FFF FFFF E868 0000 7FFF F7A5 2B45 caller’s local values caller’s stack frame 0000 7FFF FFFF E860 0000 7FFF FFFF E88C 0000 7FFF FFFF E858 0000 0004 0000 0000 caller saved reg values 0000 7FFF FFFF E850 0000 0010 FFFF E940 0000 7FFF FFFF E848 0000 0002 0040 0450 %rbp non-register parameters 0000 7FFF FFFF E840 0000 7FFF FFFF E948 return address 0000 7FFF FFFF E838 0000 0000 0040 05D9 my stack frame 0000 7FFF FFFF E830 0000 7FFF FFFF E860 caller’s %rbp 0000 7FFF FFFF E828 0000 0010 0000 0010 callee saved reg values 0000 7FFF FFFF E820 0000 0000 0000 0000 %rsp 0000 7FFF FFFF E818 0000 0010 0000 0010 my local values 0000 7FFF FFFF E810 0000 7FFF FFFF E940 the “red zone” …. Binghamton CS-220 University Spring 2019 Stack Frame Chaining

• %rbp->caller’s %rbp->caller’s caller’s %rbp->… • Chain continues until the “main” function • %rbp when main is entered does NOT point to OS stack frame top

• If you know %rbp, you know where caller’s %rsp is (%rbp+8) • Therefore, if we know %rbp and %rsp, we know the entire stack frame! • This is how gdb “where” works