CS216: Program and Data Representation University of Virginia Computer Science Spring 2006 David Evans Menu Lecture 21: Calling Conventions • x86 C Calling Convention • Java Byte Code Wizards http://www.cs.virginia.edu/cs216 UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 2 Calling Convention Calling Convention • Rules for how the caller and the Caller Callee callee make subroutine calls • Where to put • Where to find • Caller needs to know: params params • What registers can I • What registers – How to pass parameters assume are same must I not bash – What registers might be bashed by • Where to find result • Where to put result called function • State of stack • State of stack – What is state of stack after return – Where to get result UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 3 UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 4 Calling Convention: x86 C Calling Convention Easiest for Caller Caller Callee Caller Callee • Where to put • Where to find • Params on stack in • Find params on stack params params reverse order in reverse order • What registers can I • What registers • Can assume EBX, • Must not bash EBX, assume are same must I not bash EDI and ESI are EDI or ESI – All of them – None of them same • Put result in EAX • Where to find result • Where to put result • Find result in EAX • Stack rules next • State of stack • State of stack • Stack rules next Need to save and Need to save and restore restore EBX, EDI (EAX), ECX, EDX and ESI UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 5 UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 6 1 saved ESI Stack Convention StackGrowth saved EDI • EBP: Base Pointer ESP – Reference point for finding local variables local variable 3 and parameters local variable 2 – Manipulated only explicitly local variable 1 [ebp]-4 • ESP: Stack Pointer saved EBP – Pointer to last element used on the stack Addresses Higher return address • The next free element is at [esp]-4 EBP – Manipulated implicitly by many parameter 1 [ebp]+8 instructions: push , pop , call , ret parameter 2 [ebp]+12 Do we need both of these? parameter 3 [ebp]+16 UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 7 UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 8 saved ESI ESP StackGrowth saved EDI Maintaining EBP local variable 3 In which direction • Callee: prologue local variable 2 should the stack – Must save old value of EBP: push ebp local variable 1 grow? – Update EBP to point to the new frame: saved EBP mov EBP, ESP Higher Addresses Higher If first local return address variable – Use EBP to find parameters: [ebp]+8 , [ebp]+12 , … parameter 1 is int a[1] • Callee: before returning parameter 2 what is – Restore old value of EBP: pop ebp parameter 3 a[2] ? UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 9 UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 10 Maintaining ESP Caller Rules • Caller: it is maintained implicitly if • Caller Before: • Caller After: parameters are push-ed 1. Save bashable 2. Restore bashable • Callee: prologue (“caller-save”) registers registers we use – Need to make enough space for local 2. Push parameters variables: 1. Remove on stack parameters from sub esp, <space needed> 3. call <routine> stack • Callee: before return implicit: push EIP – Deallocate local variables from stack: mov esp, ebp ; must be done before pop ebp UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 11 UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 12 2 Callee Rules To work with library code, • Callee Prologue: • Callee Return: compiler- 5. ret generated code, 1. Save EBP 4. Restore EBP must follow all 2. Update EBP rules carefully 3. Allocate local 3. Deallocate local variables ( ESP ) variables ( ESP ) 4. Save callee-save 2. Restore callee- If you don’t care registers we use save registers about this, which (EBX, EDI , ESI ) 1. Leave return value rules must you in EAX follow? Which of the callee prologue steps could be done by caller? UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 13 UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 14 Byte Code Wizardry Awards Mystery.java • Implementation of SHA-1 cryptographic hash algorithm – Code posted on CS216 website now private static int obfuscate(int num, int cnt) { int a = num; int b = cnt; for (int i = 0; i < 216; i += 3) { b = b + a + i; b = b - b; } if (b == 0) { return num; } else { for (int i = 0; i < 256; i++) { a = a - 1; } return a; } } UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 15 UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 16 “Black Hat” Fastest Code Honorable Mentions Call the • Not enough variance to distinguish Tester.generate() • Effectively tied for fastest: method – Bin Zhou, Mark Mitchell – Bijan Vakili – David Trang Richard Hsu, Dan Chen 244 – Jake Fowler and Sean Talts Michael Thomas, Billy Chantree 244 – Michael Thomas and Billy Chantree Mike Liu 316 – Eric Bradbury and Robert Yip Jongmin Kim 380 UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 17 UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 18 3 Smallest Code Smallest Code Original Mystery.class 1547 bytes Jake Fowler and Sean Talts 1015 bytes Removing obfuscate 1300 bytes Michael Thomas and Billy Chantree 1021 bytes Removing line numbers 1021 bytes Will Ashford and Mike Dietz 1061 bytes Improving logic 1015 bytes Replacing operations 1061 bytes No others below 1300 bytes with calls to Integer.rotateLeft Being the Byte Code Wizard priceless UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 19 UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 20 Prizes Charge • “Byte Code Wizards”: Jake Fowler and Sean Talts • Selection of books (members of 3 teams pick • PS7 Due Monday in order) – Lots of interesting things to explore – John Battelle, “The Search” (Google) – Richard Feynman, “All the Adventures...” – Don’t wait to make progress on it – Paul Graham, “Hackers & Painters” • “x86 Guru” title(s) may be awarded – Greg Hoglung & McGraw, “Exploiting Software” for especially clever answers to – Gary McGraw, “Software Security” Question 10 – Simon Singh, “The Code Book” • Black Hat awards: Sumatra (Java’s big sister island) Coffee UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 21 UVa CS216 Spring 2006 - Lecture 21: Calling Conventions 22 4.