Preventing Exploits Against Memory Corruption Vulnerabilities
Total Page:16
File Type:pdf, Size:1020Kb
PREVENTING EXPLOITS AGAINST MEMORY CORRUPTION VULNERABILITIES A Thesis Presented to The Academic Faculty by Chengyu Song In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the School of Computer Science Georgia Institute of Technology August 2016 Copyright ⃝c 2016 by Chengyu Song PREVENTING EXPLOITS AGAINST MEMORY CORRUPTION VULNERABILITIES Approved by: Professor Wenke Lee, Advisor Professor Mustaque Ahamad School of Computer Science School of Computer Science Georgia Institute of Technology Georgia Institute of Technology Professor Taesoo Kim, Co-Advisor Dr. Weidong Cui School of Computer Science Security and Privacy Research Group Georgia Institute of Technology Microsoft Research Professor William R. Harris Date Approved: 13 July 2016 School of Computer Science Georgia Institute of Technology To my dear wife, and my parents, for all the love and support. iii ACKNOWLEDGEMENTS This work owes its existence to the support provided by a large number of people over a time span of several years, and I would like to acknowledge them. First of all, I would like to thank my terrific advisor Professor Wenke Lee, who has trained me to be an independent researcher. Wenke has always encouraged me to pursue my own interests and provided me the freedom and resources to do so. At the same time, he has also taught me the criteria of good research and the skills of a professional researcher. I am also extreme grateful to my co-advisor Professor Taesoo Kim. Working with him has always been enlightening and inspiring. I would also like to acknowledge my thesis committee members: Professor Bill Harris, Dr. Weidong Cui, and Professor Mustaque Ahamad for willing to serve on my thesis committee. Their insightful comments and suggestions have helped me make significant improvements to this thesis. This thesis would not be possible without the help my dear collaborators. Among them, I would like to give special thanks to Dr. Byoungyoung Lee, who has been a good friend and our collaboration has been fruitful. I have also been fortunate to work with the following brilliant researchers: Professor Bill Harris, Dr. Tielei Wang, Dr. Chao Zhang, Paul Royal, Dr. Simon Chung, Dr. Changwoo Min, Dr. Weidong Cui, Mr. Marcus Peinado, Dr. David Melski, Dr. Himanshu Raj, Yeongjin Jang, Kangjie Lu, Insu Yun, Billy Lau, Hyungon Moon, and Professor Alexandra Boldyreva. I also want to take this opportunity to thank my lab-mates, without them my Ph.D. would not have been the same. I especially would like to acknowledge the following who has been helpful with my research: Monirul Sharif, Junjie Zhang, Long Lu, Xinyu Xing, Brendan Dolan-Gavit, Sangho Lee, Meng Xu, Chenxiong Qian, Ren Ding, and Ming-wei Shih. Last but not least, I would like to give a special thanks to my family for their continuous iv love and support throughout all these years that I have been away from home. The decision to leave my home country and come to the United States to pursue a doctorate was not easy for either of us, but you have always encouraged me to aim high and pursue my dreams. And even though thousands of miles keep us apart, I always carry you in my heart and it gives me strength. v TABLE OF CONTENTS DEDICATION ...................................... iii ACKNOWLEDGEMENTS .............................. iv LIST OF TABLES ................................... xi LIST OF FIGURES .................................. xii SUMMARY ........................................ xiii I INTRODUCTION .................................. 1 1.1 Problem Statement and Our Approach....................2 1.2 Thesis Contributions..............................4 1.3 Thesis Outline..................................5 II BACKGROUND .................................. 6 2.1 Memory Corruption Vulnerabilities......................6 2.2 Exploit Techniques...............................6 2.3 Existing Defense Mechanisms.........................8 2.3.1 Exploit Prevention...........................8 2.3.2 Memory Error Detection........................ 12 III PREVENTING CODE INJECTION ATTACKS AGAINST DYNAMIC CODE GENERATOR ............................... 15 3.1 Motivation.................................... 15 3.2 Assumptions and Threat Model........................ 17 3.3 Related Work.................................. 19 3.3.1 Software-based Fault Isolation..................... 19 3.3.2 Memory Safety............................. 20 3.3.3 Control Flow Integrity.......................... 21 3.3.4 Process Sandbox............................. 21 3.3.5 Attacks on JIT engines......................... 22 3.4 Attacking the Code Cache........................... 22 3.4.1 Code Cache Injection Attacks..................... 22 3.4.2 Exploiting Race Conditions...................... 24 vi 3.5 System Design.................................. 28 3.5.1 Overview and Challenges........................ 29 3.5.2 Memory Map Synchronization..................... 30 3.5.3 Remote Procedure Call.......................... 31 3.5.4 Permission Enforcement........................ 32 3.5.5 Security Analysis............................ 33 3.6 Implementation................................. 33 3.6.1 Shared Infrastructure.......................... 33 3.6.2 SDT Specific Handling......................... 35 3.7 Evaluation.................................... 38 3.7.1 Setup................................... 38 3.7.2 Effectiveness............................... 38 3.7.3 Micro Benchmark............................ 39 3.7.4 Macro Benchmark............................. 41 3.8 Limitations and Future Work......................... 44 3.8.1 Reliability of Race Condition..................... 45 3.8.2 RPC Stub Generation......................... 45 3.8.3 Performance Tuning.......................... 46 3.9 Summary..................................... 47 IV PREVENTING KERNEL PRIVILEGE ESCALATION ATTACKS WITH DATA-FLOW INTEGRITY ........................... 48 4.1 Motivation.................................... 48 4.2 Threat Model and Assumptions......................... 51 4.3 Related work.................................... 51 4.3.1 Kernel Integrity............................. 52 4.3.2 Software Fault Isolation........................ 52 4.3.3 Data-flow Integrity........................... 52 4.3.4 Dynamic Taint Analysis........................ 53 4.3.5 Memory Safety............................. 53 4.3.6 Control-flow Integrity.......................... 53 4.4 Demonstration Attacks............................. 54 vii 4.4.1 Simple rooting attacks......................... 54 4.4.2 Bypassing CFI with non-control-data attacks............ 54 4.4.3 Bypassing CFI with control-data attacks............... 55 4.4.4 Diversity of non-control-data attacks................. 56 4.5 Technical Approach............................... 56 4.5.1 Inferring Distinguishing Regions.................... 57 4.5.2 Protecting Distinguishing Regions................... 60 4.6 Formal Model.................................. 63 4.6.1 Problem Definition........................... 63 4.6.2 Inferring distinguishing regions.................... 65 4.6.3 Protecting distinguishing regions................... 68 4.6.4 Protected monitors as refinements................... 69 4.7 A Prototype for Android............................ 70 4.7.1 Data-flow Isolation............................ 71 4.7.2 MMU Integrity............................. 73 4.7.3 Shadow Objects............................. 74 4.7.4 Kernel Stack Randomization...................... 76 4.8 Evaluation.................................... 77 4.8.1 Experimental setup........................... 77 4.8.2 Distinguishing Regions Discovery................... 78 4.8.3 Security Evaluation........................... 80 4.8.4 Performance Evaluation........................ 80 4.9 Limitations and Future Work......................... 84 4.9.1 Cross-platform.............................. 84 4.9.2 Better architecture support...................... 84 4.9.3 Reliability of assumptions....................... 85 4.9.4 Use-after-free.............................. 85 4.9.5 DMA protection............................. 85 4.10 Summary..................................... 85 V IMPROVE SECURITY AND PERFORMANCE WITH HARDWARE- ASSISTED DATA-FLOW ISOLATION ................... 87 viii 5.1 Motivation.................................... 87 5.2 Threat Model and Assumptions........................ 90 5.3 Background and Related Work......................... 90 5.3.1 Data-flow Integrity........................... 90 5.3.2 Tag-based Memory Protection...................... 91 5.3.3 Tag-based Hardware.......................... 93 5.3.4 Memory Safety............................. 93 5.4 HDFI Architecture............................... 94 5.4.1 ISA Extension.............................. 94 5.4.2 Memory Tagger............................. 95 5.4.3 Optimizations.............................. 97 5.4.4 Protecting the Tag Tables....................... 98 5.5 Security Applications.............................. 98 5.5.1 Shadow Stack.............................. 99 5.5.2 Standard Library Enhancement.................... 100 5.5.3 VTable Pointer Protection....................... 102 5.5.4 Code Pointer Separation........................ 103 5.5.5 Kernel Protection............................ 104 5.5.6 Information Leak............................ 104 5.6 Implementation................................. 105 5.6.1 Hardware................................ 105 5.6.2 Software Support............................ 109 5.6.3 Security Applications.......................... 109 5.6.4 Synthesized