Florida State University Libraries Electronic Theses, Treatises and Dissertations The Graduate School 2018 Securing Systems by Vulnerability Mitigation and Adaptive Live Patching Yue Chen Follow this and additional works at the DigiNole: FSU's Digital Repository. For more information, please contact [email protected] FLORIDA STATE UNIVERSITY COLLEGE OF ARTS AND SCIENCES SECURING SYSTEMS BY VULNERABILITY MITIGATION AND ADAPTIVE LIVE PATCHING By YUE CHEN A Dissertation submitted to the Department of Computer Science in partial fulfillment of the requirements for the degree of Doctor of Philosophy 2018 Copyright © 2018 Yue Chen. All Rights Reserved. Yue Chen defended this dissertation on January 23, 2018. The members of the supervisory committee were: Zhi Wang Professor Directing Dissertation Ming Yu University Representative Xiuwen Liu Committee Member An-I Andy Wang Committee Member The Graduate School has verified and approved the above-named committee members, and certifies that the dissertation has been approved in accordance with university requirements. ii To my beloved ones. iii ACKNOWLEDGMENTS Pursuing a Ph.D. degree is a unique experience in my life. Here I would like to express my gratitude to a number of people. Without them, I cannot enjoy this wonderful journey. Foremost, I feel incredibly fortunate to have been under Prof. Zhi Wang’s guidance during my Ph.D. study at Florida State. His passion and dedication for research has highly influenced me and opened my eyes to the research world. His encouragement, guidance and support are invaluable power for me to explore the horizons. I have been very lucky to work with several excellent researchers. I want to express my sincere gratitude to the colleagues during my internship at Baidu X-Lab. It is a great pleasure to have the fruitful discussions with Yulong Zhang and Tao Wei to address challenging problems, and their great thoughts and insightful advising have helped me learn a lot. It is also an enjoyable and instrumental experience to work with other colleagues: Zhaofeng Chen, Zhenyu Zhong, Yu Ding; as well as other interns in the lab: Pei Wang and Peng Wang. I am grateful to all the collaborators for their insightful ideas and helpful discussions. In par- ticular, special thanks are given to Xiaoguang Wang, Ryan Baird, David Whalley and Yajin Zhou for the helpful discussions and suggestions with their precious and profound understanding of the research topics. In the computer science department of FSU, I regularly participate in Prof. An-I Andy Wang’s research group meeting. It is a great opportunity to discuss and learn topics in computer systems. I want to express my gratitude to An-I Andy Wang and his students for great suggestions about my research and the great discussion atmosphere. I would also like to thank the rest of my dissertation committee: Prof. Xiuwen Liu and Prof. Ming Yu, for their detailed advice, comments and suggestions. Last but certainly not least, I owe a big debt of gratitude to my parents and family, who support every decision I have made, including the pursuit of this degree. iv TABLE OF CONTENTS List of Tables . viii List of Figures . ix Abstract . xi 1 Introduction 1 1.1 Problem Overview . .1 1.2 Our Approach . .3 1.3 Summary of Contributions . .5 1.4 Dissertation Organization . .5 2 Related Work 6 2.1 Memory Vulnerabilities and Exploits . .6 2.1.1 Buffer Overflow . .6 2.1.2 Information Leakage . .6 2.1.3 NULL Pointer Dereference . .7 2.1.4 Arbitrary Format String . .7 2.1.5 Use-After-Free . .7 2.1.6 Data-only Attack . .7 2.1.7 Return-oriented Programming . .8 2.2 Threat Mitigation . .8 2.2.1 Data Execution Prevention . .8 2.2.2 Software Diversity . .8 2.2.3 ROP Defenses . 10 2.2.4 Control-flow Integrity . 10 2.3 Root-Cause Analysis . 11 2.3.1 Attack/Exploit Detection and Mitigation . 11 2.3.2 Vulnerability/Bug Discovery . 12 2.3.3 Record & Replay . 12 2.4 Patch Generation . 13 2.4.1 Kernel Live Patching . 13 2.4.2 Semantic Matching . 14 2.4.3 Automatic Patch/Filter Generation . 14 3 On-demand Live Randomization 16 3.1 Introduction . 16 3.2 Design . 19 3.2.1 Overview . 19 3.2.2 Basic Block Reordering . 21 3.2.3 Basic Block Pointer Conversion . 24 3.2.4 Live Randomization of Kernel Modules . 27 3.2.5 Performance Optimization . 28 v 3.2.6 Binary-only Program Support . 29 3.3 Implementation . 30 3.4 Evaluation . 32 3.4.1 Security . 32 3.4.2 Performance . 35 3.5 Discussion . 36 3.6 Summary . 37 4 Pinpointing Vulnerabilities 39 4.1 Introduction . 39 4.2 System Overview . 40 4.3 System Design . 41 4.3.1 System Overview . 41 4.3.2 Attack Detection . 44 4.3.3 Record and Replay . 45 4.3.4 Pinpointing Vulnerabilities . 49 4.3.5 Prototype Efforts . 53 4.4 Evaluation . 53 4.4.1 Effectiveness . 53 4.4.2 Performance . 61 4.5 Discussion . 62 4.6 Summary . 63 5 Adaptive Android Kernel Live Patching 64 5.1 Introduction . 64 5.2 System Design . 67 5.2.1 Measuring Android Fragmentation . 67 5.2.2 Adaptive Multi-level Patching . 70 5.2.3 Architecture and Workflow . 71 5.2.4 KARMA Patches . 73 5.2.5 Offline Patch Adaptation . 76 5.2.6 Live Patching . 79 5.2.7 Prototype of KARMA . 82 5.3 Evaluation . 82 5.3.1 Evaluation of Applicability . 82 5.3.2 Evaluation of Adaptability . 84 5.3.3 Evaluation of Performance . 87 5.4 Discussion and Future Work . 89 5.5 Summary . 91 6 Conclusion 92 vi Appendix A KARMA Patch Writing for Recent Kernel Vulnerabilities 93 Bibliography . 97 Biographical Sketch . 111 vii LIST OF TABLES 3.1 Average NOP Space per Function . 29 3.2 Statistics of Three Web Servers . 32 4.1 Summary of the evaluation results on a number of DARPA CGC programs. 57 5.1 Devices vulnerable to two infamous root exploits as of Nov. 2016. The second column lists the dates when they are disclosed in Android Security Advisory. 65 5.2 Images obtained from popular devices. 67 5.3 Statistics of the obtained Android kernels. 68 5.4 The extension to Lua. The first five functions can only be used by the live patcher, not by patches. 79 5.5 Clustering 1; 139 kernels for each function by syntax and semantics. The last-but-two column lists the time of semantic matching to compare Nexus 5 (Android 4.4.2, kernel 3.4.0) and Samsung.