A Taxonomic Evaluation of Rootkit Deployment, Behavior and Detection
Total Page:16
File Type:pdf, Size:1020Kb
A Taxonomic Evaluation of Rootkit Deployment, Behavior and Detection A Thesis Presented in Partial Fulfillment of the Requirements for the Degree of Master of Science with a Major in Computer Science in the College of Graduate Studies University of Idaho by Maxine Major Major Professor: Jim Alves-Foss, Ph.D. Committee Members: Daniel Conte de Leon, Ph.D.; Sara Eftekharnejad, Ph.D. Department Administrator: Gregory Donohoe, Ph.D. July 2015 ii Authorization to Submit Thesis This Thesis of Maxine Major, submitted for the degree of Master of Science with a ma- jor in Computer Science and titled \A Taxonomic Evaluation of Rootkit Deployment, Behavior and Detection", has been reviewed in final form. Permission, as indicated by the signatures and dates given below, is now granted to submit final copies to the College of Graduate Studies for approval. Major Professor Date Jim Alves-Foss, Ph.D. Committee Members Date Daniel Conte de Leon, Ph.D. Date Sara Eftekharnejad, Ph.D. Computer Science Department Administrator Date Gregory Donohoe, Ph.D. iii Abstract Increased inter-connectivity between cyber and cyber-physical systems increases the danger of Advanced Persistent Threat (APT) cyber attacks, against which perimeter-focused defenses are no longer sufficient. Rootkits are debatably the most important piece of malicious software to the success of an APT. Rootkits are are often planted through social engineering, which intend to bypass perimeter{focused defenses. APTs, the most dangerous of cyber attacks, is facilitated by one of the least-detected attack methods. In order to further the practice of detecting rootkits and aid with early detection, this thesis presents a taxonomy of rootkit activities through each stage of installation and exploita- tion. Correspondingly, this thesis presents a taxonomy of rootkit detection methods to address rootkit infection vectors. These taxonomies are then applied to a real-world rootkit example to demonstrate how combined application of rootkit detection tools and techniques can provide full-coverage of the possible rootkit-targeted attack surface. iv Acknowledgments I would first like to thank my advisor, Dr. Jim Alves-Foss, for his support, encouragement, and patient guidance throughout my graduate studies. I would also like to thank my other committee members, Dr. Conte de Leon, and Dr. Sara Eftekharne- jad, for their valuable input and comments on my thesis. I would like to thank Dr. Paul Oman for launching my interest in cyber security, and structuring courses which encouraged curiosity and allowed me to explore. I would like to thank all my instructors for their hard work and dedication in providing me with a comprehensive and valuable education. I would like to thank the department chair, Dr. Gregory Donohoe, and Mrs. Arvilla Daf- fin and other staff in the Department of Computer Science and the Center for Secure and Dependable Systems for their help during my study in the department. I wish to acknowledge the National Science Foundation CyberCorps R Scholarship for Ser- vice, for supporting me during the course of my graduate studies. Last, but certainly not least, I would like to thank my family and all the friends I have met while at University of Idaho for their support, encouragement, and love which have helped me make this thesis a reality. v Table of Contents Authorization to Submit Thesis .............................. ii Abstract ............................................. iii Acknowledgments ....................................... iv List of Figures ......................................... ix List of Tables .......................................... x 1 Introduction ........................................ 1 1.1 The Problem Space . 1 1.1.1 Advanced Persistent Threats . 1 1.1.2 Rootkits . 2 1.1.3 The Practice of Rootkit Detection . 3 1.2 Motivation . 4 1.3 Objectives . 4 1.4 Thesis Impact . 5 1.5 Thesis Overview . 6 2 Background ......................................... 8 2.1 Advanced Persistent Threats . 8 2.1.1 APT Goals . 8 2.1.2 APT Persistence . 9 2.1.3 APT Launch and Detection . 9 2.1.4 Famous APTs . 12 2.2 The Kill Chain . 14 2.3 Perimeter-Focused Defenses . 16 3 Rootkit Preparation and Delivery ........................... 18 3.1 Rootkit Overview . 19 vi 3.1.1 Rootkit Kill Chain . 20 3.1.2 Breakdown of Rootkit Activities . 21 3.2 Reconnaissance . 22 3.2.1 Cyber Reconnaissance . 22 3.2.2 Social Reconnaissance . 23 3.3 Weaponization . 25 3.3.1 Delivery Considerations . 25 3.3.2 Exploits and Payloads . 31 3.3.3 Exploit Kits . 32 3.3.4 Obfuscation . 33 3.4 Rootkit Delivery . 35 3.4.1 Active vs. Passive Delivery . 36 3.4.2 Physical vs. Cyber Delivery . 38 4 Kernel{mode Rootkits .................................. 42 4.1 Rootkit Installation and Exploitation . 43 4.1.1 Operational Goals . 44 4.1.2 The Kernel . 45 4.1.3 Kernel Structures . 48 4.1.4 Installation Techniques . 51 4.1.5 Installation Process . 59 4.1.6 Rootkit Privilege Escalation . 60 4.1.7 Rootkit Persistence . 62 4.1.8 Rootkit Hiding Methods . 65 4.2 Backdoor Creation . 72 4.2.1 Backdoor Implementations . 72 4.2.2 C2 Objectives . 75 4.2.3 C2 Communication . 77 4.3 Rootkit Actions . 79 4.3.1 Disable Security Services . 80 4.3.2 Malware Dissemination . 81 4.3.3 Information Capture . 81 vii 4.4 Specialized Rootkits . 82 4.4.1 Advanced Rootkits . 82 4.4.2 SCADA/ICS Rootkits . 83 4.5 Summary of Rootkit Activities . 85 5 Rootkit Detection ..................................... 87 5.1 Rootkit Detection Methods Overview . 88 5.1.1 Detection Technique Categorization . 88 5.1.2 Detection Metrics and Considerations . 90 5.1.3 Detection Method Constraints . 92 5.2 Static Rootkit Detection Techniques . 93 5.2.1 Signatures . 94 5.2.2 Static Heuristics . 96 5.2.3 Static Memory Forensics and Mapping . 98 5.3 Dynamic Rootkit Detection Techniques . 99 5.3.1 Dynamic Behavior Analysis . 100 5.3.2 Crossview Detection . 102 5.3.3 Dynamic Memory Forensics and Mapping . 103 5.4 Detection Execution Platform . 104 5.4.1 Local Execution . 105 5.4.2 Virtualization . 106 5.4.3 Hardware . 107 5.5 Rootkit Detection Methods Evaluation . 108 5.5.1 Coverage Evaluation . 109 5.5.2 Conclusions . 110 6 Application .........................................113 6.1 The ZeroAccess Rootkit . 113 6.1.1 ZeroAccess Background . 113 6.1.2 ZeroAccess Weaponization and Delivery . 113 6.1.3 ZeroAccess Installation . 114 6.1.4 Privilege Escalation . 115 6.1.5 C2 / Botnet . 115 viii 6.1.6 Additional Activities . 116 6.2 Taxonomic Evaluation of the ZeroAccess Rootkit . 117 6.3 Application Conclusions . 120 7 Conclusions and Future Work .............................122 7.1 Summary of Work . 122 7.1.1 Research Summary . 122 7.1.2 Contribution . 126 7.2 Future Work . 127 7.2.1 Taxonomy Expansion . 128 7.2.2 Tool Evaluation . 129 Bibliography and References ................................129 Appendix A: Acronyms ...................................139 Appendix B: Compilation of Taxonomies ........................141 ix List of Figures Figure 2.1 Kill Chain . 14 Figure 3.1 Stages of the Rootkit Kill Chain . 20 Figure 3.2 Rootkit Objectives as Related to the Kill Chain . ..