Linux Kernel, Wherein Attackers Can Use Malware to Root the System
Total Page:16
File Type:pdf, Size:1020Kb
DETECTION AND PREVENTION OF ANDROID MALWARE ATTEMPTING TO ROOT THE DEVICE THESIS Justin R. Ball, Captain, USAF AFIT-ENG-14-M-08 DEPARTMENT OF THE AIR FORCE AIR UNIVERSITY AIR FORCE INSTITUTE OF TECHNOLOGY Wright-Patterson Air Force Base, Ohio DISTRIBUTION STATEMENT A: APPROVED FOR PUBLIC RELEASE; DISTRIBUTION UNLIMITED The views expressed in this thesis are those of the author and do not reflect the official policy or position of the United States Air Force, the Department of Defense, or the United States Government. This material is declared a work of the U.S. Government and is not subject to copyright protection in the United States. AFIT-ENG-14-M-08 DETECTION AND PREVENTION OF ANDROID MALWARE ATTEMPTING TO ROOT THE DEVICE THESIS Presented to the Faculty Department of Electrical and Computer Engineering Graduate School of Engineering and Management Air Force Institute of Technology Air University Air Education and Training Command in Partial Fulfillment of the Requirements for the Degree of Master of Science in Cyberspace Operations Justin R. Ball, B.S.C.S. Captain, USAF March 2014 DISTRIBUTION STATEMENT A: APPROVED FOR PUBLIC RELEASE; DISTRIBUTION UNLIMITED AFIT-ENG-14-M-08 DETECTION AND PREVENTION OF ANDROID MALWARE ATTEMPTING TO ROOT THE DEVICE Justin R. Ball, B.S.C.S. Captain, USAF Approved: //signed// 14 Feb 2014 Maj Thomas E. Dube, PhD (Chairman) Date //signed// 14 Feb 2014 Barry E. Mullins, PhD (Member) Date //signed// 14 Feb 2014 COL Grant A. Jacoby, PhD (Member) Date AFIT-ENG-14-M-08 Abstract Every year, malefactors continue to target the Android operating system. Malware which root the device pose the greatest threat to users. The attacker could steal stored passwords and contact lists or gain remote control of the phone. Android users require a system to detect the operation of malware trying to root the phone. This research aims to detect the Exploid, RageAgainstTheCage, and Gingerbreak exploits on Android operating systems. Reverse-engineering 21 malware samples lead to the discovery of two critical paths in the Android Linux kernel, wherein attackers can use malware to root the system. By placing sensors inside the critical paths, the research detected all 379 malware samples trying the root the system. Moreover, the experiment tested 16,577 benign applications from the Official Android Market and third party Chinese markets which triggered zero false positive results. Unlike static signature detection at the application level, this research provides dynamic detection at the kernel level. The sensors reside in-line with the kernel’s source code, monitoring network sockets and process creation. Additionally, the research demonstrates the steps required to reverse engineer Android malware in order to discover future critical paths. Using the kernel resources, the two sensors demonstrate efficient asymptotic time and space real-world monitoring. Furthermore, the sensors are immune to obfuscation techniques such as repackaging. iv Table of Contents Page Abstract ......................................... iv Table of Contents .................................... v List of Figures ......................................viii List of Tables ...................................... ix I. Introduction ..................................... 1 1.1 Background .................................. 1 1.2 Research Contributions ............................ 2 1.3 Assumptions/Limitations ........................... 3 1.4Preview.................................... 3 II. Background ..................................... 4 2.1 Introduction .................................. 4 2.2 Offensive Techniques (Rootkits) ....................... 4 2.2.1 Enhancing Stealthiness And Efficiency of Android Trojans and Defense Possibilities (EnSEAD) ................... 4 2.2.2 Android platform Based Linux Kernel Rootkit ........... 5 2.3 Defensive Techniques ............................. 6 2.3.1 Static Analysis ............................ 7 2.3.1.1 Extending Android Security Enforcement with a Secu- rity Distance Model .................... 7 2.3.1.2 TrustDroid (TM) ...................... 8 2.3.1.3 Semantically Rich Application-centric Security in Android 9 2.3.2 Dynamic Analysis .......................... 10 2.3.2.1 YAASE: Yet Another Android Security Extension .... 11 2.3.2.2 Security controls for Android ............... 12 2.3.2.3 Kernel-based Behavior Analysis for Android Malware Detection .......................... 13 2.3.2.4 Detecting covert communication on Android ....... 14 2.3.2.5 A Cloud-Based Intrusion Detection System For Android Smartphones ........................ 15 2.3.2.6 A Cloud-Based Intrusion Detection And Response Sys- tem For Mobile Phones .................. 16 v Page 2.3.2.7 Android Malware Detection via a Latent Network Be- havior Analysis ...................... 16 2.4Surveys.................................... 17 2.4.1 Google Android: A Comprehensive Security Assessment ...... 17 2.4.2 Dissecting Android Malware: Characterization and Evolution . 18 2.4.3 My smartphone is a safe! The user’s point of view regarding novel authentication methods and gradual security levels on smartphones 20 2.4.4 Smartphone Security Challenges ................... 22 2.4.5 Android botnets on the rise: Trends and characteristics ....... 23 2.5 Summary ................................... 24 III. Methodology .................................... 26 3.1 Introduction .................................. 26 3.2 Contribution .................................. 26 3.3 Approach ................................... 26 3.3.1 Develop Kernel Sensors Inside Critical Paths ............ 26 3.3.2 Add Sensors Inside Kernel Source Code ............... 27 3.3.3 Android Operating System ...................... 28 3.4 Testing ..................................... 29 3.5 Performance Metrics ............................. 30 3.6 Boundaries .................................. 32 3.7 Workload ................................... 33 3.8 Summary ................................... 33 IV. Reverse-Engineering Analysis ........................... 34 4.1 Introduction .................................. 34 4.2 Research Contribution ............................ 34 4.3 Reverse Engineering Malware ........................ 34 4.3.1 Android Malware Genome Project .................. 35 4.3.2 Reversing Approach ......................... 35 4.4 Gingermaster Android Malware Family .................. 35 4.4.1 Behavioral Analysis Approach and Findings ............ 36 4.4.2 Static Analysis Approach and Findings ............... 37 4.5 zHash Android Malware Family ....................... 40 4.6 DroidKungFu3 Android Malware Family .................. 43 4.7 Summary of Findings ............................. 45 4.8 Malware Detection Set-Up .......................... 48 4.8.1 Create Android Emulator Using Android Virtual Device Manager . 48 4.8.2 Installing the Sensors ......................... 49 4.8.3 Speed of the Sensors ......................... 49 vi Page 4.8.4 Testing Procedures .......................... 51 4.9 Summary ................................... 52 V. Detection Analysis ................................. 53 5.1 Introduction .................................. 53 5.2 Gingerbreak ................................. 53 5.3 zHash ..................................... 54 5.4 DroidKungFu ................................. 56 5.5 False Positives ................................. 59 5.6 Accuracy ................................... 59 5.7 Advantage and Disadvantage of the Proposed Detection Algorithm ..... 60 5.8 Summary ................................... 61 VI. Conclusion ..................................... 62 6.1 Introduction .................................. 62 6.2 Summary of Research ............................. 62 6.3 Recommendations for Future Research .................... 63 6.4 Research Contributions ............................ 63 Appendix: Tools Needed ................................ 65 Appendix: Scripts Used ................................. 67 Appendix: Sensor code inside the kernel ........................ 75 Appendix: Original malware signature paths ...................... 80 Appendix: Sequence Diagrams ............................. 86 Bibliography ...................................... 89 vii List of Figures Figure Page 2.1 Policy Tree for Saint [33] ............................. 10 3.1 Different Layers of the Android Operating System [2] .............. 27 3.2 Testing Flowchart ................................. 31 4.1 Evolution of DroidKungFu Malware[22].................... 43 4.2 Summary of the three malware families ..................... 46 5.1 Gingerbreak binary detection .......................... 53 5.2 Gingermaster detection results against the different payloads ......... 54 5.3 2011 Gingermaster detection results from four antivirus software [20] .... 54 5.4 zHash binary detection .............................. 55 5.5 2011 zHash Detection Results From Four Antivirus Software [20] ....... 55 5.6 zHash detection results against the different payloads .............. 56 5.7 DroidKungFu binary detection .......................... 57 5.8 Source code change inside /system/core/adb.c:adb main() ........ 57 5.9 DroidKungFu detection results against the different payloads .......... 58 5.10 2011 DroidKungFu detection results from four antivirus software [20] ..... 59 E.1 Sequence Diagram of Exploid exploit ...................... 86 E.2 Sequence Diagram of Gingerbreak exploit ................... 87 E.3 Sequence Diagram of RATC exploit ........................ 88 viii List of Tables Table Page 2.1 Summary