DOCSLIB.ORG

Is It Possible to Reverse Engineer Obfuscated Bytecode Back to Source Code?

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Is It Possible to Reverse Engineer Obfuscated Bytecode Back to Source Code? EXAMENSARBETE INOM TEKNIK, GRUNDNIVÅ, 15 HP STOCKHOLM, SVERIGE 2020 Is it possible to reverse engineer obfuscated bytecode back to source code? Är det möjligt att dekompilera obfuskerad bytekod tillbaka till källkod? GUSTAV SMEDBERG JENNY MALMGREN KTH SKOLAN FÖR KEMI, BIOTEKNOLOGI OCH HÄLSA Is it possible to reverse engineer obfuscated bytecode back to source code? Är det möjligt att dekompilera obfuskerad bytekod tillbaka till källkod? Gustav Smedberg Jenny Malmgren Examensarbete inom Datateknik Grundnivå, 15 hp Handledare på KTH: Anders Cajander Examinator: Ibrahim Orhan TRITA-CBH-GRU-2020:052 KTH Skolan för kemi, bioteknologi och hälsa 141 52 Huddinge, Sverige Sammanfattning Det finns mycket gammal mjukvara ute i världen som inte längre underhålls och skulle behöva uppdateras för att kunna täppa säkerhetshål alternativt uppdatera funktioner i dessa program. I fall där källkoden har förlorats eller raderats , skulle det då vara möjligt att använda dekompilering för att återfå källkoden? Rapport syftar till att redovisa vad java bytekod är och hur den används samt hur man kan gå från java bytekod tillbaka till källkod genom en process som kallas dekompilering samt hur man kan skydda sig mot detta genom obfus- kering av kod. Vidare redovisas tidigare forskning inom dekompilering samt obfuskering och kompletterar med förklaringar vad en Java Virtual Machine, Bytekod och obfuskering är och hur de fungerar. Tre program av varierande komplexitet görs om till bytekod, obfuskeras för att sedan dekompileras och jämföra resultatet gentemot källkoden. Slutligen, det är möjligt att dekompilera den obfuskerade koden men enbart vissa delar av källkoden går att återskapa. Alla variabelnamn och oanvända metoder försvinner helt samt att koden ibland ändras till icke-konventionella sätt att programmera. Nyckelord Reverse engineering, Java, JVM, bytecode, obfuskering, dekompilering, sä- kerhet. Abstract There are a lot of old software in the world that has not been supported or kept up to date and would need to be updated to seal security vulnerabilities, as well as to up- date functions in the program. In those cases where the source code has been lost or deliberately deleted, would it be possible to use reverse engineering to retrieve the source code? This study aims to show what java bytecode is and how it is used, as well as how one is able to go from java bytecode back to source code in a process called Reverse En- gineering. Furthermore, the study will show previous work in reverse engineering, in obfuscation and to explain further details about what Java Virtual machine, bytecode and obfuscation is and how they work. Three programs of various complex- ity are made into bytecode and then obfuscated. The difference between the original code and the obfuscated code are then analyzed. The results show that it is possible to reverse engineer obfuscated code but some parts. Obfuscation does protect the code, as all the variable names are changed and every unused method are removed, as well as some methods changed to non-con- ventional ways to program. Keywords Reverse engineering, Java, JVM, bytecode, obfuscation, safety. Acknowledgements The Authors of this paper has attended KTH for an entire bachelor’s degree and this is the biggest and last course. We wish to thank all of our teachers for supporting us during these years. A big thank you to Anders Cajander who has been our mentor for this project and Ibrahim Orhan who has been our examiner who both of them helped us with their experience and perspective. Thank you to Sebastian Zeerak who lend us the code to the most advanced program which was made together with Gustav Smedberg. A bit thank you to Anders Lindstöm who is the author of AudioStreamUDP.java who gave us permission to use it in our thesis. We also wish to thank AstraZeneca, and in particular Mikael Engström and Olle Sundholm that helped us settle on a report subject, even if they did not have the resources to supervise us during the project. We also wish to thank our friends and families for the support they have given us during the making of this project. Table of contents 1 Introduction ............................................................................................................................................. 1 1.1 Problem definition ............................................................................................................................ 1 1.2 Objective .......................................................................................................................................... 2 1.3 Limitations ....................................................................................................................................... 2 2 Theory and background ......................................................................................................................... 5 2.1 Coding language .............................................................................................................................. 5 2.2 Java and Bytecode .......................................................................................................................... 5 2.2.1 Class file .................................................................................................................................... 5 2.3 Virtual Machine System ................................................................................................................ 11 2.3.1 Java Virtual Machine .............................................................................................................. 11 2.4 Reverse engineering analytics methods ....................................................................................... 12 2.4.1 Dynamic reverse engineering ................................................................................................. 13 2.4.2 Static reverse engineering ...................................................................................................... 13 2.5 Bytecode Obfuscation .................................................................................................................... 13 2.6 Previous work ................................................................................................................................. 15 2.7 Tools and frameworks.................................................................................................................... 16 2.7.1 Javap - The Java Class File Disassembler .............................................................................. 16 2.7.2 Decompilers ............................................................................................................................ 16 2.7.3 Dynamic analysis tool ............................................................................................................. 16 2.7.4 Code obfuscating program ..................................................................................................... 16 2.7.5 IntelliJ IDEA ........................................................................................................................... 16 2.8 Code ................................................................................................................................................ 16 2.9 Practical implementation .............................................................................................................. 16 3 Method ................................................................................................................................................... 19 3.1 Analysis Methods ........................................................................................................................... 19 3.2 Java code obfuscation ................................................................................................................... 20 3.3 Static analysis ................................................................................................................................ 20 3.4 Dynamic analysis .......................................................................................................................... 20 4 Results ................................................................................................................................................... 21 4.1 Simple program .............................................................................................................................. 21 4.1.1 Obfuscated code ....................................................................................................................... 21 4.1.2 Decompilation and dynamic analyzation .............................................................................. 21 4.2 More complex program ................................................................................................................ 23 4.2.1 Obfuscated code ..................................................................................................................... 23 4.2.2 Decompilation and dynamic analysis ................................................................................... 24 4.3 Advanced program ........................................................................................................................ 26 4.3.1 Obfuscated code ..................................................................................................................... 26 4.3.2 Decompilation and dynamic analyzing ...............................................................................
Load more

Recommended publications
  • Executable Code Is Not the Proper Subject of Copyright Law a Retrospective Criticism of Technical and Legal Naivete in the Apple V
    Executable Code is Not the Proper Subject of Copyright Law A retrospective criticism of technical and legal naivete in the Apple V. Franklin case Matthew M. Swann, Clark S. Turner, Ph.D., Department of Computer Science Cal Poly State University November 18, 2004 Abstract: Copyright was created by government for a purpose. Its purpose was to be an incentive to produce and disseminate new and useful knowledge to society. Source code is written to express its underlying ideas and is clearly included as a copyrightable artifact. However, since Apple v. Franklin, copyright has been extended to protect an opaque software executable that does not express its underlying ideas. Common commercial practice involves keeping the source code secret, hiding any innovative ideas expressed there, while copyrighting the executable, where the underlying ideas are not exposed. By examining copyright’s historical heritage we can determine whether software copyright for an opaque artifact upholds the bargain between authors and society as intended by our Founding Fathers. This paper first describes the origins of copyright, the nature of software, and the unique problems involved. It then determines whether current copyright protection for the opaque executable realizes the economic model underpinning copyright law. Having found the current legal interpretation insufficient to protect software without compromising its principles, we suggest new legislation which would respect the philosophy on which copyright in this nation was founded. Table of Contents INTRODUCTION................................................................................................. 1 THE ORIGIN OF COPYRIGHT ........................................................................... 1 The Idea is Born 1 A New Beginning 2 The Social Bargain 3 Copyright and the Constitution 4 THE BASICS OF SOFTWARE ..........................................................................
    [Show full text]
  • Studying the Real World Today's Topics
    Studying the real world Today's topics Free and open source software (FOSS) What is it, who uses it, history Making the most of other people's software Learning from, using, and contributing Learning about your own system Using tools to understand software without source Free and open source software Access to source code Free = freedom to use, modify, copy Some potential benefits Can build for different platforms and needs Development driven by community Different perspectives and ideas More people looking at the code for bugs/security issues Structure Volunteers, sponsored by companies Generally anyone can propose ideas and submit code Different structures in charge of what features/code gets in Free and open source software Tons of FOSS out there Nearly everything on myth Desktop applications (Firefox, Chromium, LibreOffice) Programming tools (compilers, libraries, IDEs) Servers (Apache web server, MySQL) Many companies contribute to FOSS Android core Apple Darwin Microsoft .NET A brief history of FOSS 1960s: Software distributed with hardware Source included, users could fix bugs 1970s: Start of software licensing 1974: Software is copyrightable 1975: First license for UNIX sold 1980s: Popularity of closed-source software Software valued independent of hardware Richard Stallman Started the free software movement (1983) The GNU project GNU = GNU's Not Unix An operating system with unix-like interface GNU General Public License Free software: users have access to source, can modify and redistribute Must share modifications under same
    [Show full text]
  • Chapter 1 Introduction to Computers, Programs, and Java
    Chapter 1 Introduction to Computers, Programs, and Java 1.1 Introduction • The central theme of this book is to learn how to solve problems by writing a program . • This book teaches you how to create programs by using the Java programming languages . • Java is the Internet program language • Why Java? The answer is that Java enables user to deploy applications on the Internet for servers , desktop computers , and small hand-held devices . 1.2 What is a Computer? • A computer is an electronic device that stores and processes data. • A computer includes both hardware and software. o Hardware is the physical aspect of the computer that can be seen. o Software is the invisible instructions that control the hardware and make it work. • Computer programming consists of writing instructions for computers to perform. • A computer consists of the following hardware components o CPU (Central Processing Unit) o Memory (Main memory) o Storage Devices (hard disk, floppy disk, CDs) o Input/Output devices (monitor, printer, keyboard, mouse) o Communication devices (Modem, NIC (Network Interface Card)). Bus Storage Communication Input Output Memory CPU Devices Devices Devices Devices e.g., Disk, CD, e.g., Modem, e.g., Keyboard, e.g., Monitor, and Tape and NIC Mouse Printer FIGURE 1.1 A computer consists of a CPU, memory, Hard disk, floppy disk, monitor, printer, and communication devices. CMPS161 Class Notes (Chap 01) Page 1 / 15 Kuo-pao Yang 1.2.1 Central Processing Unit (CPU) • The central processing unit (CPU) is the brain of a computer. • It retrieves instructions from memory and executes them.
    [Show full text]
  • Some Preliminary Implications of WTO Source Code Proposala INTRODUCTION
    Some preliminary implications of WTO source code proposala INTRODUCTION ............................................................................................................................................... 1 HOW THIS IS TRIMS+ ....................................................................................................................................... 3 HOW THIS IS TRIPS+ ......................................................................................................................................... 3 WHY GOVERNMENTS MAY REQUIRE TRANSFER OF SOURCE CODE .................................................................. 4 TECHNOLOGY TRANSFER ........................................................................................................................................... 4 AS A REMEDY FOR ANTICOMPETITIVE CONDUCT ............................................................................................................. 4 TAX LAW ............................................................................................................................................................... 5 IN GOVERNMENT PROCUREMENT ................................................................................................................................ 5 WHY GOVERNMENTS MAY REQUIRE ACCESS TO SOURCE CODE ...................................................................... 5 COMPETITION LAW .................................................................................................................................................
    [Show full text]
  • Bypassing Web Application Firewalls
    Bypassing Web Application Firewalls Pavol Lupták [email protected] CEO, Nethemba s.r.o Abstract The goal of the presentation is to describe typical obfuscation attacks that allow an attacker to bypass standard security measures such as various input filters, output encoding mechanisms used in web-based intrusion detection systems (IDS), intrusion prevention systems (IPS) and web application firewalls (WAFs). These attacks may include different networking tricks, polymorphic shellcode and various code techniques. At the beginning we analyse and compare different HTML parsing and interpretation approaches used by most-common browsers that can lead to unique attack vectors. Javascript, with a full range of features, represents another effective way that can be used to obfuscate or de-obfuscate code – some existing obfuscation tools are mentioned. We describe how it is possible to construct “non-alphanumeric Javascript code” which does not contain alphabetic or numeric characters, but still can contain malicious executable code. Despite the fact that most current applications are immune to SQL injection attacks, it is still possible to find many vulnerable applications. We focus on different fuzzy techniques (and useful open source SQL injection tools that implement them) which can still be used to bypass weak input validation controls. We conclude our presentation with a demonstration of the most basic obfuscation techniques that can be successfully used to bypass traditional web application firewalls (WAFs). Finally, we briefly describe current mitigation techniques that are recommended for efficient malicious Javascript code analysis and sanitizing user input containing untrusted code. Keywords: WAF, IPS, IDS, obfuscation, SQL injection, XSS, CSS, CSRF.
    [Show full text]
  • Reverse Software Engineering As a Project-Based Learning Tool
    Paper ID #33764 Reverse Software Engineering as a Project-Based Learning Tool Ms. Cynthia C. Fry, Baylor University CYNTHIA C. FRY is currently a Senior Lecturer of Computer Science at Baylor University. She worked at NASA’s Marshall Space Flight Center as a Senior Project Engineer, a Crew Training Manager, and the Science Operations Director for STS-46. She was an Engineering Duty Officer in the U.S. Navy (IRR), and worked with the Naval Maritime Intelligence Center as a Scientific/Technical Intelligence Analyst. She was the owner and chief systems engineer for Systems Engineering Services (SES), a computer systems design, development, and consultation firm. She joined the faculty of the School of Engineering and Computer Science at Baylor University in 1997, where she teaches a variety of engineering and computer science classes, she is the Faculty Advisor for the Women in Computer Science (WiCS), the Director of the Computer Science Fellows program, and is a KEEN Fellow. She has authored and co- authored over fifty peer-reviewed papers. Mr. Zachary Michael Steudel Zachary Steudel is a 2021 graduate of Baylor University’s computer science department. In his time at Baylor, he worked as a Teaching Assistant under Ms. Cynthia C. Fry. As part of the Teaching Assistant role, Zachary designed and created the group project for the Computer Systems course. Zachary Steudel worked as a Software Developer Intern at Amazon in the Summer of 2019, a Software Engineer Intern at Microsoft in the Summer of 2020, and begins his full-time career with Amazon in the summer of 2021 as a software engineer.
    [Show full text]
  • C++11 Metaprogramming Applied to Software Obfuscation
    C++11 METAPROGRAMMING APPLIED TO SOFTWARE OBFUSCATION SEBASTIEN ANDRIVET About me ! Senior Security Engineer at SCRT (Swiss) ! CTO at ADVTOOLS (Swiss) Sebastien ANDRIVET Cyberfeminist & hacktivist Reverse engineer Intel & ARM C++, C, Obj-C, C# developer Trainer (iOS & Android appsec) PROBLEM Reverse engineering • Reverse engineering of an application if often like following the “white rabbit” • i.e. following string literals • Live demo • Reverse engineering of an application using IDA • Well-known MDM (Mobile Device Management) for iOS A SOLUTION OBFUSCATION What is Obfuscation? Obfuscator O O ( ) = YES! It is also Katy Perry! • (almost) same semantics • obfuscated Obfuscation “Deliberate act of creating source or machine code difficult for humans to understand” – WIKIPEDIA, APRIL 2014 C++ templates OBJECT2 • Example: Stack of objects OBJECT1 • Push • Pop Without templates Stack singers; class Stack { singers.push(britney); void push(void* object); void* pop(); }; Stack apples; apples.push(macintosh); • Reuse the same code (binary) singers apples • Only 1 instance of Stack class With C++ templates template<typename T > Stack<Singer> singers; class Stack singers.push(britney); { void push(T object); T pop(); Stack<Apple> apples; }; apples.push(macintosh); With C++ templates Stack<Singer> singers; singers.push(britney); Stack<Singers> Stack<Apples*> Stack<Apple> apples; apples.push(macintosh); singers apples C++ templates • Two instances of Stack class • One per type • Does not reuse code • By default • Permit optimisations based on types • For ex. reuse code for all pointers to objects • Type safety, verified at compile time Type safety • singers.push(apple); // compilation error Optimisation based on types • Generate different code based on types (template parameters) template<typename T> class MyClass • Example: enable_if { ..
    [Show full text]
  • Opportunities and Open Problems for Static and Dynamic Program Analysis Mark Harman∗, Peter O’Hearn∗ ∗Facebook London and University College London, UK
    1 From Start-ups to Scale-ups: Opportunities and Open Problems for Static and Dynamic Program Analysis Mark Harman∗, Peter O’Hearn∗ ∗Facebook London and University College London, UK Abstract—This paper1 describes some of the challenges and research questions that target the most productive intersection opportunities when deploying static and dynamic analysis at we have yet witnessed: that between exciting, intellectually scale, drawing on the authors’ experience with the Infer and challenging science, and real-world deployment impact. Sapienz Technologies at Facebook, each of which started life as a research-led start-up that was subsequently deployed at scale, Many industrialists have perhaps tended to regard it unlikely impacting billions of people worldwide. that much academic work will prove relevant to their most The paper identifies open problems that have yet to receive pressing industrial concerns. On the other hand, it is not significant attention from the scientific community, yet which uncommon for academic and scientific researchers to believe have potential for profound real world impact, formulating these that most of the problems faced by industrialists are either as research questions that, we believe, are ripe for exploration and that would make excellent topics for research projects. boring, tedious or scientifically uninteresting. This sociological phenomenon has led to a great deal of miscommunication between the academic and industrial sectors. I. INTRODUCTION We hope that we can make a small contribution by focusing on the intersection of challenging and interesting scientific How do we transition research on static and dynamic problems with pressing industrial deployment needs. Our aim analysis techniques from the testing and verification research is to move the debate beyond relatively unhelpful observations communities to industrial practice? Many have asked this we have typically encountered in, for example, conference question, and others related to it.
    [Show full text]
  • Decompilation of Binary Programs
    SOFTWARE—PRACTICE AND EXPERIENCE, VOL. 25(7), 811–829 (JULY 1995) Decompilation of Binary Programs CRISTINA CIFUENTES∗ AND K. JOHN GOUGH (email: cifuente@fit.qut.edu.au gough@fit.qut.edu.au) School of Computing Science, Queensland University of Technology, GPO Box 2434, Brisbane QLD 4001, Australia SUMMARY The structure of a decompiler is presented, along with a thorough description of the different modules that form part of a decompiler, and the type of analyses that are performed on the machine code to regenerate high-level language code. The phases of the decompiler have been grouped into three main modules: front-end, universal decompiling machine, and back-end. The front-end is a machine-dependent module that performs the loading, parsing and semantic analysis of the input program, as well as generating an intermediate representation of the program. The universal decompiling machine is a machine- and language-independent module that performs data and control flow analysis of the program based on the intermediate representation, and the program’s control flow graph. The back-end is a language-dependent module that deals with the details of the target high-level language. In order to increase the readability of the generated programs, a decompiling system has been imple- mented which integrates a decompiler, dcc, and an automatic signature generator, dccSign. Signatures for libraries and compilers are stored in a database that is read by the decompiler; thus, the generated programs can make use of known library names, such as WriteLn() and printf(). dcc is a decompiler for the Intel 80286 architecture and the DOS operating system.
    [Show full text]
  • Native X86 Decompilation Using Semantics- Preserving Structural Analysis and Iterative Control-Flow Structuring Edward J
    Native x86 Decompilation Using Semantics- Preserving Structural Analysis and Iterative Control-Flow Structuring Edward J. Schwartz, Carnegie Mellon University; JongHyup Lee, Korea National University of Transportation; Maverick Woo and David Brumley, Carnegie Mellon University This paper is included in the Proceedings of the 22nd USENIX Security Symposium. August 14–16, 2013 • Washington, D.C., USA ISBN 978-1-931971-03-4 Open access to the Proceedings of the 22nd USENIX Security Symposium is sponsored by USENIX Native x86 Decompilation using Semantics-Preserving Structural Analysis and Iterative Control-Flow Structuring Edward J. Schwartz JongHyup Lee Carnegie Mellon University Korea National University of Transportation Maverick Woo David Brumley Carnegie Mellon University Carnegie Mellon University Abstract in the literature assume access to source code. For in- stance, there are numerous source-based static vulnera- There are many security tools and techniques for analyz- bility finding tools such as KINT [40], RICH [9], and ing software, but many of them require access to source Coverity [6], but equivalent binary-only tools are scarce. code. We propose leveraging decompilation, the study In many security scenarios, however, access to source of recovering abstractions from compiled code, to apply code is simply not a reasonable assumption. Common existing source-based tools and techniques to compiled counterexamples include analyzing commercial off-the- programs. A decompiler should focus on two properties shelf software for vulnerabilities and reverse engineering to be used for security. First, it should recover abstractions malware. The traditional approach in security has been to as much as possible to minimize the complexity that must directly apply some form of low-level binary analysis that be handled by the security analysis that follows.
    [Show full text]
  • Software Protection Through Obfuscation
    This document is downloaded from DR‑NTU (https://dr.ntu.edu.sg) Nanyang Technological University, Singapore. Software protection through obfuscation Balachandran, Vivek 2014 Balachandran, V. (2015). Software protection through obfuscation. Doctoral thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/62930 https://doi.org/10.32657/10356/62930 Downloaded on 02 Oct 2021 00:34:50 SGT Software Protection through Obfuscation School of Computer Engineering A Thesis Submitted to the Nanyang Technological University in partial fulfillment of the requirement of the degree of Doctor of Philosophy by Vivek Balachandran under the supervision of Prof. Ng Wee Keong and Prof. Sabu Emmanuel 2015 2 Acknowledgments Foremost, I would like to express my sincere gratitude to my advisors Prof. Sabu Em- manuel and Prof. Ng Wee Keong for the continuous support of my Ph.D study and research, for their patience, motivation, enthusiasm, and immense knowledge. I thank my fellow labmates in Nanyang Technological University, Singapore: Shaheen Ansari, Deepak Subrmanyam, Chia Tee Kiah for their kind support. Many friends have helped me stay sane through these difficult years. Their support and care helped me overcome setbacks and stay focused on my graduate study. I would like to thank Aditya Venkataraman, Chitra Panchapakesan, Ganesh Bharadwaj, Girid- haran Karunagaran, Karthik Raveendran, Manaswini Ramkumar, Manisha Mujumdar, Nirnaya Sarangan, Ponnu Jacob, Roshan Wahab, Shubha Nageswaran,Vidhi Patel and Vipin Pillai for making my life wonderful as a grad student. Last but not the least, I would like to thank my family: my parents G. Balachandran and V. Santhakumari, and my brother Vishakh Balachandran who were always there for me.
    [Show full text]
  • 6.858 Final Project
    6.858 Final Project Predrag Gruevski Paul Hemberger Andres Romero Albert Wu Introduction Since remote desktop applications are such powerful tools for controlling computers from across the Internet, it stands to reason that adversaries would be motivated to discover and exploit vulnerabilities in such software. In our final project, we explored many of the most popular remote desktop applications for Android clients. Most of them provide little to no security, and we were able to easily create exploits to compromise the keyboard and mouse of the remote machine. Methods We looked at the following popular Android clients (each has at least 100,000 installs on the Google Play app market): 1. TeamViewer for Remote Control 2. Air HID 3. Remote Mouse 4. WiFi Mouse 5. Android Mouse and Keyboard 6. Mobile Mouse Lite 7. Remote Control Collection We installed each of the above clients on an Android phone and the corresponding server on a remote machine running Windows, OS X, or Linux. We used two types of tools for reverse engineering the messaging protocols: 1. Wireshark ­ Wireshark is a network protocol analyzer. Using Wireshark, we were able to analyze incoming and outgoing packets from the remote machine. This was extremely useful for applications using plaintext protocols. 2. Decompilers ­ Tools like Java Decompiler Project, JetBrains dotPeek, Decompile Android, and dex2jar allowed us to decompile many of the client APK files and server executables into very readable Java code. After reverse engineering the protocols, we set up virtual machines running the above applications and send arbitrary keyboard and mouse commands from the host machine via UDP and TCP.
    [Show full text]