Regano Phd Thesis.Pdf
Total Page:16
File Type:pdf, Size:1020Kb
POLITECNICO DI TORINO Repository ISTITUZIONALE An Expert System for Automatic Software Protection Original An Expert System for Automatic Software Protection / Regano, Leonardo. - (2019 Jul 17), pp. 1-131. Availability: This version is available at: 11583/2751495 since: 2019-09-13T08:17:09Z Publisher: Politecnico di Torino Published DOI: Terms of use: openAccess This article is made available under terms and conditions as specified in the corresponding bibliographic description in the repository Publisher copyright (Article begins on next page) 02 October 2021 Doctoral Dissertation Doctoral Program in Computer and Control Engineering (31st cycle) An Expert System for Automatic Software Protection Leonardo Regano ****** Supervisors Prof. Antonio Lioy, Supervisor Cataldo Basile, Ph.D., Co-supervisor Doctoral Examination Committee: Bart Coppens, Ph.D., Referee, Ghent University Prof. Claudia Raibulet, Referee, Università degli Studi di Milano-Bicocca Prof. Bjorn De Sutter, Ghent University Prof. Stefano Paraboschi, Università degli Studi di Bergamo Prof. Riccardo Sisto, Politecnico di Torino Politecnico di Torino 17 July 2019 This thesis is licensed under a Creative Commons License, Attribution - Noncommercial- NoDerivative Works 4.0 International: see www.creativecommons.org. The text may be repro- duced for non-commercial purposes, provided that credit is given to the original author. I hereby declare that, the contents and organisation of this dissertation constitute my own original work and does not compromise in any way the rights of third parties, including those relating to the security of personal data. ................................................ Leonardo Regano Turin, 17 July 2019 Abstract In the Information Age, software is omnipresent in almost everyone’s life. A plethora of services is offered in a digital manner, for example with e-government portals, e-banking mobile apps, and e-commerce websites. Indeed, users trust the software enabling such services with their personal information, like credit card num- bers or e-banking credentials. Thus, software must be protected with care, in order to mitigate possible threats against such valuable data. Furthermore, software com- panies have to protect the assets in their applications, such as intellectual property of algorithms, methods preventing unauthorized application distribution, and users’ personal data. Failing to do so may deeply damage software companies’ finances, due to lost application sales, and reputation, if users’ personal data is leaked. However, protecting applications is a cumbersome task, reserved to few expert prac- titioners of this field, due to the rising complexity of applications, and the availability of numerous protection techniques. Each of the latter has strengths and weakness, and their effectiveness in safeguarding the application assets depend on numerous factors, such as the structure of protected code, the tools employed to apply such protec- tions along with their configuration parameters, and the expected skills of a possible attacker that may be interested in breaching the application assets. In this thesis, an expert system for automating the protection of applications is presented. To the best of the author’s knowledge, it is the first application of the expert system paradigm to this challenging problem. Mimicking the decision process of a software security expert, the system, given the source code of an application that must be protected, is able to produce a binary of the application, hardened with the protection technique most suitable to defer, for the longest time possible, an attacker aiming to breach the application assets. Apart from the program source code, the system requires from the user only a list of the assets that must be protected, with each of them associated with one or more high-level security requirements (e.g., confidentiality, integrity), which must be safeguarded against possible attacks. The system has been developed during the EC-funded ASPIRE project, whose objective was to develop a set of protection techniques for Android applications, along with automated tools to deploy them on threatened code. The system is not only able to decide the generic protection techniques that must be applied to the program code, but also the specific parameters to drive such tools, thus providing a comprehensive protection solution specifically tailored for the targeted application. The system is based on the formalization of the mental decision processes and background knowledge of software security experts involved in the aforementioned project. This thesis advances the state of the art in the field of software security with the following contributions: (1) a meta-model for software security, able to formalize all the related concepts, such as characteristics of the application and of the components of its code, attacks that can be mounted against the application, and protection techniques that can be used to mitigate them; (2) a risk assessment methodology for software, with a formalization of attacks against applications assets, consisting in the identification of simple attack tasks, which can be then chained incomplete attacks that can be carried out to successfully breach the application assets; (3) a risk mitigation strategy, based on a game-theoretic approach, able to infer the protections best able to defer possible attacks against the application; (4) a set of asset hiding strategies, devised to increase the effort needed by an attacker to locate assets in the application binary; (5) a complete and automated workflow for software protection, implementing the aforementioned risk management processes in a fully-fledged expert system. ii Contents List of Tables vi List of Figures vii 1 Background 7 1.1 Software protection....................................7 1.1.1 Automated protection tools...........................7 1.1.2 Anti-reverse engineering techniques.......................9 1.1.3 Anti-tampering techniques............................ 21 1.1.4 Anti-debugging.................................. 26 1.2 Knowledge-based and expert systems.......................... 27 1.2.1 Knowledge bases and ontologies......................... 27 1.2.2 Inference engines................................. 29 1.2.3 Expert systems.................................. 30 2 Decision support system for software protection 33 2.1 Problem statement.................................... 33 2.1.1 Risk framing................................... 34 2.1.2 Risk assessment.................................. 35 2.1.3 Risk mitigation.................................. 36 2.1.4 Risk monitoring................................. 38 2.2 Automated workflow for software protection...................... 39 2.2.1 Software protection meta-model........................ 39 2.2.2 Risk assessment phase.............................. 41 2.2.3 Asset protection phase.............................. 42 2.2.4 Asset hiding phase................................ 43 2.2.5 Complete workflow................................ 44 2.3 Workflow execution example............................... 45 2.4 Validation......................................... 47 2.4.1 Qualitative evaluation.............................. 48 2.4.2 Experimental assessment............................ 50 3 Software security meta-model 52 3.1 Core meta-model..................................... 53 3.2 Application meta-model................................. 55 3.3 Protection meta-model.................................. 56 3.4 Attack meta-model.................................... 57 3.5 Meta-model validation.................................. 59 iv 4 Risk assessment 61 4.1 Application structure modeling............................. 63 4.2 Attacker goals modeling................................. 63 4.3 Attack steps and paths modeling............................ 64 4.4 Risk probability...................................... 66 4.5 Validation......................................... 67 5 Asset protection 69 5.1 Protection decision workflow.............................. 72 5.2 Deployed protection instances enumeration...................... 73 5.3 Inference of valid solutions................................ 75 5.4 Software metrics..................................... 77 5.4.1 Software protection complexity metrics.................... 78 5.4.2 Complexity metrics prediction......................... 80 5.4.3 Protection overhead estimation......................... 82 5.5 Asset protection solution inference........................... 84 5.5.1 Protection index................................. 86 5.5.2 Solution solver mini-max algorithm....................... 89 5.6 Validation......................................... 91 6 Asset hiding 93 6.1 Protection fingerprints.................................. 94 6.2 Asset hiding strategies.................................. 95 6.3 Mixed Integer-Linear Programming model....................... 97 6.3.1 Application structure and protections..................... 97 6.3.2 Domain Parameters............................... 99 6.3.3 Linear Problem.................................. 100 6.4 Translation algorithm.................................. 104 6.5 Validation......................................... 105 7 Conclusions and future work 106 A ESP implementation 108 A.1 Main ESP components.................................. 108 A.2 ESP workflow....................................... 109 A.3 Solution deployment................................... 110 Bibliography 115 v List of Tables 2.1 Functions