Linköping University | Department of Computer and Information Science Master’s thesis, 30 ECTS | Datateknik 2021 | LIU-IDA/LITH-EX-A--21/044--SE Automating software installation for cyber security research and testing public exploits in CRATE Att automatisera mjukvaruinstallationer för cybersäkerhets- forskning och testandet av publika angreppskoder i CRATE Johan Hedlin Joakim Kahlström Supervisor : Niklas Carlsson Examiner : Andrei Gurtov External supervisor : Jonas Almroth Linköpings universitet SE–581 83 Linköping +46 13 28 10 00 , www.liu.se Upphovsrätt Detta dokument hålls tillgängligt på Internet - eller dess framtida ersättare - under 25 år från publicer- ingsdatum under förutsättning att inga extraordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka ko- pior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervis- ning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säker- heten och tillgängligheten finns lösningar av teknisk och administrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsman- nens litterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping University Electronic Press se förlagets hemsida https://www.ep.liu.se/. Copyright The publishers will keep this document online on the Internet - or its possible replacement - for a period of 25 years starting from the date of publication barring exceptional circumstances. The online availability of the document implies permanent permission for anyone to read, to down- load, or to print out single copies for his/hers own use and to use it unchanged for non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional upon the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its www home page: https://www.ep.liu.se/. Johan Hedlin © Joakim Kahlström Abstract As cyber attacks are an ever-increasing threat to many organizations, the need for con- trolled environments where cyber security defenses can be tested against real-world at- tacks is increasing. These environments, called cyber ranges, exist across the world for both military and academic purposes of various scales. As the function of a cyber range involves having a set of computers, virtual or physical, that can be configured to replicate a corporate network or an industrial control system, having an automated method of config- uring these can streamline the process of performing different exercises. This thesis aims to provide a proof of concept of how the installation of software with known vulnerabili- ties can be performed and examines if the software is vulnerable directly after installation. The Cyber Range And Training Environment (CRATE) developed by the Swedish Defence Research Agency (FOI) is used as a testbed for the installations and FOI-provided tools are used for launching automated attacks against the installed software. The results show that installations can be performed without Internet access and with minimal network traf- fic being generated and that our solution can rewrite existing software packages from the package manager Chocolatey to work with an on-premises repository with an 85% success rate. It is also shown that very few publicly available exploits succeed without any man- ual configuration of either the exploit or the targeted software. Our work contributes to making it easier to set up environments where cyber security research and training can be conducted by simplifying the process of installing vulnerable applications. Contents Abstract iii Contents iv List of Figures vii List of Tables viii List of Listings ix Glossary x Acronyms xi 1 Introduction 1 1.1 Motivation . 1 1.2 Aim............................................ 2 1.3 Research questions . 2 1.4 Contributions . 2 1.5 Delimitations . 3 1.6 Disclaimer . 3 1.7 Thesis outline . 4 2 Background 5 2.1 Cyber Range And Training Environment (CRATE) . 5 2.2 Package managers for Windows . 6 2.2.1 Chocolatey . 7 2.2.2 Windows Package Manager . 7 2.2.3 Scoop . 7 2.2.4 Others . 7 2.3 Automation tools . 8 2.3.1 Ansible . 8 2.3.2 Chef . 9 2.3.3 Puppet . 10 2.3.4 Salt . 12 2.4 Virtual machine setup tools . 13 2.4.1 Boxstarter . 13 2.4.2 Packer . 14 2.5 Vulnerability-related naming schemes . 14 2.5.1 Common Platform Enumeration (CPE) . 14 2.5.2 Common Vulnerabilities and Exposures (CVE) . 14 2.6 Metasploit . 15 2.7 Scanning, Vulnerabilities, Exploits and Detection (SVED) . 15 iv 2.8 SVED Visualization Tool (SVIZ) . 16 2.9 Related work . 17 3 Method 20 3.1 Automating software installation . 20 3.1.1 Package managers selection . 21 3.1.2 Automation tool selection . 22 3.1.3 Summary of the tool selection . 23 3.1.4 Chocolatey list of packages . 23 3.1.5 Feedback from installation . 24 3.1.6 Database . 24 3.1.7 Mapping to vulnerabilities . 24 3.1.8 Online installation tests . 27 3.1.9 Rate limiting and excessive use . 27 3.1.10 Internal repository . 28 3.1.11 Offline installation tests . 30 3.2 Selecting exploits for evaluation . 31 3.2.1 Version difference . 31 3.2.2 Name and vendor difference . 32 3.2.3 Exploit selection criteria . 33 3.3 Automatic testing of exploits . 33 3.3.1 Preparing VMs . 34 3.3.2 Creating an attack sequence . 34 3.4 Manual testing of exploits . 35 3.5 Vulnerable state . 36 4 Results 37 4.1 Automating software installation . 37 4.1.1 With online access . 37 4.1.2 Evaluating the reliability of Chocolatey’s online repository . 38 4.1.3 Internal repository . 39 4.2 Selecting exploits for evaluation . 40 4.3 Automatic testing of exploits . 40 4.4 Manual testing of exploits . 41 4.5 Vulnerable state . 42 5 Discussion 44 5.1 Results . 44 5.1.1 Internal repository . 44 5.1.2 First usage . 45 5.1.3 Exploit testing . 46 5.2 Method . 47 5.2.1 Alternative methods of performing automated software installations . 47 5.2.2 Less focus on automated testing . 47 5.2.3 Source criticism . 47 5.3 Challenges . 47 5.3.1 Hitting the rate limit during downloading and testing packages . 47 5.3.2 Database . 48 5.3.3 Corrupt output from Ansible . 48 5.3.4 Strange URLs . 48 5.4 The work in a wider context . 49 6 Conclusion 51 v 6.1 Research questions . 51 6.2 Future work . 52 6.2.1 Improving the mapping from program and version to CPE . 52 6.2.2 Improving the exploit suggestion process . 52 6.2.3 Further automating the internalization process . 53 Bibliography 54 A Automatic exploit test results 59 vi List of Figures 2.1 Simplified illustration of CRATE . 6 2.2 Typical attack sequence in SVED . 16 2.3 Screenshot of the attack graph creator in SVED . 16 2.4 Screenshot of the attack graph section of SVIZ . 17 3.1 Information flow for automated software installation . 21 3.2 Internal repository using Sonatype Nexus 3 . 30 4.1 Output from the software installation process . 37 4.2 Installation result statistics . 38 4.3 Installation results over time . ..