3.3 Maltego Transforms for Danish OSINT-Sources

Total Page:16

File Type:pdf, Size:1020Kb

3.3 Maltego Transforms for Danish OSINT-Sources Enhancing identification and reporting of potentially harmful public data on Danish organizations by Rasmus Lau Petersen Supervisor: Christian Damsgaard Jensen M.Sc.Eng.IT master thesis DTU Compute Technical University of Denmark Kongens Lyngby, August 1st, 2017 Summary (English) This master thesis aims to aid and enhance the current processes used to identify and report in Open Source Intelligence (OSINT) on Danish organizations. This is data generated by the daily work of the organization and its employees when they act and communicate. The data is collected by public registries or commercial 3rd parties, which can provide a valuable source of information for an attacker intending to target organizations. Security professionals are aware that such data exist and report on it as part of their services, but collecting it effectively is difficult. Relating it to the domain of the organizations acting under the different Danish legislation and standards can be difficult. The organizations themselves may have a good overview of the latter, but lack overview and awareness of OSINT and the attack scenarios this enables. We attempt to enhance the process of identification and reporting in two ways: By developing plug-ins (“transforms”) for the widely used OSINT-gathering program Maltego by Paterva and by developing a framework for inputting findings from Maltego to generate a report categorizing findings and relating them to common OSINT-enabled attack scenarios and applicable legislation, standards and guidelines. We examine current methodologies for conducting vulnerability- and penetration test assessments, standards, guidelines and Danish legislation pertaining to OSINT-data and identify and analyze a dozen common OSINT-enabled attack scenarios. The section on standards finds valuable, general guidance to enhance security maturity in organizations, but only little pertains to hinder OSINT-generation. Traits of social engineering-techniques, which often are involved in these types of attacks, are also analyzed. Two transforms were successfully developed for the Danish domain registry DK Hostmaster and a commercial supplier of aggregated data from the registry of Danish vehicles and debt of these. They can be included in and enhance the work processes in a security consultancy. The development also highlights problems with developing for closed source-software; due to the difficulties faced and solved in regards to this, a section on developing transforms for Maltego is included in the appendix. Based on the examined legislation, standards and guidelines and the scenarios created, a report- generation framework has been developed. It takes an export from Maltego and by manually assigning labels to each entry, outputs a report. The report imitates examples of commercially used reports using colors and summaries to make it easy-read. It connects the findings to the scenarios describing specifically targeted attacks and three of the surveyed standards. The framework works and outputs successfully as-is, but the work highlighted difficulties with linking the domains of data labels of actual findings to standardized scenarios and formal standards. It is an essential task to get these links correct to ensure proper conclusions in the output, so the report can be used as-is in the product portfolio of e.g. a security consultancy. We list suggestions for future work in the discussion and conclusion and highlights the need for conducting the research with input from e.g. security professionals of consultancies and internal organizational security functions. Page 3 of 211 Summary (Danish) Målet for denne afhandling er at støtte og forbedre processerne omkring identifikation og afrap- portering af Open Source Intelligence (OSINT) der måtte eksistere om danske organisationer. Denne type data bliver genereret som et bi-produkt at processerne omkring det daglige arbejde organisationen og dens ansatte udfører, når de agerer og kommunikerer. Dataen opsamles af offentlige instanser og kommercielle tredjeparter og kan udgøre en værdifuld kilde til information for angribere, der måtte ønske at angribe organisationerne. IT-sikkerhedsprofessionelle er op- mærksomme på eksistensen af denne type data og rapportering af den indgår som en del af deres tilbudte services, men opsamlingen af den er besværlig og tidskrævende, herunder også relatere dataen til den virkelighed organisationerne agerer under inkl. dansk lovgivning og standarder. Mens organisationerne nok har et godt overblik over sidstnævnte, kan de mangle indsigt i hvad OSINT-data er og de angrebstyper der er forbundet herved. I projektet søger vi at forbedre processerne for identifikation og afrapportering på to måder: Dels ved at udvikle plug-ins (“transforms”) til det vidt udbredte OSINT-indsamlingsværktøj Maltego udviklet af Paterva og dels ved at udvikle software, som ud fra resultaterne fra Maltego kan generere en rapport, som kategoriserer de gjorte fund og relaterer dem til almindeligt forekom- mende cyberangreb, hvor OSINT-data spiller en rolle samt gældende lovgivning, standarder og vejledning. Vi undersøger aktuelle metoder til at udføre sårbarheds- og penetrationstests, standarder, vej- ledninger og dansk lovgivning som beskæftiger sig med OSINT-data. Endvidere identificerer og analyserer vi omkring et dusin typiske cyber angreb, muliggjort af OSINT-data. Afsnittet identifi- cerer værdifuld, generel vejledning, som kan hjælpe med at forbedre sikkerheden i organisationer, men kun en mindre del af den beskæfter sig med at forhindre at der genereres OSINT-data. Vi undersøger og beskriver også typiske træk ved “social engineering”-teknikker, som ofte bruges ifm. denne type angreb. Der blev succesfuldt udviklet to transforms, som henter data fra dels den danske domæneregistrant DK Hostmaster og dels en kommerciel udbyder af et aggregeret datasæt over motorregistret samt gæld fra Tinglysningen ifm. dette. De to transforms can inkluderes i arbejdsgangen hos IT-sikkerhedskonsulenter og forbedre arbejdsprocessen herved. Udviklingen fremhævede dog også problemerne ved at udvikle til closed source-software; pga. udfordringerne herved, er de undervejs i projektet identificerede løsninger repræsenteret særskilt i appendix. På baggrund af den undersøgte lovgivning, standarder og vejledninger og de opstillede scenarier, blev der udviklet et stykke software til at generere en rapport. Softwaren kører på en eksporteret fil fra Maltego og ved manuelt at tildele typer til hvert stykke data i filen, genererer softwaren en rapport. Rapporten efterligner eksempler på kommercielt fremstillede rapporter ved at bruge farver og små konklusioner for at gøre den let forståelig. Den forbinder de i Maltego fundne data med de opstillede scenarier (dog kun målrettede angreb) og tre af de undersøgte standarder, som bedst lod sig relatere til OSINT-data. Softwaren fungerer og genererer i den nuværende udgave, men det udførte arbejde tydeliggjorde problemer med at koble praktisk orienterede datakategorier med standardiserede angrebsscenarier og formelle standarder. Det er essentielt at kunne udføre disse koblinger korrekt, for at sikre, at konklusionerne i den automatisk genererede rapport stemmer bedst muligt overens med virkeligheden og kan bruges i produktporteføljen i et IT-sikkerhedskonsulentfirma. Vi beskriver forslag til fremtidigt arbejde i diskussionen og konklusionen og fremhæver nødven- digheden af at udføre den videre research med input fra f.eks. IT-sikkerhedsprofessionelle fra både konsulenthuse og interne IT-sikkerhedsafdelinger ude i organisationerne. Page 5 of 211 Acknowledgements I would like to thank all the people around for enabling this project. In particular I would like to thank Keld Norman and Andy Cini for growing the initial idea with me and for Christian D. Jensen to join in on the idea and supervise the project for, including bi-weekly meetings, ad-hoc late night e-mails and valuable input. I would like to thank Specialklinikken Rebild by J. Erling Pedersen-Bach and massage therapist Cathrine Thovtrup, without whose treatments it would have been next to impossible to write a master thesis with the whiplash/concussion I acquired three years back, but now finally are in recovery from. I am grateful for my wonderful girlfriend whom have supported and helped me all the way through and especially when things got a bit busy. I look forward to spend much more time with you in our new apartment, which we also managed to buy during the month up till hand-in! An extended thank you to Rosita Kanto, Frederik Eriksen & William Embarek for proof-reading my report and shout-out to all the good people on ST Nybrogård Kollegiet, whom have had to listen to my grumblings during up’s and down’s. Last, but most certainly not least, where “thank you’s” do not even suffice: A sincere, heartfelt “thank you” to my parents, Nina & John Petersen, for 27 years of love, support, blood, sweat and tears, which have enabled me to get to this point! I am most grateful and will never be able to pay it back enough. Preface This thesis was prepared at DTU Compute in fulfilment of the requirements for acquiring an M.Sc. in Engineering and IT. The thesis deals with the problems of organizations acting in an increasingly connected world where all actions leaves traces. To discover and understand the implications of data on open sources is difficult, but the goal was to enhance the process by developing new plug-ins to a popular platform for OSINT-investigations and a framework for automatically generating a report concluding on the findings and relating them to applicable legislation, standards and guidelines. An overview of the thesis’ content is found in Section 1.2. Lyngby, August 1st, 2017 Rasmus Lau Petersen Contents 1 Introduction 11 1.1 Problem definition . 13 1.1.1 Goals . 15 1.1.2 Risks . 15 1.1.3 Work methods . 16 1.2 Overview of this report . 17 2 State-of-the-art 19 2.1 Penetration testing . 19 2.1.1 Pre-engagement interactions . 22 2.1.2 Intelligence gathering . 24 2.1.3 Threat modeling . 26 2.1.4 Vulnerability analysis . 29 2.1.5 Exploitation . 31 2.1.6 Reporting . 32 2.2 Risk analysis . 33 2.2.1 A general view: DS/ISO 27000-series . 33 2.2.2 Specific methodologies .
Recommended publications
  • CHAPTER 2: Getting to Know Your Targets LEARN MORE BUY
    Sample Chapter CHAPTER 2: Getting to Know Your Targets LEARN MORE BUY NOW ©2019 McGraw-Hill All-In-One_PE / CompTIA PenTest+® Certification Practice Exams / Jonathan Ammerman / 090-7 / Chapter 2 CHAPTER Getting to Know 2 Your Targets This chapter includes questions on the following topics: • Information gathering in a given scenario using appropriate techniques • A comparison of various tools and their use cases Following the pre-engagement meetings, the definition of the scope and rules of engagement, and the signing of contracts, a penetration tester is free to begin the next phase of an assessment: information gathering. It is generally accepted that there are two types of information gather- ing: passive and active. Passive information gathering consists of any collection of intelligence by means that are effectively invisible to the target in question; active information gathering will be discussed in Chapters 3 and 4. By its most basic definition, passive information gathering is the collection of information from publicly available sources; this could mean queries in any given search engine, harvesting information from public DNS servers, or searching for the target organization’s networks with tools such as Shodan or theharvester. To define it more precisely, passive information gathering is any collection of intelligence that may be useful in a penetration test without directly connecting or identifying oneself to the target of the penetration test. Although it is not terribly common to find a quick path to an exploitable process or service via passive information gathering, the data collected is still of importance to the overall penetration test; organizations often are unaware of just how wide their digital footprint is and will be amazed at the information you can find without them being aware.
    [Show full text]
  • NCDAA Seminar: Digital Forensics AM Session
    NCDAA Seminar: Digital Forensics AM Session: Understanding the Data Context & Correlation Professor Matthew Miller, PhD University of Nebraska Kearney PM Session: Context & Correlation Shawn Kasal, Digital Forensic Expert 12 Points Technologies LLC Wednesday, October 9, 2019 Embassy Suites Hotel – La Vista Conference Center This page intentionally left blank. Text ~Learning ~ Outcomes INTRODUCTION • DOD • NC4 Cybercop • In-Q-Tel at University of Nebraska Kearney and Omaha • Expert Witness • GM, Ford Motor Company and Daimler Benz What does it take to construct, design and build a work product? A mix of ability, talent and skill is needed to construct and de- construct systems A system is larger and more complex than a computer network GeoPolitics is a system Our diversity, talent & backgrounds Our Team allow us to accomplish what others can not do. Arab Spring Election Reality Winner JP 3-13 • Computer Network Operations (CNO) consist of: • Computer Network Attack (CNA) • Computer Network Defense (CND) • Computer Network Exploitation (CNE) Computer Network Operations (CNO) • Broad term • Military and civilian application • Information is power • More critical decision-based information is digitized and conveyed over an ever- expanding network of computers and other electronic devices • Deliberate actions taken to leverage and optimize these networks to improve human endeavor and enterprise or, in warfare, to gain information superiority and deny the enemy this enabling capability CNO ≈ CNA ≈ CND ≈ CNE • What’s the difference? • Computer
    [Show full text]
  • OSINT Handbook September 2020
    OPEN SOURCE INTELLIGENCE TOOLS AND RESOURCES HANDBOOK 2020 OPEN SOURCE INTELLIGENCE TOOLS AND RESOURCES HANDBOOK 2020 Aleksandra Bielska Noa Rebecca Kurz, Yves Baumgartner, Vytenis Benetis 2 Foreword I am delighted to share with you the 2020 edition of the OSINT Tools and Resources Handbook. Once again, the Handbook has been revised and updated to reflect the evolution of this discipline, and the many strategic, operational and technical challenges OSINT practitioners have to grapple with. Given the speed of change on the web, some might question the wisdom of pulling together such a resource. What’s wrong with the Top 10 tools, or the Top 100? There are only so many resources one can bookmark after all. Such arguments are not without merit. My fear, however, is that they are also shortsighted. I offer four reasons why. To begin, a shortlist betrays the widening spectrum of OSINT practice. Whereas OSINT was once the preserve of analysts working in national security, it now embraces a growing class of professionals in fields as diverse as journalism, cybersecurity, investment research, crisis management and human rights. A limited toolkit can never satisfy all of these constituencies. Second, a good OSINT practitioner is someone who is comfortable working with different tools, sources and collection strategies. The temptation toward narrow specialisation in OSINT is one that has to be resisted. Why? Because no research task is ever as tidy as the customer’s requirements are likely to suggest. Third, is the inevitable realisation that good tool awareness is equivalent to good source awareness. Indeed, the right tool can determine whether you harvest the right information.
    [Show full text]
  • Maltego Transforms for Threatqtm
    MALTEGO TRANSFORMS FOR THREATQ MALTEGO TRANSFORMS FOR THREATQTM Maltego Transforms for ThreatQTM enables users of Maltego to query ThreatQ for information on elements that could be part of an investigation. These transforms extend the visibility of Maltego to include threat intelligence stored within ThreatQ, including commercial, industry, private, OSINT (open source intelligence) and internal sources, so you can visualize and discover additional data relationships. Since ThreatQ is commonly deployed in private networks that are not directly accessible over the Internet, Maltego Transforms for ThreatQ are delivered WHAT IS MALTEGO? as ‘local transforms’ that can be installed locally on systems running the Maltego (by Paterva) is an Maltego client software. This ensures that connectivity between the client and interactive data mining tool the ThreatQ instance is direct and does not risk sending data to public that renders directed graphs for transform servers. link analysis. The tool is used in online investigations for finding INTEGRATION BENEFITS: relationships between pieces of information from various • Query information from all Threat Library sources quickly to find additional sources located on the Internet. context to support an investigation • Correlate internal and external threat data to accelerate online WHAT IS THREATQ? investigations ThreatQ is a threat intelligence platform (TIP) that enables users • Find relationships between threats, adversaries, indicators and incidents to build a custom Threat Library™ • Easily perform
    [Show full text]
  • The Extension and Customisation of the Maltego Data-Mining Environment Into an Anti-Phishing System
    The Extension and Customisation of the Maltego Data-Mining Environment into an Anti-Phishing System Submitted in partial fulfilment of the requirements of the degree of Bachelor of Science (Honours) of Rhodes University Matthew Marx Grahamstown, South Africa November 2, 2014 Contents 1 Introduction 1 1.1 Problem Statement and Research Goals . 1 1.2 Scope . 2 1.3 Document Structure . 3 2 Background 4 2.1 History and background . 4 2.1.1 Phishing and Pharming . 5 2.1.2 The Anatomy of a phishing attack . 5 2.2 The cost of a phishing attack . 7 2.2.1 Phishing and Data Breaches . 8 2.3 Online Identity . 9 2.3.1 ICANN . 9 2.3.2 WHOIS . 10 2.3.3 Certificate Authorities . 12 2.3.3.1 The role of Certificate Authorities in Phishing and Anti- Phishing . 12 2.3.4 PhishTank . 14 1 CONTENTS 2 2.4 Types of phishing attacks . 14 2.4.1 Clone Phishing . 14 2.4.2 Tabnabbing . 16 2.4.3 Spear Phishing . 16 2.5 Anti-Phishing methodologies . 17 2.5.1 Anti-Phishing Collectives . 18 2.5.1.1 Anti-Phishing Work Group . 18 2.5.1.2 US-CERT . 18 2.5.1.3 PhishTank . 19 2.5.2 Website take down . 19 2.5.3 Browser Anti-Phishing mechanisms . 19 2.5.3.1 Anti-Phishing Heuristics . 20 2.5.3.2 Phishing Blacklists . 20 2.5.4 Email Filtering and Content Filtering . 20 2.6 Abuse Reporting Mechanisms . 21 2.6.1 Blacklisting services . 21 2.6.2 Placing an abuse report with domain registrar .
    [Show full text]
  • An Introduction to Open Source Intelligence
    An introduction to Open Source Intelligence www.8arc.com Introduction www.8arc.com Open Source Intel: what is it? where to find it? and why do we need it? www.8arc.com Data Information Intelligence www.8arc.com Closed vs Open Source Closed Open • Internal Corporate Information • Accounts • Intelligence Database • Whois • Risk Management Documents • Google (search engines) • Partner (Agency) Data • Public facing documents • Profiles: current + previous • Website Analytics (Internal) • News Channels • BI Data • Peer to Peer Forum • Financial Data • Website Analytics (External) • Intellectual Property • Social Media • CRMs • Company Information • HR records • Personnel details www.8arc.com “When I took office, only high energy physicists had ever heard of what is called the World Wide Web, now even my cat has it’s own page.” Bill Clinton, ex American President www.8arc.com We’re always looking for entities and links! The more we have the clearer the picture www.8arc.com Investigation Environment www.8arc.com Things to consider? • Stand alone network/machine • Dedicated broadband – dynamic IP address (mobile broadband) • Back up broadband & network/machine • Standard software – anti virus, firewall, IDS / IPS/ Operating System, browser etc. • Specialist software – OSINT / intelligence / evidential software & capture tools • Online legends • Visualisation Tools • Build a jumpkit www.8arc.com Also consider… • Define a set file structure • Set a file naming convention • Keep an investigation log / workbook • Investigation Plan • Risk assessment
    [Show full text]
  • Open Source Intelligence Tools and Resources Handbook
    OPEN SOURCE INTELLIGENCE TOOLS AND RESOURCES HANDBOOK 0 OPEN SOURCE INTELLIGENCE TOOLS AND RESOURCES HANDBOOK 2018 Aleksandra Bielska Natalie Anderson, Vytenis Benetis, Cristina Viehman 1 Foreword I am delighted to share the latest version of our OSINT Tools and Resources Handbook. This version is almost three times the size of the last public release in 2016. It reflects the changing intelligence needs of our clients in both the public and private sector, as well as the many areas we have been active in over the past two years. No list of OSINT tools is perfect, nor is it likely to be complete. Indeed, such is the pace of change that by the time you read this document some of our suggestions may have been surpassed or have ceased to exist. Regrettably, today's tool might also be tomorrow's vulnerability. To counter the first problem, we have included a list of toolkits provided by other OSINT practitioners working to improve the state-of-the-art. To manage the second, we recommend that all tools be tested in a secure computing environment whenever possible. Work on the next iteration of the Handbook has already begun. For now, I hope this version contributes to improving your efficiency and effectiveness as a researcher, analyst, investigator or general OSINT practitioner. Please feel free to share it with your colleagues. To encourage its broadest possible dissemination, we are publishing the Handbook under a Creative Commons CC BY License. I would like to end by thanking my colleagues at i-intelligence for their efforts in compiling the Handbook.
    [Show full text]