DOCSLIB.ORG

OSINT Handbook September 2020

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
OSINT Handbook September 2020 OPEN SOURCE INTELLIGENCE TOOLS AND RESOURCES HANDBOOK 2020 OPEN SOURCE INTELLIGENCE TOOLS AND RESOURCES HANDBOOK 2020 Aleksandra Bielska Noa Rebecca Kurz, Yves Baumgartner, Vytenis Benetis 2 Foreword I am delighted to share with you the 2020 edition of the OSINT Tools and Resources Handbook. Once again, the Handbook has been revised and updated to reflect the evolution of this discipline, and the many strategic, operational and technical challenges OSINT practitioners have to grapple with. Given the speed of change on the web, some might question the wisdom of pulling together such a resource. What’s wrong with the Top 10 tools, or the Top 100? There are only so many resources one can bookmark after all. Such arguments are not without merit. My fear, however, is that they are also shortsighted. I offer four reasons why. To begin, a shortlist betrays the widening spectrum of OSINT practice. Whereas OSINT was once the preserve of analysts working in national security, it now embraces a growing class of professionals in fields as diverse as journalism, cybersecurity, investment research, crisis management and human rights. A limited toolkit can never satisfy all of these constituencies. Second, a good OSINT practitioner is someone who is comfortable working with different tools, sources and collection strategies. The temptation toward narrow specialisation in OSINT is one that has to be resisted. Why? Because no research task is ever as tidy as the customer’s requirements are likely to suggest. Third, is the inevitable realisation that good tool awareness is equivalent to good source awareness. Indeed, the right tool can determine whether you harvest the right information. It follows that the more tools you have in your portfolio, the more flexible your OSINT capabilities are likely to be. Finally, the process of compiling this handbook is an intelligence exercise in its own right, alerting us to where we are at as a community and the challenges we are likely to face going forward – not least of which are disinformation, the fracturing of the internet, the proliferation of niche social media platforms, and the urgent need for better tools to monitor and analyse the content they provide. Our hope is that this Handbook helps you plug any gaps in your collection efforts and alerts you to the many tools and resources you can consult for better intelligence, including on the discipline of OSINT itself. Of course, the Handbook makes no claim to being complete. Something new will doubtless pop-up tomorrow, and I suspect there are hundreds of tools we haven’t even come across at all. With your help, we can address any omissions. In any event, the usual caveats apply. While every effort has been made to test and validate these resources, we caution that today’s wonder tool may be tomorrow’s security liability. Thus, whenever testing a new script, application or browser extension, please do so in a secure environment to minimise any risk to your assets or operations. 3 I would like to thank my colleagues at i-intelligence for their efforts in compiling this Handbook. Much of the credit belongs to Aleksandra Bielska who, as our Head of Training, evaluates hundreds of tools every year to determine which ones make it into our courses. I am also indebted to Yves Baumgartner and Vytenis Benetis for routinely updating us on new solutions. Thanks too are also due to Noa Kurz who worked tirelessly over the summer to help us get this document ready. I close by acknowledging the contributions of the many students, practitioners and Twitter followers we’ve interacted with over the past two years. Your generosity, insights, suggestions and support are much appreciated. Chris Pallaris Director, i-intelligence GmbH 4 Table of Contents Foreword ............................................................................................................................. 3 Search Tools ..................................................................................................................... 17 General Search .......................................................................................................................... 17 Meta Search ............................................................................................................................... 18 Visual Search and Clustering Search Engines ......................................................................... 20 Decentralised Search Engines .................................................................................................. 21 Charitable Search Engines ........................................................................................................ 22 Code Search .............................................................................................................................. 23 File and FTP Search ................................................................................................................... 25 Open Database, Directories and Services Search .................................................................. 26 Similar Sites Search ................................................................................................................... 27 Children-Friendly Search Engines ............................................................................................ 28 National Search Engine Directories ......................................................................................... 29 Custom Search ........................................................................................................................... 30 Specialty Search Engines .......................................................................................................... 31 Search Tools ............................................................................................................................... 32 Social Media ..................................................................................................................... 34 Major Social Networks .............................................................................................................. 34 Social Media Search and Monitoring ...................................................................................... 37 Working with Hashtags ............................................................................................................. 39 Finding Influencers and Other Social Media Users ................................................................ 41 Social Media Management and Content Discovery ............................................................... 42 Social Media Analytics .............................................................................................................. 43 Social Media Tools: Twitter ...................................................................................................... 44 Social Media Tools: Facebook ................................................................................................. 49 Social Media Tools: Instagram ................................................................................................. 51 5 Social Media Tools: Pinterest ................................................................................................... 55 Social Media Tools: Reddit ....................................................................................................... 56 Social Media Tools: Voat .......................................................................................................... 59 Social Media Tools: VKontakte ................................................................................................. 60 Social Media Tools: LinkedIn .................................................................................................... 61 Social Media Tools: Github ...................................................................................................... 63 Social Media Tools: TikTok ....................................................................................................... 65 Messaging and Chat Tools ....................................................................................................... 66 Social Media Tools: Snapchat .................................................................................................. 68 Social Media Tools: WhatsApp ................................................................................................ 69 Social Media Tools: Skype ........................................................................................................ 70 Social Media Tools: Kik ............................................................................................................. 71 Social Media Tools: Telegram .................................................................................................. 72 Social Media Tools: IRC ............................................................................................................ 74 Dating Apps and Sites .............................................................................................................. 75 Fitness Apps and Communities ............................................................................................... 76 Working with Blogs .......................................................................................................... 77 Blog Search ...............................................................................................................................
Load more

Recommended publications
  • The Internet and Drug Markets
    INSIGHTS EN ISSN THE INTERNET AND DRUG MARKETS 2314-9264 The internet and drug markets 21 The internet and drug markets EMCDDA project group Jane Mounteney, Alessandra Bo and Alberto Oteo 21 Legal notice This publication of the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) is protected by copyright. The EMCDDA accepts no responsibility or liability for any consequences arising from the use of the data contained in this document. The contents of this publication do not necessarily reflect the official opinions of the EMCDDA’s partners, any EU Member State or any agency or institution of the European Union. Europe Direct is a service to help you find answers to your questions about the European Union Freephone number (*): 00 800 6 7 8 9 10 11 (*) The information given is free, as are most calls (though some operators, phone boxes or hotels may charge you). More information on the European Union is available on the internet (http://europa.eu). Luxembourg: Publications Office of the European Union, 2016 ISBN: 978-92-9168-841-8 doi:10.2810/324608 © European Monitoring Centre for Drugs and Drug Addiction, 2016 Reproduction is authorised provided the source is acknowledged. This publication should be referenced as: European Monitoring Centre for Drugs and Drug Addiction (2016), The internet and drug markets, EMCDDA Insights 21, Publications Office of the European Union, Luxembourg. References to chapters in this publication should include, where relevant, references to the authors of each chapter, together with a reference to the wider publication. For example: Mounteney, J., Oteo, A. and Griffiths, P.
    [Show full text]
  • Fraud and the Darknets
    OFFICE OF THE INSPECTOR GENERAL U.S. Department of Education Technology Crimes Division Fraud And The Darknets Thomas Harper Assistant Special Agent in Charge Technology Crimes Division OFFICE OF THE INSPECTOR GENERAL U.S. Department of Education Technology Crimes Division What is an OIG? • Established by Congress • Independent agency that reports to Congress • Agency head appointed by the President and confirmed by Congress • Mission: protect the taxpayer’s interests by ensuring the integrity and efficiency of the associated agency OFFICE OF THE INSPECTOR GENERAL U.S. Department of Education Technology Crimes Division Technology Crimes Division • Investigate criminal cyber threats against the Department’s IT infrastructure, or • Criminal activity in cyber space that threatens the Department’s administration of Federal education assistance funds • Investigative jurisdiction encompasses any IT system used in the administration of Federal money originating from the Department of Education. OFFICE OF THE INSPECTOR GENERAL U.S. Department of Education Technology Crimes Division Work Examples • Grade hacking • Computer Intrusions • Criminal Forums online selling malware • ID/Credential theft to hijack Student Aid applications • Misuse of Department systems to obtain personal information • Falsifying student aid applications by U.S. government employees • Child Exploitation material trafficking OFFICE OF THE INSPECTOR GENERAL U.S. Department of Education Technology Crimes Division Fraud and the Darknets Special Thanks to Financial Crimes Enforcement Network (FINCEN) OFFICE OF THE INSPECTOR GENERAL U.S. Department of Education Technology Crimes Division Fraud and the Darknets OFFICE OF THE INSPECTOR GENERAL U.S. Department of Education Technology Crimes Division OFFICE OF THE INSPECTOR GENERAL U.S. Department of Education Technology Crimes Division OFFICE OF THE INSPECTOR GENERAL U.S.
    [Show full text]
  • Atom-Feeds for Inspire
    ATOM-FEEDS FOR INSPIRE - Perspectives and Solutions for INSPIRE Download Services in NRW WWU Münster Institute for Geoinformatics Heisenbergstraße 2 48149 Münster Masterthesis in Geoinformatics 1. Supervisor: Hon.-Prof. Dr. Albert Remke 2. Supervisor: Dr. Christoph Stasch Arthur Rohrbach [email protected] November 2014 I Plagiatserklärung der / des Studierenden Hiermit versichere ich, dass die vorliegende Arbeit ATOM-Feeds for INSPIRE – Perspectives and Solutions for Download Services in NRW selbstständig verfasst worden ist, dass keine anderen Quellen und Hilfsmittel als die angegebenen benutzt worden sind und dass die Stellen der Arbeit, die anderen Werken – auch elektronischen Medien – dem Wortlaut oder Sinn nach entnommen wurden, auf jeden Fall unter Angabe der Quelle als Entlehnung kenntlich gemacht worden sind. _____________________________________ (Datum, Unterschrift) Ich erkläre mich mit einem Abgleich der Arbeit mit anderen Texten zwecks Auffindung von Übereinstimmungen sowie mit einer zu diesem Zweck vorzunehmenden Speicherung der Arbeit in eine Datenbank einverstanden. _____________________________________ (Datum, Unterschrift) II Abstract One proposed solution for providing Download Services for INSPIRE is using pre- defined ATOM-Feeds. Up to now the realization of ATOM-Feeds in NRW is still at the beginning. This master thesis will investigate possible solutions in order to help developing a methodology for the implementation of pre-defined INSPIRE Download Services in NRW. Following research questions form the basis of the thesis: What implementing alternatives for automatic generation of ATOM-Feeds based on ISO metadata exist? How do the identified solutions suit in order to fulfil the requirements of NRW? In the first step required technologies are introduced, including ATOM, OpenSearch and OGC standards.
    [Show full text]
  • Match Group Stock Pitch
    Match Group Stock Pitch Analyst: James Campion First Things First Who in this room has used Tinder, Hinge, or another dating app? 2 Disclaimer I made these slides in 48 hours and took a lot of screenshots. You are allowed to take screenshots and make non-aesthetically pleasing slides for Investment Club (content matters more), but for SIBC and your full-time job do not do this! Thesis: 1. The Match Group currently holds a monopoly on the online- dating market, which is benefitting from changing social standards and increased connectivity 2. The Match Group has developed a diverse portfolio of brands, both through organic and inorganic methods, which enables them to serve customers of different ages and demographics 3. The Company’s combination of operating leverage and scale have led to expanding margins, driving a cash flow machine that should return cash to shareholders after the IAC spin-off 4. Valuation has gotten “frothy” but I still there is value as a long- term shareholder 4 Industry Focus Why online dating? More Couples Are Swiping Right ▸ In 2018, seven of the 53 couples profiled in the Vows column met on dating apps. ▸ And in the Times’ more populous Wedding Announcements section, 93 out of some 1,000 couples profiled this year met on dating apps ▸ The year before, 71 couples whose weddings were announced by the Times met on dating apps. Thanks to former NDIC President & VP, Dan McMurtie & Alex Draime for their report on the 6 dating market! Why Would Anyone Date Online?! ▸ Just a crazy thought, but what if in 10 years we think it is crazy that anyone met in person ▸ There are almost 8 billion people in the world, and we each get to meet about ~10,000 – And you’re telling me that you found your soulmate?! ▸ In the future, it might make more sense for machine learning trained algorithms to pair couples based on a number of different factors – There are just so many things it takes so long to tell someone about yourself (i.e.
    [Show full text]
  • ISCRAM2005 Conference Proceedings Format
    Yee et al. The Tablecast Data Publishing Protocol The Tablecast Data Publishing Protocol Ka-Ping Yee Dieterich Lawson Google Medic Mobile [email protected] [email protected] Dominic König Dale Zak Sahana Foundation Medic Mobile [email protected] [email protected] ABSTRACT We describe an interoperability challenge that arose in Haiti, identify the parameters of a general problem in crisis data management, and present a protocol called Tablecast that is designed to address the problem. Tablecast enables crisis organizations to publish, share, and update tables of data in real time. It allows rows and columns of data to be merged from multiple sources, and its incremental update mechanism is designed to support offline editing and data collection. Tablecast uses a publish/subscribe model; the format is based on Atom and employs PubSubHubbub to distribute updates to subscribers. Keywords Interoperability, publish/subscribe, streaming, synchronization, relational table, format, protocol INTRODUCTION After the January 2010 earthquake in Haiti, there was an immediate need for information on available health facilities. Which hospitals had been destroyed, and which were still operating? Where were the newly established field clinics, and how many patients could they accept? Which facilities had surgeons, or dialysis machines, or obstetricians? Aid workers had to make fast decisions about where to send the sick and injured— decisions that depended on up-to-date answers to all these questions. But the answers were not readily at hand. The U. S. Joint Task Force began a broad survey to assess the situation in terms of basic needs, including the state of health facilities. The UN Office for the Coordination of Humanitarian Affairs (OCHA) was tasked with monitoring and coordinating the actions of the many aid organizations that arrived to help.
    [Show full text]
  • Assessing the Quality of Mobile Graphical User Interfaces Using Multi-Objective Optimization
    Noname manuscript No. (will be inserted by the editor) Assessing the Quality of Mobile Graphical User Interfaces using Multi-objective Optimization Makram Soui · Mabrouka Chouchane · Mohamed Wiem Mkaouer · Marouane Kessentini · Khaled Ghedira the date of receipt and acceptance should be inserted later Abstract Aesthetic defects are a violation of quality attributes that are symp-toms of bad interface design programming decisions. They lead to deteriorating the perceived usability of mobile user interfaces and negatively impact the Users eXperience (UX) with the mobile app. Most existing studies relied on a subjective evaluation of aesthetic defects depending on end-users feedback, which makes the manual evaluation of mobile user interfaces human-centric, time-consuming, and error-prone. Therefore, recent studies have dedicated their effort to focus on the definition of mathematical formulas that each targets a specific structural quality of the interface. As the UX is tightly dependent on the user profile, the combi-nation and calibration of quality attributes, formulas, and users characteristics, when defining a defect, is not straightforward. In this context, we propose a fully automated framework which combines literature quality attributes with the users profile to identify aesthetic defects of MUI. More precisely, we consider the mobile user interface evaluation as a multi-objective optimization problem where the goal is to maximize the number of detected violations while minimizing the detection complexity of detection rules and enhancing the interfaces overall quality in means M. Soui College of Computing and Informatics Saudi Electronic University, Saudi Arabia E-mail: [email protected] Mabrouka Chouchane School of computer science of Manouba, Tunisia E-mail: [email protected] Mohamed Wiem Mkaouer Rochester Institute of Technology E-mail: [email protected] Marouane Kessentini University of Michigan E-mail: [email protected] Khaled Ghedira Honoris United Universities E-mail: [email protected] 2 Makram Soui et al.
    [Show full text]
  • The Application Usage and Risk Report an Analysis of End User Application Trends in the Enterprise
    The Application Usage and Risk Report An Analysis of End User Application Trends in the Enterprise 8th Edition, December 2011 Palo Alto Networks 3300 Olcott Street Santa Clara, CA 94089 www.paloaltonetworks.com Table of Contents Executive Summary ........................................................................................................ 3 Demographics ............................................................................................................................................. 4 Social Networking Use Becomes More Active ................................................................ 5 Facebook Applications Bandwidth Consumption Triples .......................................................................... 5 Twitter Bandwidth Consumption Increases 7-Fold ................................................................................... 6 Some Perspective On Bandwidth Consumption .................................................................................... 7 Managing the Risks .................................................................................................................................... 7 Browser-based Filesharing: Work vs. Entertainment .................................................... 8 Infrastructure- or Productivity-Oriented Browser-based Filesharing ..................................................... 9 Entertainment Oriented Browser-based Filesharing .............................................................................. 10 Comparing Frequency and Volume of Use
    [Show full text]
  • Facebook Dating Application
    Facebook Dating Application A Business Plan by Koen Heine, Alex Paninder & Fai Nopsuwanwong Idea: A Facebook dating application that will revolutionize the way people meet and date on the Internet. By choosing a superior business model, our company will outperform our competitors and change the faith of single people forever. This business plan is confidential. Business idea in itself or information from this description may not be used, reproduced or made available to third parties without the prior written permission of Koen Heine. 5/4/2011 Final Hand in –Entrepreneurship Program Facebook Dating Application Executive Summary The Internet is becoming an increasingly big part of our lives. We read the news, discover new music, watch video clips and communicate with social contacts. Meeting a romantic partner is a logical extension of the technology. This is where dating websites come in. However, research has shown that while people spend a lot of time on these websites, they are dissatisfied with the payoff. Despite the problems with current online dating solutions, the industry is estimated to be worth about $3-4 billion a year. The business model of choice for existing websites can be separated in monthly subscriptions and free dating sites that make money through advertising. According to competitive research by OkCupid, men are usually the contact initiator. On the paid dating website eHarmony, a man can expect a reply only 30% of the time, mainly because the receiving woman is not a paying member and is unable to reply. Scholarly research on a major American dating websites finds that women reply only 15.9% of the time.
    [Show full text]
  • Cursors with Sparkle Tail Tumblr
    Cursors with sparkle tail tumblr FAQS Behan boss ki chudai Cursors with sparkle tail tumblr coty depablo twitter Cursors with sparkle tail tumblr Cursors with sparkle tail tumblr fist on keyboard Cursors with sparkle tail tumblr Skyy black retired Global Audio cay cau ho bieu chanhTumblr Mouse Sparkle Trails SNAZZYSPACE IS IN NO WAY AFFILIATED WITH ANY SOCIAL NETWORKING SITE, WE SIMPLY PROVIDE RESORUCES FOR THEM. ALL IMAGES AND LOGOS ARE THE LEGAL PROPERTY OF THE INDIVIDUALS THEY REPRESENT. Suivez l'évolution de l'épidémie de CoronaVirus / Covid19 dans le monde. Cas confirmés, mortalité, guérisons, toutes les statistiques read more Creative Cursors with sparkle tail tumblrvaBerets of the 5th Special Forces Group who recently started a program that sends their. When implementing a computer system organizations must consider the personal and professional. 4 and would in all likelihood be classified as No. Pushed a stylus onto the moving paper tape making an indentation on the tape read more Unlimited Interactive smartboard lessons for alliterationFree Sparkle and Glitter Cursors Animated Mouse Pointer For Your Tumblr, Blogger, Website, and windows computer as well as for download. Results 1 - 24 of 70. Cute Candy Hamburger Cursor. ok so i was bored and i took a picture of. # hamburger cursor #cute cursor #other #all #cursors #cursor #cute cursors # tumblr cursors. cute, animated, cool, glitter, sexy, and so. Tumblr Mouse Sparkle Trails. This tumblr code will put sparkles to follow your mouse cursor! Just paste the following code right before the </head> part of your . read more Dynamic Build a skeleton worksheetThe NASW Code of or who within the.
    [Show full text]
  • Free Your Android! Not Free As in Free Beer About the FSFE This flyer Was Printed by the Free Software You Don't Have to Pay for the Apps from F-Droid
    Free as in Freedom Free Your Android! Not Free as in Free Beer About the FSFE This flyer was printed by the Free Software You don't have to pay for the apps from F-Droid. A lot Foundation Europe (FSFE), a non-profit organi- of applications from Google Play or Apple's App Store sation dedicated to promoting Free Software Get a are also free of charge. However, Free Software is not and working to build a free digital society. about price, but liberty. Free App Store Access to software de- When you don't control a program, the program termines how we can take for Your Android controls you. Whoever controls the software therefore part in our society. There- controls you. fore, FSFE is dedicated to ensure equal access and For example, nobody is allowed to study how a non- participation in the infor- free app works and what it actually does on your mation age by fighting for phone. Sometimes it just doesn't do exactly what you digital freedom. want, but there are also apps that contain malicious features like leaking your data without your knowledge. Nobody should ever be forced to use software that does not grant the freedoms to use, Running exclusively Free Software on your device puts study, share and improve the software. You you in full control. Even though you may not have the should have the right to shape technology as skills to directly exercise all of your freedom, you you see fit. benefit from a vibrant community that is enabled by freedom and uses it collaboratively.
    [Show full text]
  • Consumer Vs Professional Cloud Storage: What You Need to Know
    WHITEPAPER Consumer vs Professional Cloud Storage: What You Need to Know © 2019 MSP360. All rights reserved. msp360.com Consumer vs Professional Cloud Storage What you need to know MSP360 doesn’t support backup to consumer-grade cloud storage in its products and doesn’t recommend backing up mission-critical data to this type of cloud. See the full list of supported cloud storage options Intro Whether at work or at home, storing your digital assets is necessary, whether you need to save family photos or work documents. In response to these digital requirements, cloud data storage has become a popular option as it ensures the data is available when needed, regardless of location. There are dozens of cloud storage vendors providing cloud storage, but what are the real differences between consumer and professional-focused cloud storage options. You may already use Google Drive for storing and sharing your data. You may even consider it as a form of backup, but is it substantial? There are two types of cloud storage solutions that are commonly used for preserving essential data: Consumer solutions These solutions target consumers and provide a low-cost means for storing personal information like you images, documents, and music. Examples include: Amazon Cloud Drive, Google Drive, and Microsoft OneDrive. These solutions are quite handy and easy to use. Professional solutions These solutions primarily have more advanced functionality with features that make them a good fit for businesses (but consumers can use them too). Examples include: Amazon S3, Microsoft Azure Storage, and Google Cloud Storage. All are flexible and secure, yet they can be complicated.
    [Show full text]
  • Observer5-2015(Health&Beauty)
    the Jewish bserver www.jewishobservernashville.org Vol. 80 No. 5 • May 2015 12 Iyyar-13 Sivan 5775 Nashville crowd remembers Israel’s fallen and celebrates its independence By CHARLES BERNSEN atching as about 230 people gath- ered on April 23 for a somber remem- brance of Israel’s fallen soldiers and Wterrors victims followed immediately by a joyful celebration of the 67th anniversary of the Jewish’ state’s birth, Rabbi Saul Strosberg couldn’t help but marvel. After all, it has been only eight years since the Nashville Jewish community started observing Yom Hazikaron, the Israeli equivalent of Memorial Day. Organized by several Israelis living in Nashville, including the late Miriam Halachmi, that first, brief ceremony was held in his office at Congregation Sherith For the third year, members of the community who have helped build relations between Nashville and Israel were given the honor of lighting Israel. About 20 people attended. torches at the annual celebration of Israel’s independence. Photos by Rick Malkin Now here he was in a crowd that of three fallen Israelis – a soldier killed in Catering and music by three Israeli Defense Martha and Alan Segal, who made filled the Gordon Jewish Community combat, a military pilot who died in a Force veterans who are members of the their first ever visits to Israel this spring Center auditorium to mark Yom training accident and a civilian murdered musical troupe Halehaka (The Band). on a congregational mission. Hazikaron and then Yom Ha’atzmaut, the in a terror attack. Their stories were pre- For the third year, the highlight of • Rabbi Mark Schiftan of The Temple Israeli independence day.
    [Show full text]