Iptrust Knowledge API Technical Documentation
ipTrust Knowledge API Technical Documentation
1 PROPRIETARY AND CONFIDENTIAL
Table of Contents INTRODUCTION ...... 3 IPTRUST KNOWLEDGE API – BENEFITS ...... 3 PRODUCT DESCRIPTION...... 4 DELIVERABLE DESCRIPTION ...... 4 IPTRUST KNOWLEDGE API DATA DESCRIPTION ...... 4 IPTRUST KNOWLEDGE API DESCRIPTION...... 5 JSON Format Delivery...... 5 XML Format Delivery...... 6 CSV Format Delivery...... 6 IPTRUST SCORING METHODOLOGY...... 6 VALUE DEFINES ...... 7 SCORING ...... 7 RESEARCH METHODOLOGY...... 7 PASSIVE INSPECTION ...... 8 BOTNET SINKHOLE NETWORK ...... 8 FEATURE SETS...... 8 DATA & CORRELATION DETAILS ...... 8 Global Geo-location and Organization...... 8 Malicious Networks...... 8 Botnet Sinkholes...... 9 Intrusion Detection Systems (IDS) Feeds ...... 9 POINT OF CONTACT ...... 9 ABOUT ENDGAME SYSTEMS ...... 9
2 PROPRIETARY AND CONFIDENTIAL
Introduction Reputation systems today are one- dimensional, focusing primarily on IP Ac vity measuring spam email to determine if an Internet Protocol (IP) address is “good” Other AS Threats or “bad.” The ipTrust Knowledge API leverages Internet-wide intelligence and sophisticated multivariate analysis to compute a rational and useful metric for Time Interval Organiza on the overall trustworthiness of any given IP address. ipTrust Knowledge API is a Events cloud-based information service designed Loca on Detected to integrate rich IP reputation information into any at-risk system or Surrounding application without any expensive on-site Nodes infrastructure.
Figure 1 - Example of the confidence scoring representation
This representation shows a scored IP in which we have seen activity,
but not any malicious events. The weight was driven down based on locale and surrounding nodes.
ipTrust Knowledge API – Benefits Access to complete, in-depth IP reputation data is invaluable in being cognizant of the threat associated with both known and unknown connections. ipTrust Knowledge API offers customers:
• Leverage the full capacity of Endgame Systems’ (“Endgame”) internet threat intelligence apparatus via a lightweight and easy to use web-based API • Updated hourly on the latest threats, largest botnets, most relevant security events, and our correlated decision models • In-the-cloud or hosted deployment • Zero additional equipment needed to support a successful implementation • Inject true, real-time threat intelligence data into existing fraud and security management processes • Correlate existing fraud cases to specific security threats • Prevent fraud by leveraging granular external threat intelligence into real-time authentication systems • Improve internal enterprise security controls by leveraging external threat intelligence
3 PROPRIETARY AND CONFIDENTIAL
Product Description The ipTrust Reputation API delivers to the customer the best, most complete, and most versatile IP reputation service on the market. While others focus on single-factor scoring, ipTrust provides a much richer view of the true risk associated with any IP address in near real-time.
Endgame aggressively harvests, analyzes, and classifies malware and botnet samples obtaining information used for its IP address reputation and scoring. IP address scores are based on malicious events, event frequency, duration between events and other risk factors. All of these efforts are condensed into a single score, which puts ipTrust users in the driver’s seat to completely customize responses to specific events based on your unique and specific requirements.
Deliverable Description ipTrust Knowledge API Data Description ipTrust computes its risk score based on many variables for over 250 million Internet hosts. Our data set includes identification and descriptions of many types of devices including the following:
• Botnet tracking • Downadup/Conficker • Mariposa • BlackEnergy • Zeus • PoisonIvy • Precision weighting factors • Fewer false positives, time-scored results • Accounting for DHCP churn • Accounting for proxy hosts • Accounting for the age of events • Accounting for type of malicious traffic seen and how often events are triggered • Botnet controller or command and control nodes • Issue commands to bot-infected hosts • Anonymous Web Connections • TOR exit nodes and Open/Anonymous proxies • Worm infected hosts - Slammer, Code Red, etc. • Active hostile hosts • Brute force attacks • Malware propagation • Malicious behavior • Spam / Firewall Blacklists
4 PROPRIETARY AND CONFIDENTIAL
ipTrust Knowledge API Description The Application Programming Interface (API) calls to Endgame cloud-based instances follow industry standards, which include three types of return formats (e.g. XML, JSON, CSV).
URL: /confidence.{format} Formats: XML, JSON, CSV Methods: GET or POST Requires: APIKEY API Rate: Limited
Query String Parameters:
addr 1.2.3.4 An IP address in dotted quad notation (comma can act as a delimiter for multiple values in the same submission) key {UID} The API Key assigned to your account.
Access to the API is located: http://api.endgamesystems.com/xml-rpc/confidence.{format}?key={APIKEY}&q={QUERYLIST}
JSON Format Delivery Request: http://api.endgamesystems.com/xml-rpc/confidence.json?key={APIKEY}&q={QUERY LIST}
Response: { "hosts": [ { "addr": "200.105.189.113", "confidence": "0.90889213", "events": { "Conficker A/B": "1273724080", "Conficker C": "1273455293", "Mariposa": "1270076434" } } ] }
Inside the response is an array of hosts (one for each IP requested to be queried). Within that host record exists the last event for significant categories (e.g. Conficker A/B variant, Conficker C, Mariposa).
The confidence score is represented as a floating point to be interpreted as a percentage value between 0% - 100%. The event timestamp is recorded as the number of seconds since the standard UNIX epoch.
5 PROPRIETARY AND CONFIDENTIAL
XML Format Delivery 200
XML provides the same criteria as JSON, but in XML version="1.0" encoding="UTF-8" canonicalization format. CSV Format Delivery 200.105.189.113,0.90889213
CSV is the most limited form of return. Use CSV if you do not need insight into the last offending malicious categories seen for the queried IP. CSV will only return the confidence level. ipTrust Scoring Methodology While tracking, monitoring, reverse engineering, and analyzing malicious software (e.g. Bots), Endgame Systems creates weighted scales based on several criteria (see Figure 1) and change those weights based on current events, anomalous behavior, or various detected changes. The general scoring model for the confidence score per IP address is defined below.
6 PROPRIETARY AND CONFIDENTIAL
Value Defines Hours Is → (Short Interval) = (Hours in a Week) = 168 Week
⎛ ⎛ 1 ⎞ ⎞ Hours Im → (Medium Interval) = Hours in ⎜ ⎟ Year = 2184 ⎝⎜ ⎝ 4 ⎠ ⎠⎟ ⎛ 1 ⎞ Year ⎝⎜ 4 ⎠⎟ ⎛ ⎛ 1⎞ ⎞ Hours Il → (Long Interval) = Hours in ⎜ ⎟ Year = 4368 ⎝⎜ ⎝ 2⎠ ⎠⎟ ⎛ 1⎞ Year ⎝⎜ 2⎠⎟
Scoring dT = (Current Time(in Hours) − Event Time(in Hours))
3 ⎛ ⎛ dT ⎞ ⎞ if dT ≤ Is → Score = ⎜ −⎜ ⎟ + 1⎟ x 0.25 + 0.75 ⎝ ⎝ Is ⎠ ⎠
1.5 ⎛ ⎛ dT ⎞ ⎞ else if dT ≤ Im → Score = ⎜ −⎜ ⎟ + 1⎟ x 0.25 + 0.50 ⎝ ⎝ Im ⎠ ⎠
1.5 ⎛ ⎛ dT ⎞ ⎞ else if dT ≤ Il → Score = ⎜ −⎜ ⎟ + 1⎟ x 0.40 + 0.10 ⎝ ⎝ Il ⎠ ⎠
else → Score = 0.10
When Endgame Systems has never seen an event on a particular IP query, we will always return a score of 0.0. If an event has previously been captured, but sufficient time has lapsed (i.e. No event recorded within the long interval) a score of 0.10 we be returned and that IP address will never decay past 0.10. Additionally, the time periods we picked above are indicative of botnet observations and how rapidly infections diminish.
Research Methodology Endgame Systems has developed a unique methodology for monitoring behavior analysis on the global Internet via active and passive reconnaissance techniques. Endgame methods produce actionable intelligence by correlating the data and mapping all discovered malicious and compromised interconnected systems.
7 PROPRIETARY AND CONFIDENTIAL
Endgame tracks and correlates over 4 million unique systems per week spanning nearly every country in the world. Endgame’ research data is comprised of event information for infected or malicious nodes and corresponding metadata to describe these events.
Passive Inspection Endgame non-intrusively collects intelligence through various detection methods focused on passive discovery of compromised and malicious hosts. This determines who is currently compromised, misconfigured, unpatched, and vulnerable to intrusion. This method also determines the approximate location of hosts through IP geo-location techniques including city, country, AS Number, and AS Name.
Botnet Sinkhole Network It is common for botnets and malware networks to utilize multiple domains simultaneously for Command and Control. A sinkhole allows the capture of command and control communication trying to occur within the master and slaves (or zombies). The right intelligence allows for pre- registering domains used by the botnet giving a higher precision of visibility into the bot army.
Feature Sets Endgame research data is comprised of many heterogeneous and disparate data feeds containing over a dozen attributes collected about known suspicious or malicious hosts on the global Internet. Endgame collects the data in raw unstructured format, fuses and correlates the data, and unifies the data into a highly structured format.
Data & Correlation Details Global Geo-location and Organization This capability associates IP address ranges to organizations such as: universities or schools, telecommunication service providers, businesses, and government/military entities. Organization names lack uniformity in their structure and therefore could exist multiple variants for a single organization. Additionally, the feature provides geo-location information on IP address ranges (i.e. latitude and longitude coordinates). Geo-location information is only accurate to the geographical center of the smallest geographical boundary within which the IP address range is identified; country, region, or city.
Malicious Networks Endgame tracks information on botnet activity on the Internet and is able to track hosts that have been absorbed into and are active on one of several botnets. Data available includes host IP address, approximate time activity of occurrence, transport and application layer protocols used during the communication and information on the controlling botnet the host is participating in.
8 PROPRIETARY AND CONFIDENTIAL
Some of the botnets tracked include Storm and Kraken. Descriptive content on each botnet is provided, including URLs known to be associated with a given botnet and MD5 hashes of various versions of botnet binaries.
Botnet Sinkholes Botnet sinkholes maintained by Endgame collect information on hosts infected by various bots including Confickr A, B and C as well as newer botnets such as Mariposa. These bots (or drones) are trying to connect to a malicious URL for updates. Botnet sinkholes are useful to collect information about specific bots, as well as metadata including URLs, browser user-agent strings and command and control information.
Intrusion Detection Systems (IDS) Feeds Alongside the sinkhole network, IDS sensors are deployed watching for malicious traffic on major egress/ingress points for critical Internet infrastructure. This allows the ability to watch for known command and control connections to any of the bots currently being tracked by correlating the data in and applying the appropriate policies to match any changes detected. This provides the capability to track the rise/demise of worm propagation.
Point of Contact All questions may be submitted via email to [email protected], which is monitored by the Engineering personnel that designed and implemented the ipTrust API service.
About Endgame Systems Based in Atlanta, GA, Endgame Systems (“EGS”) is a privately held U.S. company providing IP reputation technology. Comprised of highly skilled information security veterans, Endgame is exclusively aligned to support the mission and unique challenges of our clients. Our team consists of world-class security experts, thought-leaders, and practitioners dedicated to solving 21st century problems with advanced, next-generation security solutions. Formed by industry veterans previously employed by Internet Security Systems, Inc. (ISS), a market leading commercial network vulnerability assessment, network intrusion prevention (IPS), and network security intelligence vendor, Endgame Systems’ founders and employees have extensive background in network and computer vulnerability research, design and implementation of defensive computer and network protection technologies.
9 PROPRIETARY AND CONFIDENTIAL