Calcium Vulnerability Scanner (CVS): a Deeper Look

1 Calcium Vulnerability Scanner (CVS): A Deeper Look

Sari Sultan and Ayed Salman

Abstract—Traditional vulnerability scanning methods are time-consuming several minutes to several hours per host. For example, an and indecisive, and they negatively affect network performance by generating exhaustive scanning policy, which provides the highest level high network traffic. In this paper, we present a novel vulnerability scanner that of assurance, tries to discover open ports for each connected is time-efficient, simple, accurate, and safe. We call it a Calcium Vulnerability device. Subsequently, discovered open ports are tested to Scanner (CVS). Our contribution to vulnerability scanning are the follow- identify running services. Afterwards, the scanner tries to ing: (i) minimize its required time and network traffic: compared to current technologies, we reduced the former by an average of 79% and the latter discover vulnerabilities associated with each service. This by 99.9%, (ii) increase its accuracy: compared to current technologies, we is a protracted process that suffers from numerous issues improved this by an average of 2600%, and (iii) enable the scanner to learn such as (1) generating high network traffic that might con- from previous scans in order to reduce future scanning time and enhance gest or even disrupt network operations, (2) indecisiveness accuracy: compared to current technologies, CVS reduced scanning time by because other network devices (e.g., a firewall) might drop an average of 97%. CVS enables a new frontier in vulnerability scanning scanning packets without being detected, and (3) different and allow for scalable and efficient deployment of such tools in large-scale scanning software providing different results due to lack of networks, containers, edge computing, and cloud computing. standardized procedures. Moreover, vulnerability scanners might alter the tested services because the scanner could Index Terms—Vulnerability Scanning; Security Assessments; Large-Scale Scanning; National Vulnerability Database; NVD; CVE send active probes that can change essential components such as databases. Sending malformed packets during scans might also contribute to destabilizing or causing a complete 1 Introduction crash of the system [4]. Although vulnerability scanning is Information technology (IT) components might suffer from an important tool to assess the security of IT devices, current security vulnerabilities that could allow adversaries to ex- solutions fail to provide an easy, fast, reliable, decisive, and ploit them for unintended purposes. The number of new simple method to perform scans. vulnerabilities per year increased by 15 times from 1999 to Vulnerability scanning is an essential service provided 2019. Additionally, a myriad of security breaches targeted by CERTs/CSIRTs in their proactive assessments landscape. virtually all aspects of the current technological landscape: The aforementioned downsides of vulnerability scanners afflicted individuals (e.g., identify theft), various organiza- limit their effectiveness and scalability. Scanning a single tions (e.g., ransomware that held many hospitals to ransoms target requires about 5 minutes to 4 hours1. Additionally, we [1]), governments (e.g., the Stuxnet worm that stymied Ira- found that scanning the same host more than once requires nian nuclear facilities [2]), and even nations (e.g., Facebook the same scanning time, as the scanner cannot improve itself Cambridge Analytica files that affected millions of Ameri- from previous scans of the same host. cans). This highlighted the importance of rigorous security This led us to a fundamental research question: is it testing to identify and fix vulnerabilities. Vulnerability scan- possible for a national CERT (or any organization) to provide a ning is an integral component in security assessments and vulnerability scanning service for a country that has 10 million2 risk analysis. It plays a pivotal role in safeguarding IT assets hosts? The answer to this question is: no, not with feasible by identifying their security weaknesses, to enable fixing consideration for time and resources. There are two main them before being exploited by adversaries. Due to the obstacles to achieving this. First, many hosts hide behind importance of vulnerability scanning, some governmental a NAT IP address and do not have an accessible network and industrial standards consider periodical scans as a com- interface outside the NAT. Deploying a localized scanner for pulsory requirement (e.g., PCI-DSS, GDPR); others consider each network might resolve this issue. However, this would it as a recommended requirement (e.g., ISO/IEC 27001) [3]. lead to management overhead. Second, the time required for Vulnerability scanners evaluate targets based on a prede- scanning is very long. We have been optimistic in choosing fined policy. Policies differ in scanning times that range from

Sari Sultan, Founder of Calcium Software Solutions, Toronto, Canada, email:([email protected]). 1. Some references show that it takes from 20 minutes to 4 hours. Ayed Salman, professor at the Department of Computer Engi- However, our experimental results showed that some vulnerability neering, Kuwait University, P.O. Box 5969, Safat 13060, Kuwait scanners can finish the job in 5 minutes for TCP scans. email:([email protected]) 2. Assuming 10 million is the average country’s host population. 2 the 5 minutes3 per scan time; others estimated it as 20+ min- this issue, if applicable, or restrict access to the device. Oth- utes per host. However, even with such optimism, scanning erwise, the system remains vulnerable to exploits targeting 10 million hosts, each requiring ~5 minutes, is a huge task. that vulnerability. Increasing the level of parallelism might resolve this issue, Definition 1 (Vulnerability). "weakness in an information but it would probably lead to denial of service for the CERT system, system security procedures, internal controls, or im- or hosts because the generated traffic during a scan is very plementation that could be exploited or triggered by a threat large. Based on our experimental results, scanning a single source." [5] host requires exchanging ~150,000 packets4 (for TCP scans only). Definition 2 (Vulnerability Assessment). "systematic exami- nation of an information system or product to determine the adequacy of security measures, identify security deficiencies, Contributions provide data from which to predict the effectiveness of proposed We propose a novel vulnerability scanner called CVS. It security measures, and confirm the adequacy of such measures covers all types of vulnerabilities (e.g., network, in-host), it is after implementation" [5] fast, scalable, accurate, and learns from previous scans. We To the best of our knowledge, there is no standardized compared CVS scanner to two vulnerability scanners: Ten- way to perform vulnerability scanning [4]. However, a able Nessus and Rapid7 Insight VM (Nexpose). Compared generalized approach followed by vulnerability scanners to Nessus, CVS reduced scanning time by an average of 83% is shown in Algorithm1. First, a scanner performs host for the worst-case scenarios and by 99.75% for the best-case discovery to identify the active targets. In a networked envi- scenarios. Compared to Nexpose, CVS reduced scanning ronment, this can be performed using different techniques time by an average of 30% for the worst-case scenarios and such as a ping or SYN scan, among other techniques. Nmap by 98.96% for the best-case scenarios. For both cases, CVS is a prominent tool for scanning that is also used by many reduced the network traffic by 99.9% and improved scanning vulnerability scanners to perform host discovery [12]. The accuracy by ~2600%. In this context, worst-case scenario active hosts’ ports will be scanned as well, in order to check refers to scanning a service for the first time, and best-case whether the port is open or closed. Second, if the port is scenario means scanning a service that has been scanned open, then this port scan will proceed to a service discovery previously. CVS uses the Common Platform Enumeration process to identify the running services. Afterwards, the (CPE) to the Common Vulnerabilities and Exposure (CVE) scanner will try different vulnerability payloads for the matching to perform vulnerability scanning based on au- identified service(s) in order to check whether they are tomatically generated CPEs from the properties of IT com- vulnerable or not. Figure1 shows a general process for ponents (e.g., installed program properties in a Windows vulnerability scanning. This is a lengthy process that can operating system). The client source code is available at take days or even weeks for scanning large networks [13]. https://github.com/SariSultan/CVS-client. Algorithm 1: Generalized traditional scanning pro- Organization cess The rest of this paper is organized as follows. Section2 1 ScanTargets inputs : (T): A set of targets to be scanned. supplies the background material and discusses the related T output: : Vulnerabilities set (set per target work. In Section3, we present the threat model, assump- V t ). 1, 2, , v, , V tions, and goals. Section4 discusses our methodology and ∈ T V { ··· ··· } ∀(v t). implementation. Section5 addresses methods and conven- ⇒ 2 ; tions used to generate search components in CVS. Our V ← ∅ 3 active HostDiscovery ;// Return active experimental results are discussed in Section6. Section7 T ← (T) presents the security analysis of CVS. Finally, Section8 targets. Error t . 4 foreach t active do concludes the paper. ∈ T 5 active PortScan t ;// Return active ports P ← ( ) for target t. Error p . 2 Background 6 foreach p active do ∈ P 7 active ServiceScan p ;// Return The National Institute of Standards and Technology (NIST) S ← ( ) defines a vulnerability as in Definition1. It also defines vul- active services for port p. Error s . 8 foreach s active do nerability assessment (also known as vulnerability analysis ∈ S 9 t + or vulnerability scanning) as in Definition2. Vulnerability V VulnerabilityScan s ;// Return scanning does not improve the system’s security unless the ( ) discovered vulnerabilities are fixed according to the organi- service vulnerabilities of service zation’s remediation strategy. For example, a vulnerability s. Error v . scanner could discover a critical vulnerability in a device 10 + t running the Windows 10 operating system. The system V V 11 return ; administrator should install the necessary patch to resolve V

3. We managed to achieve this number by disabling UDP ports scanning. Otherwise, it would take much longer to ﬁnish scans. 4. With Nessus and Rapid7 Nexpose scanners. 3 TABLE 1: Standards requirements related to vulnerability scanning

Document/Standard Description Requirements PCI-DSS [6] Standard for credit card merchants and service Requires periodical internal and external vulner- providers. ability scanning. NIST 800-53 [5] Special publication for security and privacy con- Has numerous vulnerability scanning require- trols for federal information systems and organi- ments such as RA-5, CA-8, SA-11, and RA-5(4). zations. NIST Cybersecurity Frame- NIST framework for improving critical infras- Has vulnerability scanning control DE.CM-8. work (CSF) [7] tructure cybersecurity. Contains best practices to improve security. CIS Critical Security Con- List of 20 eﬀective security controls for organiza- Vulnerability scanning is ranked as one of the trols [3] tion to improve their overall security practices. 20 important security controls. In particular, controls 4.1, 18.4, and 20.4 are concerned with vulnerability scanning. ISO/IEC 27002:2013 [8] Information security management system stan- Vulnerability monitoring is discussed in control dard. This document includes the best practices number 12.6.1. for information security controls. Cloud Security Alliance Addresses security principles important for Control TVM-02 address vulnerability scanning. (CSA) Cloud Controls cloud service providers. Matrix (CCM) [9] COBIT 4.1 & 5 [3] Is a good-practice framework developed by Build, Acquire, and Implement (BAI) domain ISACA for information technology (IT) manage- addresses vulnerability scanning requirements. ment and IT governance. New York State Department Governmental standard to regulate the ﬁnancial Requires vulnerability scanning twice a year (Sec- of Financial Services 23 NY- services companies. tion 500.05). CRR 500 [10] Health Insurance Portabil- Standards and regulations apply to health care Section 164.308(a)(ii)(A) addresses risk analysis, ity and Accountability Act providers. and Section 164.306 requires protection against (HIPAA) [11,3] possible threats.

TABLE 2: Vulnerability scanning types

Type Description Network- Performs host discovery, ports enumeration, services enumeration and tries different payloads for vulnerabili- based ties. Host-based More comprehensive than network-based scans because it covers local and remote vulnerabilities. The target host should run the scan locally. Wireless A special scan type to cover wireless protocols and configurations. based Application Scanning an application for application-specific vulnerabilities such as SQL injection. This is considered a risky based scan because it can alter the application [3].

(1) Check if the port is open Port 1

··· (2) If open: start service discovery scan

Port 445 Network Scanner (3) Try diﬀerent vulnerabilities for each discovered service ··· Target Host (4) Repeat for each port Port 65535

Fig. 1: Sketch of the major steps in the traditional vulnerability scanning process

Vulnerability scanning is an integral part of organi- compulsory or non-compulsory. Compulsory compliance zations in the 21st century because most of them rely drivers are usually governments or industry mandated on IT for their daily operations. Although there are no requirements [3]. For example, countries could require standardized methods to perform vulnerability scanning, organizations to adhere to specific rules, regulations, or several standards regulate vulnerability scanning outcomes. standards that require performing periodical vulnerability Vulnerability scanning compliance requirements are either scanning. Examples of those standards are: the Health Insur- 4 ance Portability and Accountability Act (HIPAA), the New port scanning for active Internet IP addresses: if the device York State Department of Financial Services, the General responds to requests, then it will be scanned and its services Data Protection Regulation (GDPR), and the Payment Card will be identified, based on the provided banner informa- Industry - Data Security Standard (PCI-DSS). Examples tion. Adversaries can use this information (which is pro- of standards that have vulnerability assessment as non- vided publicly on the project’s website) to identify potential compulsory requirements are: the ISO/IEC 27001:2013, the vulnerabilities in hosts. Some researchers call this operation Health Information Trust Alliance (HITRUST) CSF, and the contact-less active reconnaissance because the adversary does NIST SP 800-53. Table1 describes some standards in relation not contact the target directly [20]. to vulnerability scanning. There are four main vulnerability Genge and Enăchescu [21] proposed a vulnerability scanning types: network, host, wireless, and application assessment tool for Internet-connected devices, based on based [3]. Table2 describes each of those types. In this Shodan API. The main idea from their work is identifying paper, we focus on network and host-based scanning; other CPEs based on the grabbed service version number. Na types are out of the scope of this tool as this moment. et al. [17] highlighted a limitation in the work of Genge and Although vulnerability scanning is crucial for numerous Enăchescu [21] which limits the match accuracy and cor- standards and organizations, academic research is limited rectness if the product name is located far from the version in this area and it is dominated by the industry. number. O’Hare [20] proposed a new passive vulnerability Wang and Yang [14] reviewed the major tools avail- assessment tool called Scout that utilizes Censys and the able for vulnerability scanning. In Table3, we summarize National Vulnerability Database (NVD)’s feeds. Scout also important details for each of them. Although there exist tries to identify the host’s vulnerabilities passively. numerous tools for vulnerability scanning, they all tend to share similar scanning techniques. However, different 2.1.1 Critique scanners produce different scanning results, so one could We believe the aforementioned studies suffer from numer- indicate the system is vulnerable, while another indicates ous drawbacks. Firstly, almost all NAT devices are not cov- that it is not. This is frustrating for users and indicates ered by these studies and projects (unless a port forwarding the current scanners indecisiveness. For example, El et al. feature is enabled). Secondly, many firewalls drop scanning [15] benchmarked numerous vulnerability scanners where packets; this will reduce the number of scanned hosts. results showed conspicuous differences: some scanners dis- Thirdly, the scanning result is not conclusive, and, as it is covered two vulnerabilities while others discovered none. based on a trusted third party (i.e., Censys or Shodan), the Additionally, the authors highlighted the importance of user cannot guarantee their accuracy or freshness. Fourthly, enhancing vulnerability scanning performance. this cannot be used for standards compliance; for example, the PCI-DSS standard requires conducting vulnerabilities assessment quarterly or after a major infrastructure up- 2.1 Internet-wide and large-scale scanning grade; assuming that all devices within the audit scope Na et al. [16] proposed a technique for service identifica- expose their network interfaces to public IP addresses, tion based on CPEs through analyzing the service banner there is still a big challenge for data freshness provided on open ports. We believe this work could reduce the by Censys or Shodan, which might be conducted once a service scanning time by targeting specific services based year, due to the large number of IPv4 addresses (i.e., ~232 - on CPEs’ enumeration. However, accuracy could be worse reserved addresses). Fifthly, assuming we are targeting IPv4, than traditional scanning techniques because the authors’ scanning the Internet means covering 4,294,967,296 total claim is to provide security for "Internet-connected devices" addresses - 588,514,304 reserved address = 3,706,452,992 neglecting the fact that many Internet devices hide behind a devices, assuming each device requires 5 minutes (which NAT, which renders port scanning and service enumeration is less than actual time, based on some studies [22] and our useless in most cases. Additionally, their work addresses experimental results). Then, it would require the scanner the service discovery phase only (from phases: (i) host ~35,748.96 years to complete the scans. Increasing the level discovery, (ii) port discovery, (iii) service discovery, and (iv) of parallelism will decrease the time required to accomplish service vulnerability scan). This means that host and port scanning. However, based on our experimental results, we discovery phases are not enhanced. A year later, Na et al. found that current scanners require ~150,000 packets to [17] analyzed their proposed mapping technique and found scan a single host (for TCP scans only). Hence, increasing that it suffers from limitations based on the CPE to vendor or the level of parallelism should be associated with a rapid product name matching. We believe their work is still in its increase in network traffic. Even if we assume an unlimited rudimentary stages because one of their primary research network bandwidth, each server has a limited resources, and goals has not been addressed, which is mapping CPEs to increasing servers would hugely increase costs. Lastly, and CVEs. The study only analyzes creating CPEs, while being most importantly, not all services provide banners. There are oblivious to the primary goal, i.e., CVEs matching. To the no studies or statistics available in the literature regarding best of our knowledge, this paper provides the first practical the efficacy of such solutions, but we believe accurate scan- solution to map CPEs to CVEs, based on our proposed ning for Internet devices is far from being realized using convention-based matching. Our solution is not theoretical current technologies. CVS makes it possible. only but has been built in practice and compared to major industrial scanners. Some projects provide Internet-wide scanning, such as Shodan [18] and Censys [19]. These projects perform 5 TABLE 3: Summary of popular vulnerability assessment tools

Tool Name Strengths Weaknesses Nessus Simple and easy to use Licensing fees • • Compliant with numerous standards Closed-source • • Scanning takes a long time •

OpenVAS Compliant with numerous standards Scanning takes a long time • • Open-source and free Harder to use than alternatives • •

Nexpose Simple and easy to use Licensing fees • • Compliant with numerous standards Closed-source • • Scanning takes a long time •

Retina CS Simple and easy to use Licensing fees • • Compliant with numerous standards Closed-source • • Scanning takes a long time •

3 Threat Model TABLE 4: PVC properties NIST’s definition for vulnerability scanning (in Definition2) Property Req./Opt. Description is broad, and, based on our literature review, we believe it is hard to satisfy its requirements. Hence, we redefine Name Required Represents the PVC’s name. Examples: vulnerability scanning as shown in Definition3. Vulnera- Windows, Acrobat, WinRAR. bility scanners provide a report of vulnerabilities usually Version Optional Represents the PVC’s version number. based on the CVE, the Common Vulnerability Scoring Sys- Does not have to follow any stan- tem (CVSS), CPE, and the Common Weakness Enumeration dardized naming convention. However, (CWE) standards. Simply put, the CVE is a list of publicly many vendors follow a version number- known security vulnerabilities [23]. Each CVE contains an ing such as {major.minor.build.revision} identification number, a description, and references. CVSS is (e.g., 10.3.1.2344). an open industry standard to characterize and score vulner- Vendor Optional Represents the PVC’s vendor. Examples: abilities [24]. CPE is a standardized method for identifying Microsoft, Adobe, RARLabs. applications, operating systems, and hardware [25]. CWE is Edition Optional Examples: Windows 10 Pro, Windows 7 a community-developed list of software security weaknesses Enterprise. [26]. Update Optional Usually used for Windows operating systems. Examples: Service Pack 3, as in Definition 3 (Vulnerability Scanning (proposed definition)). Windows XP Service Pack 3. Represents Examining possible vulnerable components (PVCs) (see the PVC’s update level for the applica- Definition4) in order to identify vulnerabilities affecting tion, it also might indicate a patch level them, based on CVE, CVSS, CPE, and CWE standards. for the PVC. Language Optional Localization. Examples: EN-US. Definition 4 (Possible Vulnerable Component (PVC)). A PVC is any software or hardware that could suffer from a vulnerability. Each PVC has a required name property and optional version, vendor, edition, update, and lan- We assume the vulnerability scanning client (VSC) (i.e., guage properties (as described in Table4). CVS-client) is trusted by targets and safe. This is a reason- able assumption because the VSC is a tiny software with We consider a set of targets 1, 2, , t, , T , a single functionality, and it is open source where targets T { ··· ··· } where T (i.e., the cardinality of set ). We can verify that it is not collecting or reporting any personal |T | T assume there exist an exhaustive set of PVCs, identifiable information (PII). For more details about PII P 1, 2, , p, , P , P Z 0. contains all PVCs in NVDs refer to [27]. We assume that the vulnerability scanning { ··· ··· } ∈ > P database and CPE’s dictionary (publicly available). Each server (VSS) is honest. However, the VSC can follow honest, target ( t ) has a set of PVCs ( t , where t 0). honest-but-curious, or malicious adversarial models. In this ∈ T P ⊆ P |P | ≥ We consider∀ a vulnerability scanning software (VSS) that context, "honest" refers to respecting and not deviating from has two main functionalities: (i) communicate securely with the protocol, either passively or actively. An honest-but- vulnerability scanning clients (VSCs) and (ii) perform the curious adversary is a passive adversary that will follow vulnerability scanning job for each VSC, where each t the protocol but will try to learn the activities of other ∈ T has a VSC. The difference between CVS and network scan- participants. A malicious adversary is an active adversary ners is that the latter do not use VSCs. The VSC is a software that does not follow the protocol and tries to launch different that has one primary function: request vulnerability scan types of attacks, such as man-in-the-middle (MITM), denial- service from VSS while providing its set of PVCs ( t ). of-service (DoS), and distributed DoS (DDoS), among other P 6

16,000 Number of CVEs Without CPEs 14,000 2,000 Without CVSS 12,000 10,000 1,500 8,000

Count Count 1,000 6,000 4,000 500 2,000 0 0 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 Year Year

Fig. 2: NVD’s database statistics for vulnerability count (as Fig. 3: NVD’s database statistics for CVEs without CPE or of Aug. 11, 2019) CVSS entries (as of Aug. 11, 2019) attacks. We also assume a resourceful adversary called Eve Goal 5: The VSS and VSC, as well as the commu- • that can follow an honest-but-curious or malicious adversar- nication between them, should be protected against ial models. Eve has full access of the intermediary network honest, honest-but-curious, and malicious adversaries between the VSCs and the VSS; she could be a system (including Eve). administrator or even an Internet service provider (ISP); in essence, she embodies the Dolev-Yao adversarial model [28]. 4 Implementation Overview We assume there exist an exhaustive set of vulnerabilities, 1, 2, , v, , V , where V Z 0. Figure6 shows CVS’s primary components. First, common V { ··· ··· } ∈ > There exists an exhaustive set of CVSSs, libraries contain communication APIs and helper functions CVSS 1, 2, , cvss, , CVSS , where CVSS Z 0. There such as encryption/decryption, hashing, compression, log- { ··· ··· } ∈ ≥ also exists an exhaustive set of CPEs, ging, services discovery modules, and serialization. Second, CPE 1, 2, , cpe, , CPE , where CPE Z>0. Each vulner- the NVD contains XML schemas5 and the classes we created { ··· ··· } ∈ v ability has an optional CVSS set ( v : , to match those schemes; those classes are used as data v ∀ v ∈ V ∃ CVSS where Z 0 and ). Each transfer objects (DTOs). Third, the CPE’s library contains |CVSS | ∈ ≥ CVSS ⊆ CVSS v vulnerability has an optional set of CPEs ( v , the CPE XML schema and corresponding DTO classes. v v ∀ ∈ V ∃ CPE where Z 0 and ). We modeled Fourth, the client application is a small software used by |CPE | ∈ ≥ CPE ⊆ CPE CVSS and CPE sets as optional because our analysis showed targets to request scans. We used .NET Framework 4.0 for that NVD’s database contains CVEs without CVSS or CPE the client application to make it support old versions of entries as shown in Figure2 and Figure3 . Definition5 Windows OS. Currently, CVS has Windows clients only. We shows our formal definition for vulnerability scanning, consider other operating systems for future work. Finally, where t is the set of vulnerabilities that affect the target the server module is the main part of CVS and will be CVE t. Scan accuracy is shown in Equation (1). discussed in detail herein. We used Entity Framework 6.0 Definition 5 (Vulnerability Scanning Formal Definition as our object relationship mapper (ORM) and the Microsoft (proposed definition)). SQL server as our database management system (DBMS). t t t t We used C#, SQL, HTML, JavaScript (& jQuery), CSS, and Given ∗ , t : find ∗∗ P ⊆ P ∀ ∈ T CVE ⊆ CVE XML languages to build CVS server. At the time of writing, t t the current CVS version is 2.0.0.0. Figure5 shows a general ∗∗ Scan Accuracy |CVE ∩ CVE | 100% (1) overview of the scanning process in CVS. t ∗ |CVE | Figure4 shows the PVC’s categories. Each PVC can be 4.1 CVS Server Initialization and Update either vulnerable or non-vulnerable. Each vulnerable PVC CVS is very simple and easy to use. The server application can be either exploitable (if there is a known exploit for is only ~3 MB in size. We do not provide any pre-installed it) or non-exploitable. A non-exploitable vulnerability does vulnerability sets. However, CVS should be initialized at not necessarily mean that it is secure because it might suffer first run, which is a common behavior among all surveyed from zero-day vulnerabilities that are not published publicly vulnerability scanners in the literature. Figure7 shows the yet. Zero-day vulnerabilities are out of our scope. server initialization process. The user starts the server by In this context, our goals are the following: double clicking on the application executable file. Then, to Goal 1: Provide a vulnerability scanning service ( t • ∈ initialize the server, the user should go to the configurations ), based on Definition5. ∀ T tab and click on "update". This will automatically download Goal 2: Maximize the vulnerability scanning accuracy • the CPEs dictionary, the NVD database, and the exploit- based on Equation (1). db sources. Each of the aforementioned will be extracted, Goal 3: Minimize each target vulnerability scanning (i) • parsed into CVS DTOs, filtered, and persisted into CVS required time, and (ii) network traffic. Goal 4: Enable the VSS to learn from previous scanning • 5. On Oct 16, 2019, NVD stopped providing XML feeds and moved jobs (from the same target or different targets). to JSON 7

Possible Vulnerable Component (PVC)

Vulnerable (Vulnerability) Non-Vulnerable

Zero-day Exploitable Non-Exploitable Safe vulnerability Exploites available No vulnera- No known exploits A Vulnerability that for this vulnerability bilities at all. are available is not discovered yet.

Fig. 4: Possible Vulnerable Components (PVCs) categories

TABLE 5: Installation requirements comparison VSS (server) VSC (client)

Server Started Listeting Scanner Requirements Size Install Initialize Time Time Submit Scan Job Nessus CPU: Quad-core, RAM: 4 GB 70MB 50 S 10m:09s

Verify Request Nexpose CPU: Quad-core, RAM: 8 GB 736+MB 1m:30s 15m:45s CVS CPU: Quad-core, RAM: 8GB 3MB 20S 9m:15s Job ID (if valid request) Server

Add Job To Queue

Scan the Job 4.2 CVS Server Scanning Process Request Result (Job ID) Figure8 shows CVS server scanning process. It consists of five main phases: request handling, verification, queuing, Scan Result scanning core service, and CPEs generation. Phase 1 - Request Handling: This phase implements • the network interface and request/response operations. CVS server receives requests in a non-blocking fashion. Fig. 5: Sketch of CVS scanning process Our initial tests show that the server can handle up to 140 concurrent requests per second (as per machine specs in Table6). The request will be forwarded to the verification phase, which acts as a software firewall. CPE Schemas and DTOs Common Libraries CVS Client (VSC) Phase 2 - Verification: This phase acts as a rule-based • software firewall. It checks the source IP/subnet, device ID, and encryption keys. It is fail-safe where the connection will be dropped if it does not comply with CVS Server (VSS) CVE Schemas and DTOs any of the rules. Phase 3 - Queuing: The server is built with the expecta- • tion to server a very large number of hosts concurrently. Fig. 6: CVS main components Hence, it will not be prudent to keep the requests alive until the scan job is finished. Thus, if the verification phase decided to accept the scan task, it will be added database using CVS schema. This process requires about 10 to a FIFO queue. The connection with the client will be minutes (depending on Internet connection speed and work- closed and the client will be provided with a token to station specs). Table5 shows the installation requirements request the results later. The queue will serve a group comparison between Nessus, Nexpose, and CVS. We believe of jobs concurrently, based on the hardware resources, CVS has the simplest installation and update process among each of which will be forwarded to the scanner phase. surveyed scanners. Finally, the user can specify on which This implementation makes the server act somehow as port to start the scanning server listening for incoming a state-less server, which will help to protect against requests from VSCs i.e., CVS-clients. flooding attacks. 8

Parse XMLs to Data CPEs DB Install/Run CVS Transfer Objects Extract Sources Scanning Server (DTOs)

Start NVD DB

Done updating Exploit-DB CVS DB Initialization (First Yes Run) or Update Prepare Data for Persist to DB (Other Runs) CVS Persistance CVEs Database

New Resources?

Start Scanning No Service

Stop Scanner

Fig. 7: CVS server initialization/update process (using BPMN notation)

TABLE 6: Scanning machines speciﬁcations 5 CPEs Generation CVS supports CPEs generation for operating systems, ap- Item Description plications, and hardware PVCs. Section 5.1 addresses gen- For Nessus and CVS erating CPEs for operating systems. Section 5.2 discusses generating CPEs for applications. Section 5.3 discusses gen- CPU 6 cores Intel I7-8700K. erating CPEs for hardware. The aforementioned types share RAM 8 GB. the common properties of a PVC (shown in Table4). Hence, OS Windows 10 Professional. we made them utilize a common CPEs generation process, Disk SSD. as shown in Algorithm2. Network 1Gbps Ethernet. 5.1 CPEs Generation for Operating Systems For Nexpose (because Windows 10 was not supported) Algorithm3 shows the CPEs generation process for operat- CPU 4 cores Intel I7-4700HQ. ing systems. The platform CPE component is always set to RAM 12 GB. ’o’. Other components are generated automatically, based on OS Ubuntu 18.04. our proposed generation conventions. The CPE conventions Disk SSD. of the operating systems are shown in Tables 12, 10, 13,7, Network 1Gbps Ethernet. 8, and9 to generate vendors, products, versions, updates, editions, and languages (respectively).

Phase 4 - Scanner: The scanner receives jobs from the 5.2 CPEs Generation for Applications • queue to be scanned. For each PVC in the scan job, it Algorithm4 shows the CPEs generation process for appli- will launch a thread (with a concurrency cap). Each cations. The platform CPE component is always set to ’a’. thread will generate the possible CPEs for that PVC Other components are generated automatically, based on (as in Phase 5). Afterwards, the possible CPEs will be our proposed generation conventions. The application CPE matched with CVS database. Then, the database will be conventions are shown in Tables 14, 19, 15, 16, 17, and 18 to updated in order to reduce scanning time, in case the generate vendors, products, versions, updates, editions, and same PVC showed up in future scans. languages (respectively). Phase 5 - CPEs Generator: The primary function of this • phase is to map PVCs to possible CPEs. The VSS uses 5.3 CPEs Generation for Hardware CPEs to discover vulnerabilities targeting that PVC. This phase is discussed in detail in Section5. For hardware PVCs, we use the same process and conventions used for applications (discussed in Section 5.2). The only change is that in Algorithm4, we change the platform from ’a’ to ’h’. All previously discussed components inherit the base PVC’s properties (see Table4). We believe that the hardware PVCs will be the most challenging part for 9

No Request Rejected and Reject Reason

Request Security Request Yes Create Scan Job Queue Job Verification Accepted? Scan Scan Scan Request Job done, ticket Requested provided to the user

Verify Rejected Accepted Request Add Scan Job To Queue Verification Scan Jobs Rules Queue No Yes Check Jobs Queue Has Jobs? Yes

Checked All

Verification Check Rule Verified? Yes rules? De-Queue a Scan Request Security Security Request No Job (concurrency Job Added Sleep X ms limit) No to Queue Scanning Jobs Queue Jobs CVS Scanning

Enhance Done scanning Update Accuracy Based CVS this job Database On Conventions Finished Scan Reduction No Job Conventions currentPVC = one Generate Possible of the scan job CVS Match Generated CPEs Based On PVCs. Yes PVCs >0? CVS Scanner Database CPEs with CVEs Conventions Scan job PVCs -1 Scan Job

Generate CPEs Is PVC an OS? App or Hardware

Selector Generate PVC s CPEs

No No Yes CPE Possible CPE Possible Checked CPE Version = Checked Product Names = CPE All? Yes Vendor=Compute Platform = o Generate Based All? Generate Based Based on OS Type on Convention on Convention Yes Generate OS CPEs Yes

Similar process of OS Version OS Product product and version Conventions Conventions will be applied to Get OS Vendor Based On OS Type OS CPEs Generator CPEs OS edition, and Finished Generating language Possible OS CPEs CPEs Generator CPEs No No No Generate CPEs Checked CPE Possible All? Version = CPE Possible CPE Possible Generate Based Yes Checked Product Names = Checked Vendor Names = on Convention All? Generate Based All? Generate Based on Convention on Convention CPE Platform = Yes a for app, h App Version for hardware Conventions App Product Conventions App Vendor Similar process of product and Finished Generating Conventions version will be applied to edition, and language Possible App CPEs Application CPEs Generator CPEs Application

Fig. 8: CVS server scanning process (using BPMN notation) 10

Algorithm 2: CPEs generation process for PVCs Algorithm 3: CPEs generation process for OSs

1 PVCCPEsGeneration ( , , , , , , ) 1 OSCPEsGeneration ( ) inputs : : Set of possibleP V platforms.PR VR E U L inputs : PVC: PVCO properties . P Z 1, 3, p : p < null, empt y . inputs : : OS properties. Inherits PVC inputs : |P|: Set ∈ of≥ possible≤ ∀ ∈P vendor[ names. ] O V properties (in Table4). Z 1, v : v < null, empt y . |V| ∈ ≥ ∀ ∈ V [ ] output: : list of possible CPEs. inputs : : Set of possible product names. C PR Z 2 ; 1, pr : pr < null, empt y . C ← ∅ inputs : |PR|: Set ∈ of≥ possible∀ ∈ PR versions.[ ] /* For OS platform always assumed to be ’o’ */ VR |VR| ∈ Z 1, vr : vr < null, empt y . 3 GenerateBasedOnConventions(’o’) ; ≥ ∀ ∈ VR [ ] inputs : : Set of possible editions. Z 0. /*P ←OS possible vendors generated according to E |E| ∈ ≥ inputs : : Set of possible updates. Z 0. conventions in Table 12. */ U |U | ∈ Z≥ inputs : : Set of possible languages. 0. GenerateBasedOnConventions output: L: Set of possible CPEs. |L| ∈ ≥ 4 (Table 12); C /*V ←OS possible product generated according to 2 ; 3 foreachC ← ∅ p do conventions in Table 10. */ ( ∈ P) 4 foreach v do 5 GenerateBasedOnConventions(Table 10); ( ∈ V) 5 foreach pr do /*PR OS ← possible versions generated according to ( ∈ PR) 6 foreach vr do conventions in Table 13. */ 7 if ( ∈ VR)then (U ∅) 6 GenerateBasedOnConventions(Table 13); 8 .Add("cpe : p : v : pr : vr"); /*VR OS ← possible updates generated according to 9 ifC then/ (E ∅) 10 .Add("cpe : p : v : pr : vr : conventions in Table7. */ Cu"); / 7 GenerateBasedOnConventions(Table7); U ← 11 if then /* OS possible editions generated according to (L ∅) 12 .Add("cpe : p : v : pr : conventions in Table8. */ Cvr : u : e"); / 8 GenerateBasedOnConventions(Table8); .Add("cpe : p : v : /*E ←OS possible languages generated according to Cpr : vr : u : e :/"); 13 else conventions in Table9. */ 14 foreach l do 9 GenerateBasedOnConventions(Table9); ( ∈ L) L ← 15 .Add("cpe : p : v : /* CPEs are generated based on generalized C / pr : vr : u : e : l"); algorithm in Algorithm2. */ 10 PVCCPEsGeneration( , , , , , , 16 else C ← P V PR VR E U 17 foreach e do ); L 18 foreach( ∈ E)l do 11 return ; ( ∈ L) 19 .Add("cpe : p : v : C Cpr : vr : u : e/: l"); TABLE 7: OS update generation conventions 20 else 21 foreach u do # Convention Description Example 22 foreach( ∈e U) do 23 foreach( ∈ E)l do ( ∈ L) 1 Empty As the update is an We assume the up- 24 .Add("cpe : p : v : C / optional ﬁeld date is empty because pr : vr : u : e : l"); it is optional. 2 Abbreviations Abbreviations of ser- If the provided service pack property vice pack is ’service 25 return ; pack 3’ then the pos- C sible edition is ’sp3’.

CVS deployment, because, according to our analysis, there 6 Results and Discussion are ~5,000 distinct vendors. It would be an intricate task to provide one client to support all hardware products because 6.1 Methodology they use diﬀerent libraries. On the other hand, this issue We compared CVS with two commercial vulnerability scan- is easily resolved for operating systems and applications ners: Nessus and Nexpose. We used the free trial provided because they rely on a few vendors that build their software, by the vendors. Both scanners were conﬁgured for exhaus- according to general architectures (e.g., one client is enough tive scanning policy for TCP ports only. We excluded UDP for all Windows platforms and all applications running in Windows). One way to resolve this issue for hardware TABLE 8: OS edition generation conventions is to provide an API (e.g., RESTFUL), wherein hardware products use the API to request scans. # Convention Description Example

1 Empty As the edition is an op- We assume the edition tional ﬁeld is empty. 11

TABLE 9: OS language generation conventions time (more than an hour per host). Table6 shows the host machines specifications. We used two machines: Windows # Convention Description Example 10 and Linux, because Nexpose did not work on the former. Table 20 shows the tested machines’ specifications. 1 Empty As the language is an We assume the lan- optional field guage is empty. 6.1.1 Accuracy Analysis TABLE 10: OS product name generation conventions Recently, benchmarks have been introduced for web applications vulnerability scanners [29, 30, 31]. However, there are # Convention Description Example no benchmarks for network vulnerability scanning. Hence, 1 Combinations Generate combinations of If the provided OS to test the selected machines (in Table 20), we used a third- the provided OS name name was ’windows party statistics provider to get the actual number of vulner- value sub-strings based xp’, then possible abilities in each operating system. We reduced the results on empty spaces com- combinations will be bined with separators in [’windowsxp’, ’windows- comparison problem for operating system vulnerabilities Table 11. xp’, ’windows_xp’]. only, neglecting any installed applications, to make it easier to compare. Even without reducing the discovered number of vulnerabilities by Nessus and Nexpose, they were very ports from scanning because this largely increases the scan few, ranging from 4 to 6 vulnerabilities. However, the actual vulnerabilities based on [32] were 180, 199, and 391 for Windows server 20086, Windows 77, and Windows XP 8 (re- TABLE 11: Possible separator chars used for combinations spectively). Additionally, we assumed that all vulnerabilities reported by Nessus and Nexpose were correct, to show the Separator char Description upper limit for their accuracy. We calculated their accuracy "_" Underscore char by dividing the number of their discovered vulnerabilities "-" Dash char by the actual vulnerabilities. For CVS, however, we first "" none matched each of the discovered vulnerabilities with the actual vulnerabilities set, then accuracy was calculated by Algorithm 4: CPEs generation process for applica- dividing the matches over the actual number of vulnerabili- tions ties, as in Equation (1), to show the actual accuracy of CVS. 1 AppCPEsGeneration ( ) inputs : PVC: PVC propertiesA . inputs : : App properties. Inherits PVC 6.2 Results A properties (in Table4). 6.2.1 Performance Results output: : list of possible CPEs. C 2 ; /* App’s OS platform always assumed to be Table 21 shows our performance analysis results and com- C ← ∅ ’a’ */ parison between Nessus, Nexpose, and CVS. We focused on 3 GenerateBasedOnConventions(’a’) ; two performance measurements: scanning time and com- P ← /* App’s possible vendors generated according to municated network traffic. CVS had the lowest scanning conventions in Table 14. */ time and communicated network traffic for all experiments. 4 GenerateBasedOnConventions(Table 14); Compared to Nessus, CVS reduced scanning time and net- V ← /* App’s possible product generated according to work traffic by an average of 93.59% and 99.97% (respec- conventions in Table 19. */ tively). Compared to Nexpose, CVS reduced scanning time 5 GenerateBasedOnConventions(Table 19); and network traffic by an average of 64.92% and 99.97% PR ← /* App’s possible versions generated according to (respectively). conventions in Table 15. */ Additionally, we scanned each host two times using the 6 GenerateBasedOnConventions(Table 15); same scanner to check whether the scanner learns from pre- VR ← /* App’s possible updates generated according to vious scans or not. There was no improvement for Nessus conventions in Table 16. */ and Nexpose. However, CVS improved scanning time, with 7 GenerateBasedOnConventions(Table 16); an average of ~97% when the same host was scanned for the U ← /* App’s possible editions generated according to second time, because CVS learns from previous experiences conventions in Table 17. */ and the scanning engine can identify previous PVCs that 8 GenerateBasedOnConventions(Table 17); are impossible to change unless the database is updated. E ← /* App’s possible languages generated according For each database update, however, PVCs scanning results to conventions in Table 18. */ records will be adapted to the new updated vulnerabilities. 9 GenerateBasedOnConventions(Table 18); /*L ←CPEs are generated based on generalized 6. https://www.cvedetails.com/vulnerability-list/vendor_id- algorithm in Algorithm2. */ 26/product_id-11366/version_id-82464/Microsoft-Windows-Server- 10 PVCCPEsGeneration( , , , , , , 2008--.html C ← P V PR VR E U ); 7. https://www.cvedetails.com/vulnerability-list/vendor_id-26/ L product_id-17153/version_id-138704/Microsoft-Windows-7--.html 11 return ; C 8. https://www.cvedetails.com/vulnerability-list/vendor_id- 26/product_id-739/version_id-46866/Microsoft-Windows-Xp-.html 12 TABLE 12: OS vendor name generation conventions