Poster: Exploitation of Vulnerabilities - If, When and How Often

Poster: Exploitation of vulnerabilities - if, when and how often

Kartik Nayak∗, Daniel Marino†, Petros Efstathopoulos†, Tudor Dumitras‡, [email protected], Daniel [email protected], Petros [email protected], [email protected] ∗Student, University Of Maryland, College Park †Symantec Research Labs ‡Assistant Professor, University Of Maryland, College Park

I.INTRODUCTION best of our knowledge, the size and variability of attack surfaces has not been evaluated empirically in the field. It is, In order to improve the security of our software systems, therefore, difficult to determine the effectiveness of this metric we need to be able to measure how they are impacted by in capturing real-world conditions. the various defensive techniques introduced to the system. Measuring security, however, is challenging. Many security metrics have been proposed, including the total count of These examples illustrate that our ability to assess the vulnerabilities in source code, the severity of these vulner- security of systems that are deployed and actively utilized abilities, the size of the attack surface and the time window is currently limited by the metrics being used. In particular, between the vulnerability disclosure and the release of a patch. the developers and the users may employ different security System administrators and security analysts often rely on these metrics. For example, one way of estimating the vulnerability metrics to assess risk and to prioritize some patches over density and the attack surface is to use existing tools that others, while developers use them as guidelines for improving measure these properties by directly analyzing the code and software security. Practical experience, however, suggests that the configuration of the system in question [5], [6]. However, the existing security metrics exhibit a low level of correlation these measurements are conducted in lab conditions, and do with vulnerabilities and attacks, and they do not provide an not reflect the real-world security of systems that are deployed adequate assessment of security [1], [2]. and actively used in the field. The total number of vulnerabilities discovered in source code is commonly used as a measure of the system’s secu- For these reasons, users are ultimately interested in metrics rity [1], [2]. However, this metric does not account for the fact that help them assess the effectiveness of these techniques in that cyber attackers never make use of some of the discovered the field. Figure 1 illustrates this problem. The total number vulnerabilities, which may be hard to successfully exploit in of vulnerabilities discovered in the code-base of Windows the presence of security technologies such as data execution OSes does not follow a clear trend with respect to the OS prevention (DEP) and address space layout randomization release date. Basing the evaluation of system security on this (ASLR). For example, CVE-2007-1748 was exploited in the metric would lead to the conclusion that security has not wild, but there is no similar evidence for CVE-2007-1749. improved considerably over the past decade. However, the number of vulnerability exploits exhibits a decreasing trend Another popular metric is based on the observation that over time,which suggests the existence of deployment-specific attacks can succeed only if the vulnerable software accepts factors, yet to be characterized systematically, that influence input from potential attackers. For this reason, system admin- the security of systems in active use. istrators have long advocated turning off unnecessary system services to avoid exposure to exploits of unpatched or unknown vulnerabilities. For example, network-based attacks exploiting Our goal in this work is to propose new metrics that better CVE-2007-1748 are unsuccessful—even if the vulnerability reflect security in the real world and to employ these metrics was not yet patched—if the DNS server is not running. for evaluating the security of popular software. Rather than This idea is formalized in the concept of attack surface [3], measuring security in lab conditions, we derive metrics from [4], which proposes quantifying the amount and severity of field-gathered data and we study the trends for vulnerabilities potential attack vectors that a system exposes using a formula and attack surfaces exercised in attacks observed in the real that takes into account the open sockets and RPC endpoints, world. These metrics try to capture the notion of whether the running services and the privilege at which they are disclosed vulnerabilities are exploited and how often they are running, the active Web handlers, the accounts enabled, etc. exploited. While the vulnerability count and the attack surface Reducing the attack surface, however, does not always improve are metrics that capture the opportunities available to attack- security; for example, including security mechanisms in the OS ers, we instead focus on attempted, though not necessarily may increase the attack surface, but render the system more successful, attacks in the field. This new understanding will secure. Furthermore, the attack surface of software products allow developers, system administrators and policy makers to changes after they are deployed in the field, as users install improve security by focusing on the attacks and vulnerabilities new applications and modify system configuration. To the that matter most in practice. XP SP2 64-bit Windows 2000 SP4 60 XP SP2 64-bit 200 Server 2003

50 XP SP2 XP SP3 Windows 2000 Windows 2000 XP SP1 64-bit

Windows 2000 SP4 Windows 2000 SP3 150 XP Server 2003 SP2 64-bit XP SP1 Server 2003 Server 2003 SP2 40 XP XP SP2 Vista SP1, Vista SP1 64-bit Server 2008 SP1, Server 2008 SP1 64-bit Windows 2000 SP3 Vista SP2, Vista SP2 64-bit Windows 7, Windows 7 64-bit Server 2003 SP1 XP SP1 64-bit 100 XP SP1 Server 2008 R2 64-bit 30 Server 2003 SP1 64-bit Server 2008 SP2 XP SP3 Server 2008 SP2 64-bit Server 2008 R2 SP1 64-bit Server 2003 SP2 64-bit Server 2003 SP1,Server 2003 SP1 64-bit Server 2003 SP2 Vulnerabilities exploited Vulnerabilities Vulnerabilities disclosed Vulnerabilities Windows 7 SP1 20 Vista SP1, Vista SP1 64-bit Windows 7 SP1 64-bit Server 2008 SP1, Server 2008 SP1 64-bit 50 Vista, Vista 64-bit Windows 2000 SP2 Vista, Vista 64-bit Vista SP2, Vista SP2 64-bit Windows7, Windows 7 64-bit 10 Server 2008 SP2 Server 2008 SP2 64-bit Windows 2000 SP1 Windows 2000 SP2 Server 2008 R2 SP1 64-bit Server 2003 R2 Server 2008 R2 64-bit 0 Server 2003 R2 64-bit Windows 7 SP1, Windows 7 SP1 64-bit

2000 2005 2010 2015 2000 2005 2010 2015 Release date Release date

(a) All vulnerabilities disclosed publicly. (b) Vulnerabilities exploited in the wild. Fig. 1. Number of vulnerabilities disclosed and exploited for Microsoft Windows over 11 years of releases, estimated using the National Vulnerability Database and WINE. No clear trend exists for total number of vulnerabilities disclosed but number of vulnerabilities actually exploited in the ﬁeld decreases with newer OSes.

II.PROPOSEDMETRICS is used to compute the exploitation metrics. Using the binary reputation dataset, we find the products installed on the hosts. We propose several metrics, derived from field-gathered The IPS telemetry dataset identifies an attack on a host. data, that capture the state of system security as experienced Knowing the products installed on the host as well as the by the users. signatures corresponding to vulnerabilities, relevant attacks on The following metrics capture the notion of whether dis- the hosts can be detected. This information can be used to closed vulnerabilities get exploited. compute the attack volume and attack surface metrics.

1) Count of vulnerabilities exploited in the wild. IV. CONCLUSION 2) Exploitation ratio. The exploitation ratio is the pro- portion of disclosed vulnerabilities for product p that Our ability to improve system security rests on our under- have been exploited up until time t. It captures the standing of how to measure and assess security under real- likelihood that a vulnerability will be exploited. world conditions. We believe that our metrics would better reflect security in the real world and help evaluate the security We also propose the following metrics that capture how often of softwares. This can help the design of future security vulnerabilities are exploited on hosts in the wild. technologies. 1) Attack Volume. The attack volume is a measure that REFERENCES captures how frequently a product p is attacked. 2) Exercised Attack Surface. The exercised attack sur- [1] Y. Shin, A. Meneely, L. Williams, and J. A. Osborne, “Evaluating complexity, code churn, and developer activity metrics as indicators face captures the portion of the theoretical attack of software vulnerabilities,” IEEE Trans. Software Eng., vol. 37, no. 6, surface of a host that is targeted in a particular month. pp. 772–787, 2011. [2] T. Zimmermann, N. Nagappan, and L. A. Williams, “Searching for a III.EXPERIMENTAL METHODS needle in a haystack: Predicting security vulnerabilities for windows vista,” in ICST, 2010, pp. 421–428. We use the National Vulnerability Database (NVD) [7], [3] M. Howard, J. Pincus, and J. M. Wing, “Measuring relative attack Symantec attack signatures [8], [9] and Symantec’s Worldwide surfaces,” in Workshop on Advanced Developments in Software and Intelligence Network Environment (WINE) [10] for our study. Systems Security, Taipei, Taiwan, Dec 2003. The NVD is a widely accepted database for research related to [4] P. K. Manadhata and J. M. Wing, “An attack surface metric,” IEEE software vulnerabilities. Symantec security products include an Trans. Software Eng., vol. 37, no. 3, pp. 371–386, 2011. extensive database of attack signatures obtained from Syman- [5] Microsoft Corp., “Microsoft Attack Surface Analyzer - Beta,” http://bit.ly/A04NNO. tec’s intrusion-protection systems as well as descriptions of [6] Coverity, “Coverity scan: 2011 open source integrity report,” 2011. anti-virus signatures that are used to scan files for known threats. WINE contains records of vulnerabilities exploited [7] National Vulnerability Database, http://nvd.nist.gov/. in the field (IPS telemetry dataset) as well as information of [8] “Symantec Attack Signatures,” http://bit.ly/1hCw1TL. binaries (binary reputation dataset) on end-user hosts running [9] Symantec Corporation, “A-Z listing of threats and risks,” http://bit.ly/11G7JE5. Symantec products. We combine these three databases to [10] T. Dumitras and D. Shou, “Toward a standard benchmark for identify attacks on end hosts running the following products computer security research: The worldwide intelligence network - Windows Operating systems, Microsoft Office, Internet Ex- environment (wine),” in Proceedings of the First Workshop on Building plorer and Adobe Reader. Analysis Datasets and Gathering Experience Returns for Security, ser. BADGERS ’11. New York, NY, USA: ACM, 2011, pp. 89–96. Using NVD and Symantec signatures, we identify the [Online]. Available: http://doi.acm.org/10.1145/1978672.1978683 vulnerabilities in the products that are exploited as well as the signatures used to exploit the vulnerabilities. This information