Critical Vulnerability in Browser Security Metrics

Mustafa Acer Collin Jackson Carnegie Mellon Silicon Valley Carnegie Mellon Silicon Valley [email protected] [email protected]

Abstract 1 Introduction

We are constantly barraged by news about the latest embarrassing browser exploit, and the problem seems to be growing larger every day. Every time a browser vendor releases a patch for Browser vendors are evaluated by their ability to a critical vulnerability, the popular news media stay out of the news, and security vendors like publishes a slew of negative press article detail- Cenzic, IBM, and Symantec regularly publish re- ing the security holes that have been announced ports [6, 15, 16] that damage the reputation of in the product. Users who read these articles of- browser vendors with the most reported bugs. ten decide to switch to a “safer” browser. The Unfortunately, these reports discourage secure negative press associated with security patch re- browser vendor behavior by punishing proactive leases has a number of unhealthy effects on the patching and ignoring many factors that are im- industry. We challenge the conventional wis- portant for end-user security. dom of the current browser security evaluation We argue that the most prevalent metric, the paradigm: that browsers that receive infrequent number of publicly disclosed vulnerabilities in a security patches are safer than browsers that browser for specific interval of time, does not rep- receive frequent patches, that browsers with a resent a useful measure of vulnerability for any lower bug count are safer, and that reducing browser. We discuss four major weaknesses of browser vulnerabilities is the only path that a this metric and propose a new metric that more browser vendor can follow to improve security. accurately represents the actual risk to users. We argue that patch deployment matters vastly more than patch frequency, that bug count fails 2 Flaws of Current Metrics to take into account differences in severity and vendor reporting methodologies, and that the Consider the example of Cenzic, a leading secu- security features that matter most are ignored rity vendor, who recently released a report [5] de- by negative news articles. We propose methods scribing the breakdown of browser vulnerabilities for evaluating browser security that take into ac- for the first half of 2009. The report compared count new industry best practices such as silent the number of publicly reported vulnerabilities patch deployment and sandboxing. in browsers for that six month period and indi-

1 cated that was the riskiest browser with of browsers by buying ad impressions or com- 44% of vulnerabilities. Their methodology has promising a popular web site [12]. It is critical several major limitations: that browser vendors reduce actual exploitabil- ity, rather than waging a public relations battle • Ignores Patch Deployment. Patches over metrics that have no connection to reality. that are quickly deployed have a minimal effect on user security. Silent update patch 3 Our Proposal deployment technologies can have a signifi- cant effect on user security [10, 9]. We propose that browsers be evaluated on the • Discourages Disclosure. To improve per- percentage of users who have at least one un- ceived security, vendors often combine unre- critical-severity vulnerability (or at least lated bugs into a single disclosure or avoid one unpatched high-severity vulnerability) on an reporting them entirely, even after security average day during the specified interval. Unless patches are widely deployed [13]. sandboxing is used to restrict vulnerabilities in plug-ins, we propose that vulnerabilities in plug- • Ignores Severity. Improved browser se- ins also count for this calculation. This metric curity architectures such as the sandboxed addresses the problems described above: renderer in [4] can reduce • many security bugs from critical severity Takes Account of Patch Deployment. (arbitrary code execution) to high sever- Browsers that use faster update techniques ity (accessing confidential data belonging to will benefit, because users will have vulner- other web sites) [2], but simple bug counting abilities for a shorter period of time. does not account for severity. • Encourages Disclosure. Browser vendors • Ignores Plug-ins. According to Adobe, who disclose vulnerabilities but patch users Flash Player is installed by 99% of web quickly will not be penalized. Combining users [7], and a recent report [17] suggests multiple bugs into a single disclosure will that 80% of Flash Player users have not not improve a vendor’s score. This reflects yet installed the latest critical security up- the reality that an attacker only needs one dates. Firefox includes an update check ser- vulnerability to exploit a user’s browser. vice that help the user keep Flash Player up • Takes Account of Severity. Separate to date, but existing metrics do not measure scores for critical and high-severity vulnera- its effect on end user security. bilities ensure that browsers that use sand- boxing technologies to reduce the severity of None of these problems are browser-specific; vulnerabilities will receive better scores. they affect security evaluations of all soft- ware. However, they are particularly severe • Includes Plug-ins. Browser vendors that for browsers, which constantly interact with un- do an effective job of sandboxing or up- trusted code (HTML, JavaScript, CSS, and so dating a user’s plugins will receive a better on). Attackers can easily run exploits on millions score.

2 4 Measurement

We have begun measuring proposed browser risk metric using techniques we developed in our pre- vious work [12, 3, 1]. Our server collects browser and plug-in version data from the web by run- ning JavaScript advertisements on ad networks. We compare these observations against a vul- nerability database compiled from browser and plug-in vendors [8, 14, 11]. We then assign a risk score to browser and plug-in combinations. We define risk score as the percentage of browsers Figure 1: Average browser risk scores over a that have at least one known critical or high forty-day span. vulnerability. For any given day, this score is calculated based on the vulnerabilities reported by vendors before that day. iment. Our preliminary results are shown Figure 1. We found that the percentage of vulnerable of browsers increases significantly if the plug-ins 5 Conclusion vulnerabilities are included in the calculation. We expect that adoption of more constructive For example, we found that only about 4% of metrics for browser security will increase open- Google Chrome users have critical or high vul- ness and disclosure without harming user secu- nerabilities. However, 30% are vulnerable if rity. By evaluating the security functionality Flash Player vulnerabilities included. Fortu- that matters most to users and web application nately, the latest beta version of Google Chrome developers, we will encourage browser vendors to now includes Flash Player updates in its silent innovate and compete on security features that update process [18]. We expect Google Chrome’s can have the most positive impact. plugin-adjusted risk score to drop significantly once this beta version is rolled out to all users. One limitation of our experimental methodol- References ogy is that we are unable to evaluate the patch level of . Unlike other browsers, [1] Gaurav Aggarwal, Elie Burzstein, Collin Internet Explorer does not advertise which secu- Jackson, and Dan Boneh, An analysis of rity patches have been applied when querying modes in modern browsers, the user agent string. It might be possible to de- To appear in USENIX Security 2010. tect the browser’s patch status by triggering an exploit, but this would potentially expose users [2] The Authors, Sever- to risk or interfere with their browsing session. ity guidelines for security issues, An open research problem is how to evaluate In- http://www.chromium.org/developers/ ternet Explorer’s patch level in an ethical exper- severity-guidelines.

3 [3] Adam Barth, Collin Jackson, and John C. [11] Google Inc, Google Chrome re- Mitchell, Robust defenses for cross-site re- leases, November 2009, http: quest forgery, Proceedings of the 15th ACM //googlechromereleases.blogspot.com. Conference on and Communica- tions Security (CCS), 2008. [12] Collin Jackson, Adam Barth, Andrew Bortz, Weidong Shao, and Dan Boneh, Pro- [4] Adam Barth, Collin Jackson, Charles Reis, tecting browsers from DNS rebinding at- and the Google Chrome Team, The secu- tacks, Proceedings of the 14th ACM Con- rity architecture of the Chromium browser, ference on Computer and Communications September 2008, Technical Report. Security (CCS), 2007. [13] Window Snyder, Critical vul- [5] Cenzic, Web application secu- nerability in metrics, rity trends report, November 2009, November 2007, http://blog. http://www.cenzic.com/downloads/ mozilla.com/security/2007/11/30/ Cenzic_AppSecTrends_Q1-Q2-2009.pdf. critical-vulnerability-in-microsoft-metrics/.

[6] Inc. Cenzic, Web application se- [14] Software, Opera security advisory, curity trends report, 2009, http: September 2009, http://www.opera.com/ //www.cenzic.com/downloads/Cenzic_ support/kb. AppSecTrends_Q1-Q2-2009.pdf. [15] IBM Security Solutions, X-Force 2009 trend and risk report, http: [7] Adobe Corporation, Flash usage statistics, //www-935.ibm.com/services/us/iss/ September 2009, http://www.adobe.com/ xforce/trendreports/. products/player_census/flashplayer. [16] Symantec, threat re- [8] Mozilla Corporation, Known vulnerabilities port, 2010, http://www.symantec. in mozilla products, September 2009, com/business/theme.jsp?themeid= http://www.mozilla.org/security/ threatreport. known-vulnerabilities. [17] Trusteer, Flash security hole advisory, [9] Thomas Duebendorfer and Stefan Frei, Why August 2009, http://www.trusteer.com/ silent updates boost security, CRITIS 2009 files/Flash_Security_Hole_Advisory. Critical Infrastructures Security Workshop, pdf. May 2009. [18] Linus Upson, Bringing improved support for adobe flash player [10] Stefan Frei, Thomas Duebendorfer, and to google chrome, March 2010, Bernhard Plattner, Firefox (in)security http://blog.chromium.org/2010/03/ update dynamics exposed, ACM SIG- bringing-improved-support-for-adobe. COMM Computer Communication Review . 39 (2009), no. 1, 16–22.

4