Browseraudit: Automated Testing of Browser Security Features
Total Page:16
File Type:pdf, Size:1020Kb
ifact t * r * Comple t te A A n * te W E is s * e C n l l o D C A o * * c T BrowserAudit: Automated Testing u e m S s E u e S e n I R t v e o d t y * s E a * a l d u e a of Browser Security Features t Charlie Sergio Maffeis Chris Novakovic Hothersall-Thomas Department of Computing Department of Computing Netcraft Ltd, UK Imperial College London, UK Imperial College London, UK [email protected] [email protected] [email protected] ABSTRACT As such, browsers need to offer a variety of standardised The security of the client side of a web application relies on security mechanisms which can be relied upon uniformly browser features such as cookies, the same-origin policy and by the client side of web applications, in order to deliver HTTPS. As the client side grows increasingly powerful and security guarantees to their users. For example, the same- sophisticated, browser vendors have stepped up their offering origin policy (SOP) [34] is effective at preventing a range of security mechanisms which can be leveraged to protect of cross-site scripting (XSS) attacks [38] against users' web it. These are often introduced experimentally and informally browsers and is an integral aspect of modern web-based se- and, as adoption increases, gradually become standardised curity. On the other hand, it is sometimes excessively strict; (e.g., CSP, CORS and HSTS). Considering the diverse land- for instance, it forbids the sharing of information between scape of browser vendors, releases, and customised versions different subdomains, a common requirement of large web for mobile and embedded devices, there is a compelling need sites. It is also coarse-grained, and several attempts have for a systematic assessment of browser security. been made to enforce finer-grained access control [41, 39] We present BrowserAudit, a tool for testing that a de- and origins [22, 23, 29] in the browser. A variety of contem- ployed browser enforces the guarantees implied by the main porary web browsers implement the Cross-Origin Resource standardised and experimental security mechanisms. It in- Sharing (CORS) [46] standard, which may be used to control cludes more than 400 fully-automated tests that exercise the flow of information between server-side resources and a broad range of security features, helping web users, ap- client-side scripts that attempt to access those resources via plication developers and security researchers to make an APIs. However, even fully-compliant implementations of the informed security assessment of a deployed browser. We SOP and CORS mechanisms in some cases do not regulate validate BrowserAudit by discovering both fresh and known access to other resources, such as images, embedded objects security-related bugs in major browsers. and web fonts, that can leave web applications vulnerable to cross-site request forgery (CSRF) attacks [20], clickjack- ing [36], framebusting [43] and CSS-based attacks [33]. The Categories and Subject Descriptors Content Security Policy (CSP) standard [45] enables much D.4.6 [Operating systems]: Security and Protection finer-grained control over the loading of arbitrary resources on a web page, mitigating several of these issues. These Keywords are just some examples of established and emerging security mechanisms offered by modern browsers. Web security, web browser testing, same-origin policy, Con- Such mechanisms are often introduced experimentally and tent Security Policy, Cross-Origin Resource Sharing, click- informally. As adoption increases, they gradually become jacking, cookies standardised, and after numerous security reviews and bug reports they can eventually be relied upon consistently across 1. INTRODUCTION browsers [19, 37, 20]. Reaching that stage is not easy. For Personal data, business transactions, critical infrastruc- example, correctly implementing the CSP specification is non- ture and even cars, refrigerators and lightbulbs are exposed trivial: it is a lengthy document with many cross-references through web interfaces to a wide variety of web browsers. to other standards and RFCs, many of which have been su- Hence, the browser plays a key role in the modern information perseded by newer (and conflicting) standards and RFCs. It infrastructure, as the main gateway to access the information is possible that a browser vendor could incorrectly implement and capabilities made available online. some part of the CSP and thus fail to provide some of its security guarantees to their users. There is therefore a need for an automated tool that enables browser developers to complement low-level unit tests targeted at isolated source Permission to to make make digital digital or hard or hard copies copies of part of or all all ofor this part work of this for personal work for or code modules with high-level testing of the effectiveness of classroompersonal oruse classroom is granted without use is feegranted provided without that copies fee provided are not made that or copies distributed are the implementation of the security features once the browser fornot profit made or or commercial distributed advantage for profit and or that commercial copies bear thisadvantage notice and and the that full copies citation on the first page. Copyrights for third-party components of this work must be honored. is deployed. Forbear all this other notice uses, and contact the the full Owner/Author. citation on the first page. To copy otherwise, to In this paper we introduce BrowserAudit, a framework republish, to post on servers or to redistribute to lists, requires prior specific Copyright is held by the owner/author(s). for testing whether a deployed browser correctly enforces permission and/or a fee. ISSTA’15,, July July 12–17, 12–17, 2015, 2015, Baltimore, Baltimore, MD, MD, USA USA the security guarantees implied by the main standardised ACM.Copyright 978-1-4503-3620-8/15/07 2015 ACM 978-1-4503-3620-8/15/07 ...$15.00. http://dx.doi.org/10.1145/2771783.2771789 37 security mechanisms. For practical purposes, we present • We implemented the first fully-automated web application BrowserAudit as a standalone web application that auto- that comprehensively tests browser security features and matically tests the browser used to access it. BrowserAudit provides detailed information to a variety of user bases. has been designed with different sets of users in mind. A casual web user can run the tests to gain a simple secur- • We used BrowserAudit to discover previously unknown ity assessment of their browser: critically vulnerable, non- vulnerabilities in a major web browser. critically vulnerable, or okay. With the recent surge of se- curity breaches reported in the news, people are becoming 2. DESIGN OVERVIEW increasingly security-conscious and we believe there is an The goals underlying the design of BrowserAudit are the increasing demand for tools that inform the public about se- following: curity. A security researcher can benefit even more, viewing a detailed breakdown of each test result, and seeing which • Wide coverage: BrowserAudit should demonstrate that a security features passed our tests and which had problems. wide range of browser security mechanisms can be tested We display textual descriptions for each category of tests and automatically, reliably and efficiently. Complete test cov- the client-side source code of the tests. Browser developers erage of any such mechanism is not practically feasible, can use BrowserAudit to debug their security features and and beyond the scope of this project.1 web developers can use it as a way to ascertain the secur- ity capability of users' browsers (Section 2). We chose to • Extensibility: By its very nature, BrowserAudit will always implement a careful selection of tests that covers both the be a work in progress. As the browser threat landscape most important browser security mechanisms that should be evolves, more tests will be needed to cover new security implemented in any browser, and some of the most prom- mechanisms, or to extend the coverage of existing ones. ising experimental ones that are not yet widely implemented. Our design should ease the task of creating, debugging Starting from the code of individual test cases, we identified and integrating additional test cases. and generalised common patterns in order to automatically • Ease of use: BrowserAudit should be easily accessible on generate hundreds of tests. BrowserAudit automatically tests any modern browser connected to the Internet, without over 400 behaviours where a certain action should either be the need to install additional software. It should require allowed or blocked according to an implied browser security no interaction from the user, otherwise running hundreds policy (Section 3). of tests would be impractical. Moreover, relying on user We designed BrowserAudit to be efficient and scalable, interaction would prevent the desired aim of running the and we evaluated its performance and its accuracy extens- tests transparently in the background. ively by running it on a number of browsers and platforms. Using BrowserAudit, we have discovered several previously • Broad audience: Our design should support a diverse unknown security bugs in recent versions of Mozilla Firefox, range of users. A report on the security effectively offered which we have reported to the developers (Section 4.4). by a deployed browser should benefit browser developers, Whilst there are well-understood methodologies for gen- penetration testers, security researchers and web users. erating unit tests for a given code base, there is no general solution to the problem of testing the end-to-end security be- • Scalability: Our design should be scalable on the server haviour of a family of applications (in our case web browsers) side. Several users may be testing their browser at the that must respect precise interoperability constraints (web same time, and many security tests concern features that standards) but can widely differ in implementation architec- involve communicating with the server.