Radboud University Nijmegen Master Thesis Computer Science A benchmark approach to analyse the security of web frameworks Author: Supervisor: K. Reintjes, BSc. Prof. dr. M.C.J.D. van Eekelen [email protected] [email protected] Second reader: Dr. ir. E. Poll [email protected] February 19, 2014 Abstract Web frameworks often offer various security functionalities and protection mechanisms that developers can use to secure their applications. However, as it turns out, these frameworks themselves are not always that secure, which can have severe consequences. For example, one vulnerability in the Ruby on Rails framework was so severe that many web applications had to be taken off-line temporarily, among which the Dutch government's authentication system DigiD. Also other popular frameworks have had several security problems in the past. Unfortunately, it appears there is no good way to analyse the security of these frameworks and detect vulnerabilities before they occur in released versions. We also note a significant lack of scientific research on the security of web frameworks. Therefore, the goal for this research was to find a general methodology to analyse the security of web frameworks. With this methodology it should be possible to detect vulnerabilities in any web framework, preferably in a (partially) automated way. There are several challenges when trying to analyse the security of a web framework. These challenges make it hard to analyse the framework directly. Therefore we propose a benchmark approach. This approach uses a benchmark implemented in the target framework, which is analysed with well known dynamic web vulnerability scanners. The approach includes a general, framework-independent design for this benchmark. To use the approach, the benchmark needs to be implemented in the target framework. This can be seen as an instantiation of the benchmark for that framework. We then use dynamic web vulnerability scanners on this implementation to analyse the security of the framework. A vulnerability discovered in the benchmark implementation could indicate a vulnerability in the framework. During this research we developed our approach and designed the required benchmark for SQL in- jections and XSS vulnerabilities. We also tried the approach in practice, by applying it on the Ruby on Rails web framework. We implemented the benchmark in this framework and analysed it using two dynamic scanners, Arachni and W3af. Using our approach we discovered five vulnerabilities, of which three were completely new. In this thesis we present our approach and the design of the benchmark. We discuss the benchmark implementation in Rails and the results of the analysis. Finally, we evaluate the approach and results, and present potential improvements and other ways to analyse the security of web frameworks. We conclude that our approach is indeed capable of analysing web frameworks for security vulnerabilities, but is not perfect either, since it has several weaknesses. ii Acknowledgements I would like to thank several people for their help with my research and/or my thesis. First of all, my supervisor, Marko van Eekelen, for suggesting to widen the topic from \Ruby on Rails" to \web frameworks in general", as well as for his help during the research and his constructive feedback on my thesis. Secondly, Erik Poll, for acting as second reader for this thesis. Furthermore, I thank Christiaan Thijssen for his useful advice and helpful discussions, as well as reviewing the complete final draft version of this thesis. I also would like thank Arjan Diepenbroek for reviewing several chapters of the thesis, and of course for answering all my questions about English grammar and spelling. For the latter, I also thank Floris de Lange. Finally, I wish to thank my parents for their general support and advice throughout the complete project. iii Contents 1 Introduction 1 1.1 Background and motivation . 1 1.2 Analysing the security of web frameworks . 2 1.3 Research goal . 3 1.4 Research contributions . 3 1.5 Organisation of this thesis . 4 2 Web frameworks and security 5 2.1 Web frameworks . 5 2.1.1 Architecture . 6 2.1.2 Features . 7 2.1.3 Database interface . 8 2.1.4 Security . 9 2.1.5 Examples of web frameworks . 10 2.2 Common security functionalities in web frameworks . 11 2.2.1 Protection against SQL injections . 11 2.2.2 Protection against other injection attacks . 12 2.2.3 Cross-Site Scripting protection . 13 2.2.4 Cross-Site Request Forgery protection . 15 2.2.5 Protection against HTTP Header injection . 16 2.2.6 Protection against Unvalidated Redirects . 17 2.2.7 File Inclusion protection . 18 2.2.8 Mass assignment protection . 19 2.2.9 Session management . 20 2.2.10 Other security functionalities . 20 3 Ruby on Rails 21 3.1 Ruby and Rails . 21 3.2 Rails versions . 22 3.3 Security functionalities of Rails . 23 3.4 Recent vulnerabilities in Rails . 26 3.4.1 Categories . 26 3.4.2 Trends . 28 4 A benchmark approach 30 4.1 Motivation . 30 4.2 The approach . 31 4.3 Global benchmark design . 32 4.3.1 Benchmark goal . 33 4.3.2 Benchmark requirements . 33 4.3.3 Benchmark design choices . 34 4.3.4 Global benchmark architecture . 37 4.4 SQL injection module design . 38 4.4.1 SQLi module design choices . 39 4.4.2 SQLi submodules design . 42 4.4.3 SQLi module design summary . 45 iv 4.5 XSS module design . 46 4.5.1 XSS module design choices . 46 4.5.2 XSS submodules design . 48 4.5.3 XSS module design summary . 50 5 Implementing the benchmark in Rails 51 5.1 The base application . 52 5.2 SQLi module implementation . 52 5.2.1 Database system . 52 5.2.2 Types of methods . 53 5.2.3 SQLi submodules implementation . 54 5.2.4 SQLi module implementation challenges . 55 5.3 XSS module implementation . 57 5.3.1 XSS submodules implementation . 57 5.3.2 XSS module implementation challenges . 58 5.4 Evaluation of the implementation . 59 5.4.1 General observations . 59 5.4.2 Amount of work per module . 59 5.4.3 Evaluation conclusion . 60 5.5 Availability of the implementation . 60 6 Options for the benchmark analysis 61 6.1 Analysis options . 61 6.1.1 Main categories of analysis options . 61 6.1.2 Unsuitable analysis options . 62 6.1.3 Motivation for dynamic scanners . 63 6.2 Dynamic security scanners . 64 6.2.1 Internal workings . 64 6.2.2 Detecting SQLi and XSS . 66 6.2.3 Usage and configuration . 68 6.2.4 Strengths and limitations . 69 6.2.5 Choosing the scanners . 70 6.3 Arachni and W3af . 71 6.3.1 Introduction . 71 6.3.2 SQLi and XSS detection capabilities . 71 6.3.3 Basic configuration . 73 7 Analysing Rails with the benchmark 75 7.1 Analysis environment . 75 7.2 Analysis approach . 76 7.3 Results of the analysis . 76 7.3.1 Results of the SQLi benchmark . 76 7.3.2 Results of the XSS benchmark . 78 7.4 Vulnerabilities in Rails . ..