On the Impact and Defeat of Regular Expression Denial of Service
Total Page：16
File Type：pdf, Size：1020Kb
On the Impact and Defeat of Regular Expression Denial of Service James C. Davis Dissertation submitted to the Faculty of the Virginia Polytechnic Institute and State University in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science and Applications Dongyoon Lee, Chair Francisco Servant Danfeng (Daphne) Yao Ali R. Butt Patrice Godefroid April 30, 2020 Blacksburg, Virginia Keywords: Regular expressions, denial of service, ReDoS, empirical software engineering, software security Copyright 2020, James C. Davis On the Impact and Defeat of Regular Expression Denial of Service James C. Davis (ABSTRACT) Regular expressions (regexes) are a widely-used yet little-studied software component. Engi- neers use regexes to match domain-specific languages of strings. Unfortunately, many regex engine implementations perform these matches with worst-case polynomial or exponential time complexity in the length of the string. Because they are commonly used in user-facing contexts, super-linear regexes are a potential denial of service vector known as Regular ex- pression Denial of Service (ReDoS). Part I gives the necessary background to understand this problem. In Part II of this dissertation, I present the first large-scale empirical studies of super-linear regex use. Guided by case studies of ReDoS issues in practice (Chapter 3), I report that the risk of ReDoS affects up to 10% of the regexes used in practice (Chapter 4), and that these findings generalize to software written in eight popular programming languages (Chapter 5). ReDoS appears to be a widespread vulnerability, motivating the consideration of defenses. In Part III I present the first systematic comparison of ReDoS defenses. Based onthe necessary conditions for ReDoS, a ReDoS defense can be erected at the application level, the regex engine level, or the framework/runtime level. In my experiments I report that application-level defenses are difficult and error prone to implement (Chapter 6), that find- ing a compatible higher-performing regex engine is unlikely (Chapter 7), that optimizing an existing regex engine using memoization incurs (perhaps acceptable) space overheads (Chap- ter 8), and that incorporating resource caps into the framework or runtime is feasible but faces barriers to adoption (Chapter 9). In Part IV of this dissertation, we reflect on our findings. By leveraging empirical soft- ware engineering techniques, we have exposed the scope of potential ReDoS vulnerabilities, and given strong motivation for a solution. To assist practitioners, we have conducted a sys- tematic evaluation of the solution space. We hope that our findings assist in the elimination of ReDoS, and more generally that we have provided a case study in the value of data-driven software engineering. On the Impact and Defeat of Regular Expression Denial of Service James C. Davis (GENERAL AUDIENCE ABSTRACT) Software commonly performs pattern-matching tasks on strings. For example, when vali- dating input in a Web form, software commonly tests whether an input fits the pattern of a credit card number or an email address. Software engineers often implement such string- based pattern matching using a tool called regular expressions (regexes). Regexes permit software engineers to succinctly describe the sequences of characters that make up common “languages” like the set of valid Visa credit card numbers (16 digits, starting with a 4) or the set of valid emails (some characters, an ‘@’, and more characters including at least one ‘.’). Using regexes on untrusted user input in this manner may be a dangerous decision be- cause some regexes take a long time to evaluate. These slow regexes can be exploited by attackers in order to carry out a denial of service attack known as Regular expression Denial of Service (ReDoS). To date, ReDoS has led to outages affecting hundreds of websites and tens of thousands of users. While the risk of ReDoS is well known in theory, in this dissertation I present the first large-scale empirical studies measuring the extent to which slow regular expressions are used in practice. I found that about 10% of real regular expressions extracted from hundreds of thousands of software projects can exhibit longer-than-expected worst-case behavior in pop- ular programming languages including JavaScript, Python, and Ruby. Motivated by these findings, I then consider a range of ReDoS solution approaches: application refactoring, regex engine replacement, regex engine optimization, and resource caps. I report that application refactoring is error-prone, and that regex engine replacement seems unlikely due to incom- patibilities between regex engines. Some resource caps are more successful than others, but all resource cap approaches struggle with adoption. My novel regex engine optimizations seem the most promising approach for protecting existing regex engines, offering significant time reductions with acceptable space overheads. Dedication Praise God from whom all blessings flow. iv Acknowledgments So many people have contributed to this labor! Many thanks to all of you, and particularly... To Kirsten: Thank you for suggesting we go to graduate school, for sharing in its triumphs and tribulations, and for listening with a smile to years of incomprehensible jargon. To my multitudinous family: Thank you for your love, your questions, and for fruitful family colloquia. To quote my sister, Dr. Nan Pond: “I joyfully acknowledge the absurd cross- disciplinary thinktank that is my family.” I am also grateful to B. Danielak — you were the first of my friends to go to graduate school, and your passion has been an inspiration tome. To my friends (you know who you are): Thank you, variously, for many rounds of rac- quetball, many madcap conversations, many tea parties, and many Tuesday evenings in the Word. To L.C. Gayne, S. Duersch, and the GPFS team: Thank you for introducing me to the practice of computing, and for supporting me in the transition into research. To P. De Arras and the Idego Coffee team: Thank you for helping me through the days and (a few) nights. To my collaborators: Thank you for challenging me and for expanding my perspectives. I am particularly grateful to C. Coghlan, J. Donohue, Sk A. Hassan, A. Kazerouni, L. Michael IV, D. Moyer, and E. Williamson, for their expert and enthusiastic contributions to the research presented in this dissertation. To D. Craig, S. Fulton, T. Pennings, and T. Nishikawa: Thank you for holding my hand in my first fumbling attempts at research. To my doctoral committee: Thank you for your candid criticisms, and for your guidance and assistance on this path. To my advisor, Dongyoon Lee: Thank you for inviting me into your research lab, infect- ing me with your curiosity and delight in discovery, and asking more of me than I thought possible. I would not be here without your unceasing support. v List of Figures xi List of Tables xiii Part I Introduction and Background 1 Chapter 1 Introduction 2 1.1 Context and problem statement ......................... 2 1.2 Thesis ....................................... 4 1.3 Scientific contributions and applications .................... 4 1.4 Organization ................................... 5 1.5 Statement of authorship, attribution, and copyright .............. 5 Chapter 2 Background and related work 6 2.1 Outline ...................................... 6 2.2 The theory of regular languages ......................... 6 2.3 Algorithms for regex membership testing .................... 15 2.4 Regular expressions in software engineering practice .............. 25 2.5 Regular expression denial of service (ReDoS) .................. 34 2.6 Research on the use of regexes in practice ................... 40 2.7 What we don’t yet know ............................. 44 Part II Is ReDoS a Problem in Practice? 45 Chapter 3 Case studies of problematic super-linear regex behavior 47 3.1 Summary ..................................... 47 3.2 CVE 2015-6736 at MediaWiki .......................... 48 3.3 July 2019 service outage at Cloudflare ..................... 49 3.4 July 2016 service outage at Stack Overflow ................... 50 3.5 Performance problem in the Atom editor .................... 51 vi 3.6 Lessons learned .................................. 53 Chapter 4 Measuring the use of super-linear regexes in practice 55 4.1 Summary ..................................... 55 4.2 Study design and research questions ....................... 56 4.3 RQ1: How prevalent are super-linear regexes in practice? ........... 56 4.4 RQ2: How strongly vulnerable are the super-linear regexes? ......... 61 4.5 RQ3: Which application domains do super-linear regexes affect? ....... 63 4.6 Discussion ..................................... 65 4.7 Threats to validity ................................ 66 Chapter 5 Generalizing regex measurements 68 5.1 Summary ..................................... 68 5.2 Motivation ..................................... 69 5.3 Study design and research questions ....................... 71 5.4 Regex metrics for use in hypothesis testing ................... 72 5.5 RQ1: Does the Extraction Methodology Hypothesis hold? .......... 77 5.6 RQ2: Does the Cross-Language Hypothesis hold? ............... 83 5.7 RQ3: Does super-linear behavior generalize to other regex engines? ..... 89 5.8 RQ4: Can we replicate other previous regex research? ............. 92 5.9 Discussion ..................................... 93 5.10 Threats to validity ................................ 95 Part III Evaluating