computers & security 77 (2018) 720–736 Available online at www.sciencedirect.com j o u r n a l h o m e p a g e : w w w . e l s e v i e r . c o m / l o c a t e / c o s e Software systems at risk: An empirical study of cloned vulnerabilities in practice ∗ Seulbae Kim, Heejo Lee Department of Computer Science and Engineering, Korea University, Seoul 136-713, Republic of Korea a r t i c l e i n f o a b s t r a c t Article history: With the growth of open source software (OSS), code clones –code fragments that are copied Received 1 October 2017 and pasted within or between software systems –are proliferating. Although code cloning Revised 10 February 2018 may expedite the process of software development, it often critically affects the security of Accepted 12 February 2018 software because vulnerabilities and bugs can easily be propagated through code clones. Available online 27 February 2018 These vulnerable code clones are increasing in conjunction with the growth of OSS, poten- tially contaminating many systems. Although researchers have attempted to detect code Keywords: clones for decades, most of these attempts fail to scale to the size of the ever-growing OSS Software vulnerability code base. The lack of scalability prevents software developers from readily managing code Open source software clones and associated vulnerabilities. Moreover, most existing clone detection techniques Vulnerability propagation focus overly on merely detecting clones and this impairs their ability to accurately find “vul- Code clone detection nerable” clones. Vulnerability exploitation In this paper, we propose VUDDY, an approach for the scalable detection of vulnerable code clones, which is capable of detecting security vulnerabilities in large software programs efficiently and accurately. Its extreme scalability is achieved by leveraging function-level granularity and a length-filtering technique that reduces the number of signature compar- isons. This efficient design enables VUDDY to preprocess a billion lines of code in 14 hours and 17 minutes, after which it requires a few seconds to identify code clones. In addition, we designed a vulnerability-preserving abstraction technique that renders VUDDY resilient to common modifications in cloned code, while preserving the vulnerable conditions even after the abstraction is applied. This extends the scope of VUDDY to identifying variants of known vulnerabilities, with high accuracy. An implementation of VUDDY has been ser- viced online for free at IoTcube, an automated vulnerability detection platform. In this study, we describe its principles, evaluate its efficacy, and analyze the vulnerabilities VUDDY de- tected in various real-world software systems, such as Apache HTTPD server and an Android smartphone. © 2018 Elsevier Ltd. All rights reserved. servers. The number of repositories in GitHub increased from 1. Introduction 1 million to 10 million between July 2010 and December 2013, and then to more than 66 million in August 2017, with most of For the past decade, open source software (OSS) has grown the repositories being software projects ( GitHub ). Black Duck rapidly in size, and is becoming a foundation for a major- Software and North Bridge revealed in 2016 Future of Open ity of applications, operating systems, databases, and web Source survey that 78% of the companies run open source ∗ Corresponding author. E-mail address: [email protected] (H. Lee). https://doi.org/10.1016/j.cose.2018.02.007 0167-4048/© 2018 Elsevier Ltd. All rights reserved. computers & security 77 (2018) 720–736 721 software ( Blackduck ). In literature, researchers showed that ing hash values through bloom filter. However, when it comes open source projects have linear to quadratic growth patterns to finding vulnerable code clones in massive code bases, Re- ( Godfrey and Tu, 2000; Scacchi, 2006; Succi et al., 2001 ). DeBug is still not satisfactory both in terms of accuracy and The considerable increase and the ubiquitous use of OSS scalability. Some techniques leverage a combination of vari- programs have naturally led to an increase in software vulner- ous approaches. VulPecker ( Li et al., 2016 ) characterizes a vul- abilities caused by code cloning, thereby posing dire threats nerability with a predefined set of features, then selects one of to the security of software systems ( Li et al., 2015 ). Numer- the existing code-similarity algorithms (e.g. Jang et al. (2012), ous instances in practice substantiate the claim that software Li et al. (2006) and Pham et al. (2010) which is optimal for the vulnerabilities are propagated through code cloning ( Dang type of vulnerable code fragment. A considerable amount of et al., 2017 ). For example, the OpenSSL Heartbleed vulnerabil- time required for the learning phase and algorithm selection ity (CVE-2014-0160) has affected a number of different types makes it improper to be used against massive open source of systems including web applications as well as OS distri- projects. An approach to analyze code heterogeneity for de- butions, and even hardware products such as routers and tecting repackaged Android malware ( Tian et al., 2016, 2017 ) switches, because the affected systems cloned a part or the utilizes machine learning algorithms, which are heavily de- entire OpenSSL library into their systems. pendent on the manual efforts to select features. Moreover, the life cycle of vulnerabilities even aggravates Meanwhile, our preliminary work, called VUDDY ( Kim such problem. That is, even if a vendor were to release a se- et al., 2017a ), proposed the most scalable and accurate curity patch immediately after the discovery of vulnerability approach for vulnerable code clone detection by leverag- in the original program, it would take time for the patch to ing function-level granularity and applying vulnerability- be fully deployed through every program that cloned the vul- preserving abstraction method. In this paper, we briefly de- nerable code of the original program ( Nappa et al., 2015 ). This scribe the approach we used for scalable and accurate detec- “time lag” between patch release and deployment increases tion of vulnerable code clones, then provide a detailed analysis when more steps and parties are involved in manufacturing of code reuse patterns and associated vulnerabilities in real- an end product. For example, Google takes the Linux kernel world software systems. In practice, we discovered several to make the Android operating system, and then smartphone kernel vulnerabilities in the recently released smartphones, vendors take the Android OS in order to make firmware for and successfully exploited the vulnerabilities. In addition, we their smartphone. Therefore, we can hardly expect that secu- identified several famous open source software which are re- rity patches issued for Linux kernel are immediately applied leased with old, vulnerable libraries. These results show that to smartphone’s firmware. Taking advantage of this fact, the security patches are deployed at a slow pace if code is reused, attackers in 2016 could successfully deploy the “Dogspectus” and because of this, unpatched vulnerabilities can easily be ransomware to a lot of Android smartphones by exploiting exploited for malicious activities even though they are not two-year-old kernel vulnerability (CVE-2014-3153) residing in zero-day vulnerabilities. the firmware. The contributions of this study include: To address such clone-related problems, many researchers have proposed code clone detection techniques ( Rattan et al., • Scalable clone detection: We propose VUDDY, an approach to 2013 ). However, few techniques are suitable for accurately scalable yet accurate code clone detection, which adopts finding vulnerability in a scalable manner. Token-based lex- a robust parsing and a novel fingerprinting mechanism for ical techniques such as CCFinder ( Kamiya et al., 2002 ) and CP- functions. VUDDY is able to process a billion lines of code in Miner ( Li et al., 2006 ) not only suffer from low scalability of only 14 hours and 17 minutes, which is an unprecedented complex token sequence comparing algorithms they take, but speed. also have high false positive rate caused by their aggressive ab- • Vulnerability-preserving abstraction: We present an effec- straction and filtering heuristics. This implies that this design tive abstraction scheme optimized for detecting unknown does not guarantee sufficient reliability to be useful for vulner- vulnerable code clones. This allows VUDDY to detect un- ability detection. Similarly, abstract data structure based ap- known vulnerable code clones, as well as known vulnera- proaches (e.g., Deckard ( Jiang et al., 2007 b), or Baxter’s ( Baxter bilities in a target program. Owing to this design, VUDDY et al., 1998 )) have to apply expensive tree-matching opera- detects 24% more vulnerable clones which are unknown tions or graph mining techniques for similarity estimation. Al- variants of known vulnerabilities. though such approaches would be capable of discovering code • Analysis of cloned vulnerabilities in real-world software sys- fragments with similar syntactic patterns, this does not guar- tems: We analyze and verify the cloned vulnerabilities de- antee an accurate vulnerability detection because two code tected in the software systems in practice, including fa- fragments with identical abstract syntax trees (ASTs) do not mous projects on GitHub and SourceForge, and smart- necessarily contain the same vulnerability ( Jiang et al., 2007 a). phones with