2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement Security Vulnerabilities in Categories of Clones and Non-Cloned Code: An Empirical Study Md Rakibul Islam Minhaz F. Zibran Aayush Nagpal University of New Orleans, USA University of New Orleans, USA University of New Orleans, USA Email: [email protected] Email: [email protected] Email: [email protected] Abstract—Background: Software security has drawn immense changes [11], [19], [24], [25], [26], [30], [36], [43], [49], the importance in the recent years. While efforts are expected in impacts of clones on program’s changeability [17], [31], [32] minimizing security vulnerabilities in source code, the developers’ and comparative fault-proneness of cloned and non-cloned practice of code cloning often causes multiplication of such vulnerabilities and program faults. Although previous studies code [22], [42]. examined the bug-proneness, stability, and changeability of clones However, the security aspects of code clones have never against non-cloned code, the security aspects remained ignored. been studied before, although the reuse of vulnerable com- Aims: The objective of this work is to explore and understand ponents and source code (i.e., code clones) multiply security the security vulnerabilities and their severity in different types vulnerabilities [23], [29], [46]. This paper presents, a large of clones compared to non-clone code. Method: Using a state- of-the-art clone detector and two reputed security vulnerability empirical study on the security vulnerabilities in code clones detection tools, we detect clones and vulnerabilities in 8.7 million and presents a comparative analysis of these vulnerabilities in lines of code over 34 software systems. We perform a comparative different types of clones as opposed to non-cloned code. In study of the vulnerabilities identified in different types of clones particular, we address the following five research questions. and non-cloned code. The results are derived based on quan- titative analyses with statistical significance. Results: Our study RQ1: Do code clones contain higher number of security reveals that the security vulnerabilities found in code clones have vulnerabilities than non-cloned code or vice versa? higher severity of security risks compared to those in non-cloned — The answer to this question will add to our understanding code. However, the proportion (i.e., density) of vulnerabilities of the negative impacts of clones, which will be useful in in clones and non-cloned code does not have any significant cost-benefit analysis [40] for improved clone management. difference. Conclusion: The findings from this work add to our understanding of the characteristics and impacts of clones, which RQ2: Do clones of a certain category contain more security will be useful in clone-aware software development with improved vulnerabilities than others? software security. — If a certain type of clones are found to have higher number I. INTRODUCTION of security vulnerabilities, those clones will be high-priority candidates for removal or careful maintenance. Software security has become one of the most pressing concerns recently. Software developers are expected to write RQ3: Do code clones contain more severe (i.e. riskier) vul- secure source code and minimize security vulnerabilities in the nerabilities compared to non-cloned code or vice versa? systems under development. However, the developers’ copy- — All the security vulnerabilities are not equally risky in paste practice for code reuse often cause the multiplication and terms of security threat. The result of this research question propagation of program faults and security vulnerabilities [22], will advance our understanding of clones’ impacts and will be [25], [52]. Thus, there is a possibility that such vulnerabilities useful in cost-effective clone management [40]. can exist in cloned code at a higher rate. RQ4: Do clones of a certain category contain relatively severe Code clone (i.e., similar or duplicated code) is already (i.e., riskier) security vulnerabilities than others? identified as a notorious code smell (i.e., a symptom indicating — If a certain type of clones are found to have riskier security source of future problems) [14] that cause other problems vulnerabilities, those clones will demand especial attention and such as reduced code quality, code inflation, and change dif- high-priority in clone-removal process. ficulties [25], [31], [50], [52]. Nevertheless, software systems RQ5: Can we distinguish some security vulnerabilities that typically have 9%-17% [53] cloned code, and the proportion appear more frequently in cloned code as opposed to non- is sometimes found to be even 50% [37] or higher [13]. cloned code? Clones are arguably a major contributor to the high main- — If we can distinguish a set of vulnerabilities that ap- tenance cost for software systems, and as much as 80% pear more frequently in code clones, the finding will help of software costs are spent on maintenance [20]. Thus, to software developers to stay cautious of such vulnerabilities understand the characteristics and contexts of the detrimental while cloning source code. In addition, that particular set of impacts of clones, earlier studies examined the compara- vulnerabilities can be minimized by clone refactoring. tive stability of clones as opposed to non-cloned code [16], To answer the aforementioned research questions, we con- [18], [27], [28], [34], relationships of clones with bug-fixing duct a large-scale empirical study over 8.7 million lines of 978-1-5090-4039-1/17 $31.00 © 2017 IEEE 20 DOI 10.1109/ESEM.2017.9 source code in 34 open-source software systems written in C. Type-1 Clones: Identical pieces of source code with or Using a wide range of metrics and characterization criteria, we without variations in whitespaces (i.e., layout) and comments carry out in-depth quantitative analyses on the source code of are called Type-1 clones [51]. the systems with respect to different categories of code clones, Type-2 Clones: Type-2 clones are syntactically identical code non-cloned code, and a set of security vulnerabilities. In this fragments with variations in the names of identifiers, literals, regard, this paper makes the following two contributions: types, layout, and comments [51]. We present a large-scale comparative study of the security • Type-3 Clones: Code fragments, which exhibit similarities as vulnerabilities in code clones and non-cloned code. To of Type-2 clones and also allow further differences such as the best of our knowledge, no such study of the security additions, deletions or modifications of statements are known vulnerabilities in code clones exists in the literature. as Type-3 clones [51]. We also perform a comparative analysis of security • By the definitions above, Type-2 clones include Type-1 while vulnerabilities in different types of clones (e.g., Type-1, Type-3 clones include both Type-1 and Type-2. Let, T1, T2, and Type-2, and Type-3), which informs the relative security T3 respectively denote the sets of Type-1, Type-2, and Type-3 implications of clones at different similarity levels. clones in a software system. Mathematically, T T T . 1 ✓ 2 ✓ 3 II. TERMINOLOGY AND METRICS Thus, two additional subsets of Type-2 and Type-3 clones are In this section, we describe and define the terminologies characterized as follows. and metrics that are used in our work. Some of the metrics Pure Type-2 Clones: A set of pure Type-2 clones include are adapted from the literature [22], [38]. only those Type-2 clones that do not exhibit Type-1 similarity. Mathematically, T p = T T , where T p denotes the set of A. Security Vulnerabilities 2 2 − 1 2 pure Type-2 clones. A software security vulnerability is defined as a weakness in a software system that can lead to a compromise in integrity, Pure Type-3 Clones: A set of pure Type-3 clones include only those Type-3 clones, which do not exhibit similarities at availability or confidentiality of that software system. For p example, buffer overflow and dangling pointers are two well the levels of Type-1 or Type-2 clones. Mathematically, T3 = T T , where T p denotes the set of pure Type-3 clones. known security vulnerabilities. The cyber security community 3 − 2 3 maintains a community-developed list of common software C. Metrics security vulnerabilities where each category of vulnerability is enumerated with a CWE (Common Weakness Enumeration) The required metrics are defined in terms of density of number [1]. For example, CWE-120 refers to those vulnerabil- vulnerabilities with respect to (w.r.t.) syntactic blocks of code ities that fall into the CWE category of classic buffer overflow. (BOC) as well as w.r.t. lines of code (LOC). Only source lines More examples of security vulnerabilities along with their of code are taken into consideration excluding comments and CWE enumerations are presented in Table I. blank lines. Let, denote the set of all cloned code blocks and ¯ denote C C B. Code Clones the set of all non-cloned code blocks. Our study includes code clones at the granularity of syntac- Also let, denote the set of vulnerabilities found in VC C tic blocks (located between curly braces { and }) known as and denote the set of vulnerabilities located in ¯.A block clones VC C . We study clones at different levels of syntactic cloned code block in can be of category clone where similarities as mentioned below. p pC X T1,T2,T3,T2 ,T3 . Thus, can be split into multiple X2{ } VC sets with denoting the set of vulnerabilities identified in TABLE I VX clones of category . SECURITY VULNERABILITIES FREQUENTLY IDENTIFIED IN THE SYSTEMS X Security Vulnerability Description Density of vulnerabilities w.r.t. BOC in category clones, Buffer Overflow An application attempts to write data past the end of a buffer (CWE-120).