DOCSLIB.ORG

Code Similarity and Clone Search in Large-Scale Source Code Data

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Code Similarity and Clone Search in Large-Scale Source Code Data Code Similarity and Clone Search in Large-Scale Source Code Data Chaiyong Ragkhitwetsagul A dissertation submitted in fulfillment of the requirements for the degree of Doctor of Philosophy of University College London. Department of Computer Science University College London September 30, 2018 Declarations I, Chaiyong Ragkhitwetsagul, confirm that the work presented in this thesis is my own. Where information has been derived from other sources, I confirm that this has been indicated in the thesis. The papers presented here are original work undertaken between September 2014 and July 2018 at University College London. They have been published or submitted for publication as listed below along with a summary of my contributions to the papers. 1. C. Ragkhitwetsagul, J. Krinke, D. Clark, Similarity of source code in the presence of pervasive modifications, in Proceedings of the 16th International Working Conference on Source Code Analysis and Manipulation (SCAM ’16), 2016. Contribution: Dr. Krinke designed and started the initial experiment which I later continued. I incorporated more tools, handled the tool executions, collected the results, analysed and discussed the results with Dr. Krinke. The three authors wrote the paper. 2. C. Ragkhitwetsagul, J. Krinke, D. Clark, A comparison of code similarity analysers, Empirical Software Engineering (EMSE), 2017. Contribution: This publication is an invited journal extension of the SCAM ’16 paper to a special issue of Empirical Software Engineering. I substantially expanded the experiment by doubling the data set size, rerunning the first and second experimental scenarios, and introducing three additional experimental scenarios under the supervision of Dr. Krinke. The paper has been written by the three authors. 3 3. C. Ragkhitwetsagul and J. Krinke, Using compilation/decompilation to en- hance clone detection, in Proceedings of the 11th International Workshop on Software Clone (IWSC ’17), 2017. Contribution: I designed and performed the experiment and analysed the results with suggestions from Dr. Krinke. The paper was written by the two authors. 4. C. Ragkhitwetsagul and J. Krinke, Awareness and experience of developers to outdated and license-violating code on Stack Overflow : An online survey, UCL Research Note, 2017. Contribution: I completed the two surveys and analysed the results under the supervision of Dr. Krinke. The research note is written by me and revised by Dr. Krinke. 5. C. Ragkhitwetsagul, M. Paixoa, J. Krinke, G. Bianco, R. Oliveto, Toxic code snippets on Stack Overflow, Submitted to IEEE Transactions on Software Engineering (TSE), 2018 (after the first round of Revise and Resubmit as New). Contribution: I lead the project and performed all the experiments and the surveys under the supervision of Dr. Krinke. G. Bianco, an exchange student from University of Molise under the supervision of Dr. Oliveto, helped on the data collection from Stack Overflow. My PhD colleague, M. Paixao, was involved in the manual clone classification. The paper was written by me and revised by Dr. Krinke and Dr. Oliveto. 6. C. Ragkhitwetsagul, J. Krinke, Siamese: Scalable and incremental code clone search via multiple code representations, Submitted to Empirical Software Engineering (EMSE), 2018. Contribution: I performed the whole study under the supervision of Dr. Krinke. The paper was written by me and revised by Dr. Krinke. 4 7. R. White, J. Krinke, E. Barr, C. Ragkhitwetsagul, F. Sarro, M. Al-arfaj, Exploiting test-to-code traceability links for reuse, Submitted to The 33rd International Conference on Automated Software Engineering (ASE ’18). Contribution: I provided and configured the Siamese clone search engine as a code similarity tool for the RELATEST test recommendation tool. I also involved in the manual validations of the recommended tests. The paper was written by R. White, Dr. Krinke, me, and Dr. Barr. In addition, the list below includes the papers that I authored or co-authored and published or submitted for peer review during the programmed of my study. They are not featured in this thesis. 1. C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke, J. H. Drake, Searching for configurations in clone evaluation – A replication study, in Proceedings of the 8th Symposium on Search-Based Software Engineering (SSBSE ’16), 2016. 2. M. Paixao, J. Krinke, D. Han, C. Ragkhitwetsagul, and M. Harman, Are developers aware of the architectural impact of their changes?, in Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE ’17), 2017. 3. M. Alonaizan, D. Han, J. Krinke, C. Ragkhitwetsagul, D. Schwartz- Narbonne, and B. Zhu, Recommending related code reviews, Submitted to IEEE Transactions on Software Engineering (TSE), 2018 (Major Revisions). 4. C. Ragkhitwetsagul, J. Krinke, B. Marnette, A picture is worth a thousand words: Code clone detection based on image similarity, in Proceedings of the 12th International Workshop on Software Clones (IWSC ’18), 2018. 5 5. J. Wilkie, Z. Al Halabi, A. Karaoglu, J. Liao, G. Ndungu, C. Ragkhitwetsagul, M. Paixao, and J. Krinke, Who’s this? Developer identification using IDE event data, in Proceedings of the 15th International Conference on Mining Software Repositories (MSR ’18), 2018. Chaiyong Ragkhitwetsagul Date Abstract Software development is tremendously benefited from the Internet by having online code corpora that enable instant sharing of source code and online developer’s guides and documentation. Nowadays, duplicated code (i.e., code clones) not only exists within or across software projects but also between online code repositories and websites. We call them “online code clones.” They can lead to license violations, bug propagation, and re-use of outdated code similar to classic code clones between software systems. Unfortunately, they are difficult to locate and fix since the search space in online code corpora is large and no longer confined to a local repository. This thesis presents a combined study of code similarity and online code clones. We empirically show that many code snippets on Stack Overflow are cloned from open source projects. Several of them become outdated or violate their original license and are possibly harmful to reuse. To develop a solution for finding online code clones, we study various code similarity techniques to gain insights into their strengths and weaknesses. A framework, called OCD, for evaluating code similarity and clone search tools is introduced and used to compare 34 state-of-the- art techniques on pervasively modified code and boiler-plate code. We also found that clone detection techniques can be enhanced by compilation and decompilation. Using the knowledge from the comparison of code similarity analysers, we create and evaluate Siamese, a scalable token-based clone search technique via multiple code representations. Our evaluation shows that Siamese scales to large- scale source code data of 365 million lines of code and offers high search precision and recall. Its clone search precision is comparable to seven state-of-the-art clone Abstract 7 detection tools on the OCD framework. Finally, we demonstrate the usefulness of Siamese by applying the tool to find online code clones, automatically analyse clone licenses, and recommend tests for reuse. Impact Statement The knowledge and methods of software analysis presented in this thesis are beneficial to software quality improvement and could have impacts on both software engineering research and industry. The phenomenon of code duplication, i.e., code cloning, is well-known in both research and industry, although sometimes with different terms used to describe it. There are two camps of beliefs that clones lead to bug propagation and degrades software maintenance, and that clones rarely relate to bugs and do not affect software quality. However, they both agree that clones have to be made explicit. This thesis strengthens the body of knowledge in code clone research by studying a new type of code clones, a cloning activity to and from online sources such as Stack Overflow programming Q&A website. The thesis shows that the result of such cloning for which we call online code clones, have at least two issues: outdated code and software license incompatibility. Since Stack Overflow is a popular website with 7.6 million users, the issues from online code cloning can affect a large number of programmers around the world. The findings lead to an urgent need to mitigate the issues, both by research and Stack Overflow itself. On the research side, the thesis develops a scalable code clone search technique and a tool called Siamese, to provide a scalable solution to locate online code clones from large source code corpora. The thesis demonstrates that Siamese can be put to use to efficiently locate clones between Stack Overflow and a hundred open source projects. The technique can incorporate automated license analysis to detect violations of software licenses and has a potential to be transformed to a cloud Abstract 9 service that offers real-time checks for online code clones in any software projects. On the industry side, the thesis calls for action from Stack Overflow to mitigate the issues of outdated code and software license violations. The survey results in the thesis clearly show that Stack Overflow users are aware of the issues and some of them need a guideline from Stack Overflow. Moreover, the website must collect the origin of the source code examples to check for their newer version and their original software license. Lastly, the comparison of 34 code similarity analysers presented in the thesis is the largest in existence and potentially an invaluable guide for future users of similarity detection in source code. The findings can be used both in academia and outside academia on any studies or projects related to code similarity. Acknowledgements Many people have helped me during the four years of my PhD at UCL, London. I would not have made it this far without them. Thus, I would like to express my gratitude to the following people. First and foremost, I would like to thank my supervisor, Dr.
Load more

Recommended publications
  • R L W D by Oktie Hassanzadeh A
    R L W D by Oktie Hassanzadeh A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy Graduate Department of Computer Science University of Toronto Copyright © 2012 by Oktie Hassanzadeh Abstract Record Linkage for Web Data Oktie Hassanzadeh Doctor of Philosophy Graduate Department of Computer Science University of Toronto 2012 Record linkage refers to the task of nding and linking records (in a single database or in a set of data sources) that refer to the same entity. Automating the record linkage process is a challenging problem, and has been the topic of extensive research for many years. Several tools and techniques have been developed as part of research prototypes and commercial so- ware systems. However, the changing nature of the linkage process and the growing size of data sources create new challenges for this task. In this thesis, we study the record linkage problem for Web data sources. We show that tra- ditional approaches to record linkage fail to meet the needs of Web data because 1) they do not permit users to easily tailor string matching algorithms to be useful over the highly heteroge- neous and error-riddled string data on the Web and 2) they assume that the attributes required for record linkage are given. We propose novel solutions to address these shortcomings. First, we present a framework for record linkage over relational data, motivated by the fact that many Web data sources are powered by relational database engines. is framework is based on declarative specication of the linkage requirements by the user and allows linking records in many real-world scenarios.
    [Show full text]
  • Using String Metrics to Improve the Design of Virtual Conversational Characters: Behavior Simulator Development Study
    JMIR SERIOUS GAMES García-Carbajal et al Original Paper Using String Metrics to Improve the Design of Virtual Conversational Characters: Behavior Simulator Development Study Santiago García-Carbajal1*, PhD; María Pipa-Muniz2*, MSc; Jose Luis Múgica3*, MSc 1Computer Science Department, Universidad de Oviedo, Gijón, Spain 2Cabueñes Hospital, Gijón, Spain 3Signal Software SL, Parque Científico Tecnológico de Gijón, Gijón, Asturias, Spain *all authors contributed equally Corresponding Author: Santiago García-Carbajal, PhD Computer Science Department Universidad de Oviedo Campus de Viesques Office 1 b 15 Gijón, 33203 Spain Phone: 34 985182487 Email: [email protected] Abstract Background: An emergency waiting room is a place where conflicts often arise. Nervous relatives in a hostile, unknown environment force security and medical staff to be ready to deal with some awkward situations. Additionally, it has been said that the medical interview is the first diagnostic and therapeutic tool, involving both intellectual and emotional skills on the part of the doctor. At the same time, it seems that there is something mysterious about interviewing that cannot be formalized or taught. In this context, virtual conversational characters (VCCs) are progressively present in most e-learning environments. Objective: In this study, we propose and develop a modular architecture for a VCC-based behavior simulator to be used as a tool for conflict avoidance training. Our behavior simulators are now being used in hospital environments, where training exercises must be easily designed and tested. Methods: We define training exercises as labeled, directed graphs that help an instructor in the design of complex training situations. In order to increase the perception of talking to a real person, the simulator must deal with a huge number of sentences that a VCC must understand and react to.
    [Show full text]
  • Redacted VV MQP Report with Math
    ______________________________________________________________________________ Developing Models to Visualize & Analyze User Interaction for Financial Technology Websites ______________________________________________________________________________ Project Team Amanda Ezeobiejesi Computer Science Guy Katz Industrial Engineering Alissa Ostapenko Computer Science and Mathematical Sciences Project Advisor Professor Michael Ginzberg Foisie Business School Project Co-Advisors Professor Rodica Neamtu Department of Computer Science Professor Sara Saberi Foisie Business School Professor Jon Abraham Department of Mathematical Sciences This report represents the work of WPI undergraduate students submitted to the faculty as evidence of completion of a degree requirement. WPI routinely publishes these reports on its website without editorial or peer review. For more information about the projects program at WPI, please see: http://www.wpi.edu/academics/ugradstudies/project-learning.html Abstract Vestigo Ventures manually processes website traffic data to analyze the business performance of financial technology companies. By analyzing how people navigate through company websites, Vestigo aims to understand different customer activity patterns. Our team designed and implemented a tool that automatically processes clickstream data to visualize different customer activity within a website and compute statistics about user activity. This tool will provide Vestigo insight on the effectiveness of their clients’ website structures and help them make recommendations to their clients. 1 Acknowledgements Our team would like to extend our sincere gratitude to the individuals mentioned below, from Vestigo Ventures, Cogo Labs, and Link Ventures, as well as our advisors for their support and encouragement throughout the duration of this project: Vestigo Ventures Ian Sheridan, a managing director and co-founder of Vestigo Ventures, who made this project possible. He established the initial connection with WPI and checked in on us from time to time.
    [Show full text]
  • Textalgorithmen Vorlesung Im Wintersemester 2003/04 Otto-Von
    Textalgorithmen Vorlesung im Wintersemester 2003/04 Otto-von-Guericke-Universit¨at Magdeburg Fakult¨at fur¨ Informatik Ralf Stiebe [email protected] Inhaltsverzeichnis Einleitung 4 0 Begriffe und Notationen 6 1 Exakte Suche nach einem Wort 8 1.1 Naiver Algorithmus . 8 1.2 R¨ander und Perioden . 9 1.3 Der Algorithmus von Knuth-Morris-Pratt . 12 1.4 Suche mit deterministischen endlichen Automaten . 15 1.5 Boyer-Moore-Algorithmus . 17 1.6 Apostolico-Giancarlo-Algorithmus . 22 1.7 Algorithmus von Vishkin . 25 1.8 Der Shift-And-Algorithmus . 29 1.9 Algorithmen mit Suffixautomaten . 30 1.10 Karp-Rabin-Algorithmus . 33 1.11 Eine untere Schranke fur¨ Algorithmen zur Wortsuche . 35 2 Exakte Suche nach komplexen Mustern 38 2.1 Suche nach endlich vielen W¨ortern . 38 2.2 Suche nach zweidimensionalen Bildern . 44 2.3 Suche nach regul¨aren Ausdrucken.........................¨ 48 2.4 Einige Spezialf¨alle regul¨arer Ausdrucke......................¨ 51 3 Inexakte Suche 54 3.1 Einleitung . 54 3.2 L¨osung durch dynamische Programmierung . 56 3.3 Alignments mit beschr¨ankter Fehlerzahl . 63 3.4 Alignments mit Lucken(gaps)...........................¨ 66 3.5 L¨osung mit linearem Platzbedarf . 68 3.6 L¨osung in subquadratischer Zeit . 69 3.7 Multiple Alignments . 72 4 Indexieren von Texten 77 4.1 Einfuhrung¨ ..................................... 77 4.2 Datenstrukturen fur¨ Indexe . 77 4.2.1 Suffixb¨aume . 77 4.2.2 Suffix-Arrays . 79 1 4.2.3 Varianten und Erweiterungen . 79 4.3 Konstruktion von Suffixb¨aumen . 80 4.3.1 Naiver Algorithmus . 81 4.3.2 Algorithmus von McCreight .
    [Show full text]
  • A Comparative Study on Sequence Matcher and Levenshtein Distance Procedure
    International Journal of Computer Sciences and Engineering Open Access Research Paper Volume-6, Issue-4 E-ISSN: 2347-2693 Characteristic mining of Mathematical Formulas from Document - A Comparative Study on Sequence Matcher and Levenshtein Distance procedure G. Appa Rao 1*, G. Srinivas 2, K.Venkata Rao 3, P.V.G.D. Prasad Reddy 4 1* Department of CSE, GIT, GITAM,VISAKHAPATNAM,INDIA 2 Department of IT, ANITS, VISAKHAPATNAM, INDIA 3, 4 Department of CSSE, AndhraUniversity,VISAKHAPATNAM,INDIA *Corresponding Author: [email protected] Available online at: www.ijcseonline.org Received: 20/Mar/2018, Revised: 28/Mar/2018, Accepted: 19/Apr/2018, Published: 30/Apr/2018 Abstract— The key predicament in the present circumstances is how to categorize the mathematically related keywords from a given text file and store them in one math text file. As the math text file contains only the keywords which are related to mathematics. The math dataset is a collection of huge amount of tested documents and stored in math text file. The dataset is trained with giant amount of text files and the size of dataset increases, training with various text samples. Finally the dataset contains only math-related keywords. The proposed approaches evaluated on the text containing individual formulas and repeated formulas. The two approaches proposed are one is Sequence matcher and another one is Levenshtein Distance, both are used for checking string similarity. The performance of the repossession is premeditated based on dataset of repetitive formulas and formulas appearing once and the time taken for reclamation is also measured. Keywords—Levenshtein distance,Sequence matcher I.
    [Show full text]
  • A Bibliography on Pattern Matching, Regular Expressions, and String Matching
    A Bibliography on Pattern Matching, Regular Expressions, and String Matching Nelson H. F. Beebe University of Utah Department of Mathematics, 110 LCB 155 S 1400 E RM 233 Salt Lake City, UT 84112-0090 USA Tel: +1 801 581 5254 FAX: +1 801 581 4148 E-mail: [email protected], [email protected], [email protected] (Internet) WWW URL: http://www.math.utah.edu/~beebe/ and David Salomon Professor Emeritus Department of Computer Science California State University Northridge Northridge, CA, USA 12 August 2021 Version 1.170 Title word cross-reference (l; d) [Tan14]. 1 [Mun07]. 2 [ASG99, BSM+07, BZ98, CR95b, KPR97, KD15, LT90b, OND98, SHCY93, Via02, Via04]. $29.95 [Ano97a]. 3 [BSM+07, CJ93, LT90b, TCCK90]. $65.00 [Ano97b]. 2 [Ram94]. 33 [BGFK15]. 4 [ZLN11]. c [WD99]. cn [Rob79]. d [CGK08, CDEK95, GKP19, dRL95]. δ [CIL+03]. [Gef03, HM98, HSW01, Lif03]. K [COZ09, LVN87, ATX21, ALP04, CLZ+15, CKP+21, CW18a, CW18b, CN21, 1 2 FWW13a, FGKU15, GG86, GU16, GN19, GGF13, Gra15, GL89, HT17, JL93, JL96, KRS19a, KRS19b, KVX12, LV88, NYuR15, NR17, QQC+13, WD99]. L1 [LP08, LP11]. L2 [LP11]. L1 [LP11]. Lp [WC14]. µ [DJ96]. N [ML96a, ML96b, KST94, KWL07, KWLL08, Wag74]. O(modT mod 3) [KKM+85]. O(log log n)[BG90].O(n2 log m) [CNS18]. O(ndm=we) [GF08]. O(n log2(n)) [HM98]. O(n log3 m) [CH97b]. O(n log n)[Gef03].O(s2)[CZ01]. ! [BdFED+20]. !n [Cho78]. !T [BdFED+20]. q [HKN14, KPA10, STK06, Sal12, ST95, ST96b, ST04, Ukk92]. r [Pol13]. ρ [CFK07]. xmyn = zp [NC92]. -automaton [COZ09]. -bit [KVX12].
    [Show full text]
  • Code Similarity and Clone Search in Large-Scale Source Code Data
    Code Similarity and Clone Search in Large-Scale Source Code Data Chaiyong Ragkhitwetsagul A dissertation submitted in fulfillment of the requirements for the degree of Doctor of Philosophy of University College London. Department of Computer Science University College London September 30, 2018 Declarations I, Chaiyong Ragkhitwetsagul, confirm that the work presented in this thesis is my own. Where information has been derived from other sources, I confirm that this has been indicated in the thesis. The papers presented here are original work undertaken between September 2014 and July 2018 at University College London. They have been published or submitted for publication as listed below along with a summary of my contributions to the papers. 1. C. Ragkhitwetsagul, J. Krinke, D. Clark, Similarity of source code in the presence of pervasive modifications, in Proceedings of the 16th International Working Conference on Source Code Analysis and Manipulation (SCAM ’16), 2016. Contribution: Dr. Krinke designed and started the initial experiment which I later continued. I incorporated more tools, handled the tool executions, collected the results, analysed and discussed the results with Dr. Krinke. The three authors wrote the paper. 2. C. Ragkhitwetsagul, J. Krinke, D. Clark, A comparison of code similarity analysers, Empirical Software Engineering (EMSE), 2017. Contribution: This publication is an invited journal extension of the SCAM ’16 paper to a special issue of Empirical Software Engineering. I substantially expanded the experiment by doubling the data set size, rerunning the first and second experimental scenarios, and introducing three additional experimental scenarios under the supervision of Dr. Krinke. The paper has been written by the three authors.
    [Show full text]
  • Automated Discovery of Parameter Pollution Vulnerabilities in Web Applications
    Automated Discovery of Parameter Pollution Vulnerabilities in Web Applications Marco Balduzzi∗, Carmen Torrano Gimenez ‡, Davide Balzarotti∗, and Engin Kirda∗§ ∗ Institute Eurecom, Sophia Antipolis {balduzzi,balzarotti,kirda}@eurecom.fr ‡ Spanish National Research Council, Madrid [email protected] § Northeastern University, Boston [email protected] Abstract 1 Introduction In the last twenty years, web applications have grown In the last twenty years, web applications have grown from simple, static pages to complex, full-fledged dynamic from simple, static pages to complex, full-fledged dynamic applications. Typically, these applications are built using applications. Typically, these applications are built using heterogeneous technologies and consist of code that runs heterogeneous technologies and consist of code that runs both on the client and on the server. Even simple web ap- on the client (e.g., Javascript) and code that runs on the plications today may accept and process hundreds of dif- server (e.g., Java servlets). Even simple web applications ferent HTTP parameters to be able to provide users with today may accept and process hundreds of different HTTP interactive services. While injection vulnerabilities such as parameters to be able to provide users with rich, interactive SQL injection and cross-site scripting are well-known and services. As a result, dynamic web applications may con- have been intensively studied by the research community, a tain a wide range of input validation vulnerabilities such new class of injection vulnerabilities called HTTP Parame- as cross site scripting (e.g., [4, 5, 34]) and SQL injec- ter Pollution (HPP) has not received as much attention. If tion (e.g., [15, 17]).
    [Show full text]
  • Python Library Reference Release 2.1.1
    Python Library Reference Release 2.1.1 Guido van Rossum Fred L. Drake, Jr., editor July 20, 2001 PythonLabs E-mail: [email protected] Copyright c 2001 Python Software Foundation. All rights reserved. Copyright c 2000 BeOpen.com. All rights reserved. Copyright c 1995-2000 Corporation for National Research Initiatives. All rights reserved. Copyright c 1991-1995 Stichting Mathematisch Centrum. All rights reserved. See the end of this document for complete license and permissions information. Abstract Python is an extensible, interpreted, object-oriented programming language. It supports a wide range of applications, from simple text processing scripts to interactive WWW browsers. While the Python Reference Manual describes the exact syntax and semantics of the language, it does not describe the standard library that is distributed with the language, and which greatly enhances its immediate usability. This library contains built-in modules (written in C) that provide access to system functionality such as file I/O that would otherwise be inaccessible to Python programmers, as well as modules written in Python that provide standardized solutions for many problems that occur in everyday programming. Some of these modules are explicitly designed to encourage and enhance the portability of Python programs. This library reference manual documents Python’s standard library, as well as many optional library modules (which may or may not be available, depending on whether the underlying platform supports them and on the configuration choices made at compile time). It also documents the standard types of the language and its built-in functions and exceptions, many of which are not or incompletely documented in the Reference Manual.
    [Show full text]
  • A Comparison of Code Similarity Analysers
    Empir Software Eng https://doi.org/10.1007/s10664-017-9564-7 A comparison of code similarity analysers Chaiyong Ragkhitwetsagul1 · Jens Krinke1 · David Clark1 © The Author(s) 2017. This article is an open access publication Abstract Copying and pasting of source code is a common activity in software engineer- ing. Often, the code is not copied as it is and it may be modified for various purposes; e.g. refactoring, bug fixing, or even software plagiarism. These code modifications could affect the performance of code similarity analysers including code clone and plagiarism detectors to some certain degree. We are interested in two types of code modification in this study: pervasive modifications, i.e. transformations that may have a global effect, and local modifications, i.e. code changes that are contained in a single method or code block. We evaluate 30 code similarity detection techniques and tools using five experimental sce- narios for Java source code. These are (1) pervasively modified code, created with tools for source code and bytecode obfuscation, and boiler-plate code, (2) source code normalisation through compilation and decompilation using different decompilers, (3) reuse of optimal configurations over different data sets, (4) tool evaluation using ranked-based measures, and (5) local + global code modifications. Our experimental results show that in the presence of pervasive modifications, some of the general textual similarity measures can offer similar performance to specialised code similarity tools, whilst in the presence of boiler-plate code, highly specialised source code similarity detection techniques and tools outperform textual similarity measures. Our study strongly validates the use of compilation/decompilation as a normalisation technique.
    [Show full text]