UCLA UCLA Electronic Theses and Dissertations Title Leveraging Program Commonalities and Variations for Systematic Software Development and Maintenance Permalink https://escholarship.org/uc/item/25z5n0t6 Author Zhang, Tianyi Publication Date 2019 Peer reviewed|Thesis/dissertation eScholarship.org Powered by the California Digital Library University of California UNIVERSITY OF CALIFORNIA Los Angeles Leveraging Program Commonalities and Variations for Systematic Software Development and Maintenance A dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy in Computer Science by Tianyi Zhang 2019 c Copyright by Tianyi Zhang 2019 ABSTRACT OF THE DISSERTATION Leveraging Program Commonalities and Variations for Systematic Software Development and Maintenance by Tianyi Zhang Doctor of Philosophy in Computer Science University of California, Los Angeles, 2019 Professor Miryung Kim, Chair Software is becoming increasingly pervasive and complex. During software development and maintenance, developers often make ad hoc decisions based on local program contexts and their own experience only, which may increase technical debt and raise unknown conse- quences as software evolves. This dissertation explores several opportunities to guide software development and maintenance by exploiting and visualizing the commonalities and varia- tions among similar programs. Our hypothesis is that unveiling what has and has not been done in other similar program contexts can help developers make more systematic decisions, explore implementation alternatives, and reduce the risk of unknown consequences. The inherent repetitiveness in software systems provides a solid foundation for identify- ing and exploiting similar code. This dissertation first presents two approaches that leverage the syntactic similarity and repetitiveness in local codebases to improve code reviews and software testing. First, in contrast to inspecting local program edits line by line, Critics en- ables developers to reason about related edits scattered across multiple files by summarizing similar edits and detecting inconsistencies in similar locations. Second, Grafter boosts test coverage by transplanting test cases between similar programs and helps developers discern the behavioral similarity and differences between these programs via differential testing. This dissertation further extends the idea of systematic software development from local codebases to the open world, by exploiting the large and growing body of successful open- ii source projects on the Internet. In particular, we present three approaches that analyze a massive number of open-source projects and provide systematic guidance based on software reuse and adaptation patterns in the open-source community. First, ExampleCheck mines common API usage patterns from 380K GitHub projects and contrasts online code examples with these patterns to alert developers about potential API misuse during opportunistic code reuse. Second, to help developers comprehensively understand the variety of API usage at scale, Examplore aligns and super-imposes hundreds of code examples into a single code skeleton with statistical distributions of distinct API usage features among these examples. Finally, ExampleStack guides developers to adapt a code example to a target project by detecting similar code fragments in GitHub and by illuminating possible variations of this example with respect to its GitHub counterparts. The viability and usability of these techniques are evaluated by their application to large- scale projects as well as within-subjects user studies. First, in a user study with twelve Java developers, Critics helps developers identify 47% more edit mistakes with 32% less time than a line-level program diff tool. Second, Grafter transplants test cases in three open- source projects with 94% success rate and improves the statement and branch coverage by 22% and 16% respectively. Third, ExampleCheck reduces the risk of bug propagation by detecting API usage violations in almost one third of code examples in a popular Q&A forum, Stack Overflow. These API usage violations can potentially lead to software anomalies such as program crashes and resource leaks in target programs. Fourth, in a study with sixteen Java developers, Examplore helps developers understand the distribution of API usage patterns and answer common API usage questions accurately with concrete details. Finally, in another study with sixteen Java developers, ExampleStack helps developers identify more adaptation opportunities about code safety and logic customization, rather than shallow adaptations such as variable renaming. These key findings demonstrate that discovering and representing the commonalities and variations in similar contexts helps a developer achieve better program comprehension, discover more design alternatives, and capture more errors in different software development and maintenance tasks. iii The dissertation of Tianyi Zhang is approved. Todd D. Millstein Jens Palsberg Westley Weimer Miryung Kim, Committee Chair University of California, Los Angeles 2019 iv To my mom and dad v TABLE OF CONTENTS 1 Introduction :::::::::::::::::::::::::::::::::::::: 1 1.1 Thesis Statement . .3 1.2 Leveraging Inherent Repetitiveness in Software Systems . .5 1.2.1 Interactive Code Review for Similar Program Edits . .6 1.2.2 Automated Test Transplantation between Similar Programs . .7 1.3 Discovering Common Practices in Open Source Communities . .8 1.3.1 Mining Common API Usage Patterns from Massive Code Corpora . .9 1.3.2 Visualizing Common and Uncommon API Usage at Scale . 10 1.3.3 Adapting Code Examples with Similar GitHub Counterparts . 11 1.4 Contributions . 12 1.5 Outline . 13 2 Related Work ::::::::::::::::::::::::::::::::::::: 14 2.1 Software Repetitiveness and Code Clones . 14 2.1.1 Empirical Studies of the Presence and Evolution of Code Clones . 14 2.1.2 Existing Tool Support for Maintaining Code Clones . 16 2.2 Opportunistic Code Reuse . 20 2.2.1 Code Reuse from Local Codebases . 20 2.2.2 Code Reuse from the Web . 21 2.2.3 Tool Support for Searching and Integrating Code Snippets . 24 2.3 Mining and Visualizing API Usage . 26 2.4 Interfaces for Exploring Collections of Concrete Examples . 28 2.4.1 Exploring and Visualizing Multiple Code Instances . 28 vi 2.4.2 Exploring and Visualizing Other Complex Objects . 30 2.5 Modern Code Reviews and Program Change Comprehension . 31 2.6 Differential Testing . 32 2.7 Software Transplantation . 33 3 Interactive Code Review for Similar Program Edits ::::::::::::: 35 3.1 Introduction . 35 3.2 Motivating Example . 37 3.3 Change Template Construction and Refinement . 39 3.3.1 Program Context Extraction . 40 3.3.2 Change Template Customization . 40 3.4 Interactive Code Search and Anomaly Detection . 41 3.4.1 Tree-based Program Matching . 41 3.4.2 Change Summarization and Anomaly Detection . 42 3.5 Implementation and Tool Features . 44 3.6 Evaluation . 48 3.6.1 User Study with Professional Developers at Salesforce.com . 48 3.6.2 Lab Study: Comparison with Eclipse Diff and Search . 52 3.6.3 Comparison with LASE . 58 3.7 Threats to Validity . 61 3.8 Summary . 62 4 Automated Test Transplantation for Similar Programs ::::::::::: 63 4.1 Introduction . 63 4.2 Motivating Example . 65 4.3 Automated Code Transplantation and Differential Testing . 68 vii 4.3.1 Variation Identification . 68 4.3.2 Code Transplantation . 71 4.3.3 Data Propagation . 72 4.3.4 Fine-grained Differential Testing . 75 4.4 Tool Support and Demonstration . 75 4.5 Evaluation . 80 4.5.1 Grafting Capability . 81 4.5.2 Behavior Comparison Capability . 83 4.5.3 Fault Detection Robustness . 86 4.6 Threats to Validity . 89 4.7 Summary . 90 5 Mining Common API Usage Patterns from Massive Code Corpora ::: 91 5.1 Introduction . 91 5.2 Motivating Examples . 93 5.3 Scalable API Usage Mining on 380K GitHub Projects . 95 5.3.1 Structured Call Sequence Extraction and Slicing on GitHub . 96 5.3.2 Frequent Subsequence Mining . 99 5.3.3 Guard Condition Mining . 100 5.4 Evaluation of the API Usage Mining Framework . 102 5.4.1 Pattern Mining Accuracy . 102 5.4.2 Scalability . 105 5.5 A Study of API Misuse in Stack Overflow . 106 5.5.1 Data Collection . 106 5.5.2 Manual Inspection of Stack Overflow . 110 viii 5.5.3 Is API Misuse Prevalent on Stack Overflow? . 114 5.5.4 Are highly voted posts more reliable? . 115 5.5.5 What are the characteristics of API misuse? . 116 5.6 Augmenting Stack Overflow with API Usage Patterns mined from GitHub . 119 5.6.1 Tool Features and Implementations . 119 5.6.2 Demonstration Scenario . 123 5.7 Threats to Validity . 126 5.8 Summary . 127 6 Visualizing Common and Uncommon API Usage at Scale ::::::::: 128 6.1 Introduction . 128 6.2 Constructing Synthetic API Usage Skeleton . 131 6.3 Usage Scenario: Interacting with Code Distribution . 133 6.4 System Architecture and Implementation . 137 6.4.1 Data Collection . 137 6.4.2 Post-processing . 139 6.4.3 Visualization . 140 6.5 User Study . 141 6.5.1 API Methods . 141 6.5.2 Participants . 143 6.5.3 Methodology . 143 6.6 Results . 144 6.6.1 Quantitative Analysis . 144 6.6.2 Qualitative Analysis . 146 6.7 Threats to Validity . 149 ix 6.8 Conclusion . 150 7 Analyzing and Supporting Adaptation of Online Code Examples :::: 152 7.1 Introduction . 152 7.2 Data Collection: Linking Stack Overflow to GitHub . 155 7.3 Adaptation Type Analysis . 157 7.3.1 Manual Inspection and Adaptation Taxonomy . 157 7.3.2 Automated Adaptation Categorization . 161 7.3.3 Accuracy of Adaptation Categorization . 161 7.4 An Empirical Study of Common Adaptations of Stack Overflow Code Examples162 7.4.1 How many edits are potentially required to adapt a SO example? . 162 7.4.2 What are common adaptation and variation types? . 163 7.5 Tool Support and Implementation . 165 7.5.1 ExampleStack Tool Features . 165.