Automated Program Transformation for Improving Software Quality Rijnard van Tonder CMU-ISR-19-101 October 2019 Institute for Software Research School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 Thesis Committee: Claire Le Goues, Chair Christian Kästner Jan Hoffmann Manuel Fähndrich, Facebook, Inc. Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Software Engineering. Copyright 2019 Rijnard van Tonder This work is partially supported under National Science Foundation grant numbers CCF-1750116 and CCF-1563797, and a Facebook Testing and Verification research award. The views and conclusions contained in this document are those of the author and should not be interpreted as representing the official policies, either expressed or implied, of any sponsoring corporation, institution, the U.S. government, or any other entity. Keywords: syntax, transformation, parsers, rewriting, crash bucketing, fuzzing, bug triage, program transformation, automated bug fixing, automated program repair, separation logic, static analysis, program analysis Abstract Software bugs are not going away. Millions of dollars and thousands of developer-hours are spent finding bugs, debugging the root cause, writing a patch, and reviewing fixes. Automated techniques like static analysis and dynamic fuzz testing have a proven track record for cutting costs and improving software quality. More recently, advances in automated program repair have matured and see nascent adoption in industry. Despite the value of these approaches, automated techniques do not come for free: they must approximate, both theoretically and in the interest of practicality. For example, static analyzers suffer false positives, and automatically produced patches may be insufficiently precise to fix a bug. Such limitations continue to impose substantial human effort amid the benefits of automation. Software development activities revolve around changing code. Thus, per- forming and reasoning about program change has extensive bearing on the effectiveness of automated techniques. From this perspective, we develop new automated techniques for changing programs to improve analysis behavior, and, correspondingly, use automated reasoning and analysis to specialize program changes for automated program repair. We present the first evidence that automated program transformation, program analysis, and program repair are interrelated and cooperative. We first show that automated program transfor- mation leads to higher quality static analysis (by reducing false positives) and dynamic fuzz testing (by reducing duplicate bug reports). We then show how high-quality static analyses can feed into and enable automated program repair, and how automated repair can circle back to further improve static analysis (e.g., by revealing more true positive bugs). The thesis is that automated syn- tactic and semantic search and application of program transformations enables efficient, scalable, and unassisted techniques for improving the effectiveness of existing program analyses and end-to-end repair of real-world programs. We show that these techniques are effective compared to current approaches in the respective domains of static analysis, dynamic fuzz testing, and program repair. We demonstrate relevance and real-world applicability by evaluating on large, popular, and active projects across multiple languages. Our vision for this work is that new capabilities and techniques for automated program transformation foster effective ways to automate burdensome human effort and reasoning incurred by limitations in program analysis and repair. iv Acknowledgments I’m incredibly grateful to my advisor, Claire, for taking me as her student. My first impression of Claire was that she would make a good advisor. Over time I realized that she’s exceptional. I admire the focus and tact with which she’s helped steer my work; always through keen questions and candid feedback. She’s been my advocate and collaborator, and a valuable source of optimism. Our interactions had a special sense of levity that I’ll always remember. I thank my committee members, Christian, Jan, and Manuel. Their engage- ment was excellent and I greatly value their time and feedback. Their input has raised the quality of this dissertation. I want to acknowledge the friendship and support of the following people during my studies: Arné, Nelius, Robert, Gerdus, Dominic, Nam, and Ivan. Then also those in the research group: Chris, Zack, Deby, Mau, Afsoon, Cody, and Jeremy. I thank my family for their unceasing encouragement, prayers, and love: my father, Gerhard, my mother, Riana, and my two sisters, Monique and Karli. I’m especially grateful for the decisions and sacrifices that my parents made years and years ago—I realize now that those choices made it possible for me to accomplish what I have today. None of what I accomplish matters without acknowledging Jesus Christ, who grants me all that I have. “Now, for you: So what? The question I would leave you with is: Have you ever, as an adult, faced up to the question of whether the claims of Jesus Christ are true? And that’s the most important question I can ask you.” — Fred Brooks, “Last Blast”, 2015. vi Contents List of Terms xv Preface 1 1 Introduction3 1.1 Scope and Success Criteria...........................7 1.1.1 Lightweight Declarative Syntax Transformation...........7 1.1.2 Tailoring Programs for Static Analysis................8 1.1.3 Tailoring Programs under Dynamic Fuzz Testing...........9 1.1.4 Augmenting Static Analysis for Automated Program Repair.... 10 1.2 Contributions.................................. 11 1.3 Outline...................................... 13 2 Lightweight Declarative Syntax Transformation 15 2.1 Introduction................................... 16 2.2 Motivating Example.............................. 19 2.3 Dyck-Extended Languages........................... 23 2.4 The Rewrite Process.............................. 26 2.4.1 Declarative Matching: The Intuition................. 26 2.4.2 Parser Parser Combinators for Matching............... 29 2.4.3 Match Rules............................... 33 vii 2.4.4 The Rewrite Phase........................... 34 2.5 Evaluation.................................... 35 2.5.1 Real, Large Scale Multi-Language Rewriting............. 37 2.5.2 Comparison with Existing Tools.................... 46 2.5.3 Discussion................................ 49 2.6 Related Work.................................. 51 2.7 Summary.................................... 53 3 Tailoring Programs for Static Analysis 55 3.1 Introduction................................... 55 3.2 Motivation.................................... 59 3.3 Approach.................................... 62 3.3.1 Match Template Behavior....................... 63 3.3.2 Tailoring Programs for Analysis.................... 63 3.4 Evaluation.................................... 67 3.4.1 Experimental Setup........................... 69 3.4.2 Experimental Results.......................... 71 3.4.3 Discussion................................ 82 3.5 Related Work.................................. 84 3.6 Summary.................................... 86 4 Semantic Crash Bucketing 87 4.1 Introduction................................... 87 4.2 Motivating Example.............................. 91 4.3 Semantic Crash Bucketing........................... 94 4.3.1 Problem Formulation.......................... 95 4.3.2 Detecting Duplicates.......................... 97 4.3.3 Semantic Crash Bucketing Procedure................. 98 viii 4.4 Generating Approximate Fixes......................... 100 4.4.1 Tb Production............................... 100 4.4.2 Null Dereferences............................ 101 4.4.3 Buffer Overflows............................ 103 4.5 Experimental Design.............................. 105 4.5.1 Bugs with Ground Truth........................ 106 4.5.2 Crash Corpus Generation....................... 108 4.5.3 Evaluating Fuzzers........................... 108 4.6 Experimental Results.............................. 111 4.6.1 Overall Results............................. 111 4.6.2 Project-specific Results......................... 113 4.7 Discussion.................................... 114 4.8 Related Work.................................. 116 4.9 Summary.................................... 118 5 End-to-End Static Automated Program Repair 119 5.1 Introduction................................... 120 5.2 Preliminaries.................................. 125 5.2.1 Program Model and Assertion Language............... 126 5.2.2 Frame Inference............................. 128 5.2.3 Finding Bugs Using Separation Logic................. 129 5.3 Repair with Separation Logic......................... 130 5.3.1 Formulating Repair........................... 131 5.3.2 Searching for Repairs.......................... 133 5.3.3 Applying Repairs: from Logic to Programs.............. 136 5.4 Evaluation.................................... 140 5.4.1 Setup.................................. 141 ix 5.4.2 Repair Results............................. 144 5.4.3 Patch Quality.............................. 145 5.4.4 Fixing by Semantic Effects....................... 147 5.5 Limitations and Discussion........................... 149 5.6 Related Work.................................. 150 5.7 Summary.................................... 152 6 Conclusion and Future Work 153 6.1 Summary.................................... 153 6.2 Open Questions and Research Pursuits.................... 155 6.2.1 Extending Automated