Dissertation Submitted in Partial Fulfillment of the Requirements for the Degree Of
Total Page:16
File Type:pdf, Size:1020Kb
©Copyright 2015 Brian Burg Understanding Dynamic Software Behavior with Tools for Retroactive Investigation Brian Burg A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy University of Washington 2015 Supervisory Committee: Michael D. Ernst, Chair Andrew J. Ko, Chair Steve Tanimoto James Fogarty Sean Munson Program Authorized to Offer Degree: Computer Science and Engineering University of Washington Abstract Understanding Dynamic Software Behavior with Tools for Retroactive Investigation Brian Burg Co-Chairs of the Supervisory Committee: Professor Michael D. Ernst Computer Science and Engineering Associate Professor Andrew J. Ko The Information School The web is a widely-available open application platform, where anyone can freely inspect a live program’s client-side source code and runtime state. Despite these platform ad- vantages, understanding and debugging dynamic behavior in web programs is still very challenging. Several barriers stand in the way of understanding dynamic behaviors: re- producing complex interactions is often impossible; finding and comparing a behavior’s runtime states is time-consuming; and the code that implements a behavior is scattered across multiple DOM, CSS, and JavaScript files. This dissertation demonstrates that these barriers can be addressed by new program understanding tools that rely on the ability to capture a program execution and revisit past program states within it. We show that when integrated as part of a browser engine, deterministic replay is fast, transparent, and pervasive; and these properties make it a suitable platform for such program understanding tools. This claim is substantiated by several novel interfaces for understanding dynamic behaviors. These prototypes exem- plify three strategies for navigating through captured program executions: (1) by visual- izing and seeking to input events—such as user interactions, network callbacks, and asyn- chronous tasks; (2) by retroactively logging program states and reverting execution back to log-producing statements; and (3) by working backwards from differences in visual output to the source code responsible for inducing output-affecting state changes. Some of these capabilities have been incorporated into the WebKit browser engine, demonstrat- ing their practicality. ACKNOWLEDGMENTS I would like to thank Jan Vitek and Gregor Richards for taking a chance on me back when I was a sarcastic, inexperienced undergraduate at Purdue University. Without their enthusiasm for research, danger, and hacking, it’s unlikely I would have chosen such a fun dissertation topic! A huge thanks to Filip Pizlo, Timothy Hatcher, and others who convinced me to contribute my replay prototypes back to the WebKit project; the experi- ence has improved this work significantly, and may yet lead to big real-world impacts. My studies at the University of Washington would have been short-lived if not for the patient, firm guidance of my co-advisors. Mike taught me the fundamentals of doing research back when I had big ideas and few concrete plans. Andy sold me on the vision of HCI and software engineering when I was at my most cynical, and helped to channel my energy and ideas into the compelling set of projects contained in this dissertation. I will miss both Andy and Mike’s honest and open feedback—red marks and all. Many other people have helped to shape my research in one way or another. My lab buddies—Ben, Todd, Colin, Sai, Kivanc, Ivan—kept me company in our windowless 3rd floor research den. I was fortunate to supervise the UI-building efforts of two won- derful undergraduate research assistants, Katie Madonna and Jake Bailey. Discussions with external researchers—Greg Wilson, Chris Parnin, Thomas LaToza, Adrian Kuhn, Joel Brandt, David Herman, Patrick Walton, Lars Bergstrom, Wolfram Schulte, among many others—helped me fine-tune my ideas and see the broader impacts of my research agenda. Outside of academia, my skill as a researcher and hacker leveled up each time I interned: once at Microsoft Research, twice at Mozilla Research, and once Apple Inc., where I will continue this line of work after graduation. iii Lastly, I thank my friends and family for their support, friendship, and love during this long uphill climb. On campus, Katie, Conrad, Karl, Sonya, Katelin, Eric, Lillian, Nicola, Nick, Mark, Matt, Brandon(s), Will, Vincent, Ricardo, Igor, Adrian, Lindsay, Elise, Rebecca, Tracy, and many others have brightened my day countless times. At home, my wife Steph is my co-conspirator, the roof over my head, the floor I stand on, and the one who presses me to keep my hopes up and my cynic down. Thanks to my family for cheer- ing me on and always supporting me regardless of the direction I venture. Brian Burg Mariposa, California June 26, 2015 TABLE OF CONTENTS Page Abstract............................................. iii Acknowledgments...................................... iv Table of Contents....................................... ix List of Figures.........................................x List of Tables.......................................... xii Chapter 1: Introduction..................................1 1.1 The Problem.....................................2 1.2 Addressing the Problem..............................3 1.3 Definitions......................................4 1.4 Contributions....................................6 1.5 Outline........................................7 Chapter 2: Related Work.................................9 2.1 Capturing Executions................................9 2.1.1 Deterministic Replay............................9 2.1.2 Post-mortem Approaches......................... 11 2.1.3 Navigating Captured Executions..................... 13 2.2 Extracting Program States............................. 14 2.2.1 Specifying Instrumentation........................ 14 2.2.2 Inserting Instrumentation......................... 15 2.2.3 Composing and Scoping Instrumentation................ 17 2.2.4 Decoupling Execution and Instrumentation............... 18 2.3 Designing Developer Tools............................ 18 v 2.3.1 Cognitive Models of Comprehension and Tool Use.......... 19 2.3.2 Information Needs and Developers’ Questions............. 19 2.4 Understanding Dynamic Behavior........................ 20 2.4.1 Visualizing Dynamic Behaviors...................... 21 2.4.2 Visualizing and Exploring Recordings and Traces........... 22 2.4.3 Supporting Behavior Dissemination................... 23 2.5 Feature Location................................... 24 2.5.1 Locating Features using Visual Output................. 24 2.5.2 Explaining How Interactive Behaviors Work.............. 24 Chapter 3: Deterministic Replay for Web Programs.................. 26 3.1 Background..................................... 27 3.1.1 Web Programs................................ 27 3.1.2 Rendering Engines............................. 29 3.1.3 Features, Ports, and Platforms...................... 29 3.1.4 Browser Architecture............................ 30 3.2 Design........................................ 30 3.2.1 Types of Nondeterminism......................... 31 3.2.2 Intercession Mechanisms......................... 34 3.2.3 Recording and Input Structure...................... 35 3.3 Implementation................................... 35 3.3.1 Capturing and Replaying Executions.................. 36 3.3.2 External Nondeterminism......................... 37 3.3.3 Internal Nondeterminism......................... 39 3.4 Evaluation...................................... 40 3.4.1 Fidelity.................................... 40 3.4.2 Performance................................. 41 3.4.3 Scalability of the Approach........................ 42 3.4.4 Limitations................................. 43 3.5 Summary....................................... 45 Chapter 4: Replay Extensions and Applications.................... 52 4.1 Navigating to Program States........................... 53 vi 4.1.1 Indexing Executed Statements...................... 53 4.1.2 Debugger Bookmarks........................... 54 4.2 Extracting Program States............................. 55 4.2.1 Scanning Algorithms............................ 55 4.2.2 Virtualizing State Extraction........................ 56 4.3 Mitigating Replay Faults.............................. 58 4.4 Summary....................................... 61 Chapter 5: Interfaces for Navigating Executions.................... 62 5.1 Basic Replay Controls................................ 63 5.2 Navigating via Inputs............................... 65 5.2.1 Interface Design............................... 65 5.2.2 Example................................... 67 5.2.3 Debugger Bookmarks........................... 69 5.2.4 Breakpoint Radar.............................. 70 5.2.5 Tool Integration............................... 72 5.3 Navigating via Runtime States.......................... 73 5.3.1 Interface................................... 75 5.3.2 Example................................... 76 5.4 Summary....................................... 79 Chapter 6: Explaining Visual Changes in Web Interfaces............... 80 6.1 Motivation...................................... 80 6.2 Example....................................... 82 6.3 Interface Design................................... 86 6.3.1 Design Rationale.............................. 86 6.3.2 Tracking an Element............................ 89 6.3.3 Relating Outputs and States........................ 90 6.3.4 Comparing Internal States........................