Software Debugging: Past, Present, and Future Alessandro (Alex) Orso School of Computer Science – College of Computing Georgia Institute of Technology http://www.cc.gatech.edu/~orso/ Partially supported by: NSF, Google, IBM, and MSR Automated Debugging Past Present Future Past A Short History of Debugging not so The Birth of Debugging ??? First reference to software errors Your guess? 2017 The Birth of Debugging 1840 1843 • Software errors mentioned in Ada Byron's notes on Charles Babbage's analytical engine • Several uses of the term bug to indicate defects in computers and software • First actual bug and actual debugging (Admiral Grace Hopper’s associates working on Mark II Computer at Harvard University) 2017 The Birth of Debugging 1840 ... • Software errors mentioned in Ada Byron's notes on Charles Babbage's analytical engine 1940 • Several uses of the term bug to indicate defects in computers and software • First actual bug and actual debugging (Admiral Grace Hopper’s associates working on Mark II Computer at Harvard University) 2017 The Birth of Debugging 1840 • Software errors mentioned in Ada Byron's notes on Charles Babbage's analytical engine • Several uses of the term bug to indicate defects in computers and software 1947 • First actual bug and actual debugging (Admiral Grace Hopper’s associates working on Mark II Computer at Harvard University) 2017 The Birth of Debugging 1840 • Software errors mentioned in Ada Byron's notes on Charles Babbage's analytical engine • Several uses of the term bug to indicate defects in computers and software 1947 • First actual bug and actual debugging (Admiral Grace Hopper’s associates working on Mark II Computer at Harvard University) 2017 Symbolic Debugging 1840 • UNIVAC 1100’s FLIT (Fault Location by Interpretive Testing) • Richard Stallman’s GDB DDD 1962 • • ... 2017 Symbolic Debugging 1840 • UNIVAC 1100’s FLIT (Fault Location by Interpretive Testing) • GDB • DDD 1986 • ... 2017 Symbolic Debugging 1840 • UNIVAC 1100’s FLIT (Fault Location by Interpretive Testing) • GDB • DDD 1996 • ... 2017 Symbolic Debugging 1840 • UNIVAC 1100’s FLIT (Fault Location by Interpretive Testing) • GDB • DDD 1996 • ... 2017 Program Slicing 1960 • Intuition: developers “slice” backwards when debugging • Weiser’s breakthrough paper • Korel and Laski’s dynamic slicing 1981 • Agrawal • Ko’s Whyline 2017 Program Slicing 1960 • Intuition: developers “slice” backwards when debugging • Weiser’s breakthrough paper • Korel and Laski’s dynamic slicing 1981 • Agrawal • Ko’s Whyline 2017 Static Slicing Example mid() { int x,y,z,m; 1: read(“Enter 3 numbers:”,x,y,z); 2: m = z; 3: if (y<z) 4: if (x<y) 5: m = y; 6: else if (x<z) 7: m = y; // bug 8: else 9: if (x>y) 10: m = y; 11: else if (x>z) 12: m = x; 13: print(“Middle number is:”, m); } Program Slicing 1960 • Intuition: developers “slice” backwards when debugging • Weiser’s breakthrough paper • Korel and Laski’s dynamic slicing 1981 • Agrawal • Ko’s Whyline 2017 Program Slicing 1960 • Intuition: developers “slice” backwards when debugging • Weiser’s breakthrough paper • Korel and Laski’s dynamic slicing Agrawal 1988 • 1993 • Ko’s Whyline 2017 Dynamic Slicing Example mid() { int x,y,z,m; 1: read(“Enter 3 numbers:”,x,y,z); 2: m = z; 3: if (y<z) 4: if (x<y) 5: m = y; 6: else if (x<z) 7: m = y; // bug 8: else 9: if (x>y) 10: m = y; 11: else if (x>z) 12: m = x; 13: print(“Middle number is:”, m); } Dynamic Slicing Example Test Cases mid() { int x,y,z,m; 3,3,5 1,2,3 3,2,1 5,5,5 5,3,4 2,1,3 1: read(“Enter 3 numbers:”,x,y,z); • • • • • • 2: m = z; • • • • • • 3: if (y<z) • • • • • • 4: if (x<y) • • • • 5: m = y; • 6: else if (x<z) • • • 7: m = y; // bug • • 8: else • • 9: if (x>y) • • 10: m = y; • 11: else if (x>z) • 12: m = x; 13: print(“Middle number is:”, m); • • • • • • } Pass/Fail P P P P P F Dynamic Slicing Example Test Cases mid() { int x,y,z,m; 3,3,5 1,2,3 3,2,1 5,5,5 5,3,4 2,1,3 1: read(“Enter 3 numbers:”,x,y,z); • • • • • • 2: m = z; • • • • • • 3: if (y<z) • • • • • • 4: if (x<y) • • • • 5: m = y; • 6: else if (x<z) • • • 7: m = y; // bug • • 8: else • • 9: if (x>y) • • 10: m = y; • 11: else if (x>z) • 12: m = x; 13: print(“Middle number is:”, m); • • • • • • } Pass/Fail P P P P P F Program Slicing 1960 • Intuition: developers “slice” backwards when debugging • Weiser’s breakthrough paper • Korel and Laski’s dynamic slicing Agrawal 1988 • 1993 • Ko’s Whyline 2017 Program Slicing 1960 why is this • Intuition: developers “slice” backwardsstroke black? when debugging Weiser’s breakthrough paper why• didn’t this •colorKorel panel and Laski’s dynamic slicing change? • Agrawal Ko’s Whyline 2008 • 2017 Program Slicing 1960 why is this • Intuition: developers “slice” backwardsstroke black? when debugging Weiser’s breakthrough paper why• didn’t this •colorKorel panel and Laski’s dynamic slicing change? • Agrawal Ko’s Whyline 2008 • 2017 Program Slicing 1960 : developers “slice” backwards • Intuition why is this when debugging stroke black? • Weiser’s breakthrough paper why didn’tKorel this and Laski’s dynamic slicing color• panel change?• Agrawal Ko’s Whyline 2008 • 2017 Delta Debugging 1960 • Intuition: it’s all about differences! • Isolates failure causes automatically • Zeller’s “Yesterday, My Program Worked. Today, It Does Not. Why?” 1999 2017 Delta Debugging 1960 • Intuition: it’s all about differences! • Isolates failure causes automatically • Zeller’s “Yesterday, My Program Worked. Today, It Does Not. Why?” • Applied in several contexts 1999 2017 Today ✘ ✔ ✔ Yesterday Today ✘ ✘ ✔ ✔ Yesterday Today ✘ ✘ ✘ ✔ ✔ Yesterday Today ✘ ✘ ✘ ✔ ✔ Yesterday Today ✘ ✘ ✘ … ✘ ✔ … Failure cause ✔ ✔ Yesterday Today ✘ ✘ ✘ … ✘ ✔ … Failure cause ✔ ✔ Yesterday Applied to programs, inputs, states, ... Statistical Debugging 1960 • Intuition: debugging techniques can leverage multiple executions • Tarantula • Liblit’s CBI 2001 • Many others! 2017 Statistical Debugging 1960 • Intuition: debugging techniques can leverage multiple executions • Tarantula • Liblit’s CBI 2001 • Many others! 2017 Tarantula Test Cases mid() { suspiciousness int x,y,z,m; 3,3,5 1,2,3 3,2,1 5,5,5 5,3,4 2,1,3 1: read(“Enter 3 numbers:”,x,y,z); • • • • • • 0.5 2: m = z; • • • • • • 3: if (y<z) • • • • • • 4: if (x<y) • • • • 5: m = y; • 6: else if (x<z) • • • • • 7: m = y; // bug • • 8: else • • 9: if (x>y) • 10: m = y; • 11: else if (x>z) 12: m = x; • • • • • • 13: print(“Middle number is:”, m); Pass/Fail P P P P P F } Tarantula Test Cases mid() { suspiciousness int x,y,z,m; 3,3,5 1,2,3 3,2,1 5,5,5 5,3,4 2,1,3 1: read(“Enter 3 numbers:”,x,y,z); • • • • • • 0.5 2: m = z; • • • • • • 0.5 3: if (y<z) • • • • • • 0.5 4: if (x<y) • • • • 0.6 5: m = y; • 0.0 6: else if (x<z) • • • 0.7 • • 7: m = y; // bug 0.8 • • 8: else 0.0 • • 9: if (x>y) 0.0 • 0.0 10: m = y; • 0.0 11: else if (x>z) 0.0 12: m = x; • • • • • • 0.5 13: print(“Middle number is:”, m); Pass/Fail P P P P P F } Tarantula Test Cases mid() { suspiciousness int x,y,z,m; 3,3,5 1,2,3 3,2,1 5,5,5 5,3,4 2,1,3 1: read(“Enter 3 numbers:”,x,y,z); • • • • • • 0.5 2: m = z; • • • • • • 0.5 3: if (y<z) • • • • • • 0.5 4: if (x<y) • • • • 0.6 5: m = y; • 0.0 6: else if (x<z) • • • 0.7 • • 7: m = y; // bug 0.8 • • 8: else 0.0 • • 9: if (x>y) 0.0 • 0.0 10: m = y; • 0.0 11: else if (x>z) 0.0 12: m = x; • • • • • • 0.5 13: print(“Middle number is:”, m); Pass/Fail P P P P P F } Statistical Debugging 1960 • Intuition: debugging techniques can leverage multiple executions • Tarantula • Liblit’s CBI 2001 • Many others! 2017 Statistical Debugging 1960 • Intuition: debugging techniques can leverage multiple executions • Tarantula • CBI 2003 • Many others! 2017 Statistical Debugging 1960 • Intuition: debugging techniques can leverage multiple executions • Tarantula • CBI • Ochiai 2006 • Many others! 2017 Statistical Debugging 1960 • Intuition: debugging techniques can leverage multiple executions • Tarantula • CBI • Ochiai • Causal inference based 2010 • Many others! 2017 Statistical Debugging 1960 • Intuition: debugging techniques can leverage multiple executions • Tarantula • CBI • Ochiai • Causal inference based 2008 • IR-based techniques 2017 IR-Based Techniques Bug ID: 90018 Summary: Native tooltips left around on CTabFolder. Description: Hover over the PartStack CTabFolder inside eclipse until some native tooltip is displayed. For example, the maximize button. When the tooltip appears, change perspectives using the keybinding. the CTabFolder gets hidden, but its tooltip is permanently displayed and never goes away. Even if that CTabFolder is disposed (I'm assuming) when the perspective is closed. -------------------------------------------------------------------------- IR-Based Techniques Source code file: CTabFolder.java public class CTabFolder extends Composite { // tooltip int [] toolTipEvents = new int[] {SWT.MouseExit, SWT.MouseHover, SWT.MouseMove, Bug ID: 90018 SWT.MouseDown, SWT.DragDetect}; Summary: Native tooltips left around on Listener toolTipListener; CTabFolder. … Description: Hover over the PartStack / * Returns <code>true</code> if the CTabFolder CTabFolder inside eclipse until some native only displys the selected tab tooltip is displayed. For example, the * and <code>false</code> if the CTabFolder maximize button. When the tooltip appears, displays multiple tabs. change perspectives using the keybinding. */ the CTabFolder gets hidden, but its tooltip is …void onMouseHover(Event event) { permanently displayed and never goes away. showToolTip(event.x, event.y); Even if that CTabFolder is disposed (I'm } assuming) when the perspective is closed.