My View the Science of Debugging

4/12/2012

Claim: You probably suck at debugging

• Basis of claim: Debugging Years of helping students debug Art, Science: Neither, Either or Both? • It’s not entirely your fault Perry Kivolowitz Few teach how to debug University of Wisconsin –Madison Some say it is an art and cannot be taught Computer Sciences Department

4/7/2012 Perry Kivolowitz 2

My view The Science of Debugging

• Certainly there is an art to debugging 1. The Scientific Method • Mastery matures over years of experience 4. Kivolowitz’s Corollary 5. Kivolowitz’s Maxim 1 6. Maxwell Cohen’s Law • But the art is based on science 7. Kivolowitz’s Maxim 2 – Understanding the science can accelerate 8. Kivolowitz’s Maxim 3 the growth of your artistry 9. Conan Doyle’s Law – Science can be taught

4/7/2012 Perry Kivolowitz 3 4/7/2012 Perry Kivolowitz 4

Case study from a CS class* Case study from a CS class

• (Single threaded) code ran well on two • No, it was theirs different machines • Crashed on a third • In fact: • Third machine got OS patches – Symptoms tell you the type of bug it likely is • Bug went away – Symptoms explain why the bug went into hiding • Students asked: Was it an OS bug? (Heisenbug)

(Note to self: Give other examples of Heisenbugs) * All scenarios in this talk happened … more or less

4/7/2012 Perry Kivolowitz 5 4/7/2012 Perry Kivolowitz 6

1 4/12/2012

Case study from a CS class Common Debugging Algorithm

• The bug is likely a pointer problem Program doesn’t – Using an uninitialized pointer work – Using a pointer after what it pointed to was freed – Using a pointer clobbered by something else • Why did an OS patch “fix” the bug? Make change – Because memory shifted around – The bogus pointer now points to a less “crash inducing” value (or the clobbered location…) Does program • The bug is still there, waiting… work?

4/7/2012 Perry Kivolowitz 7 4/7/2012 Perry Kivolowitz 8

Principles 1 through 3 Kivolowitz Corollary

• Missing from the preceding is application of the Scientific Method A fix is not a fix until you completely Observe / Formulate Test hypothesis Gather Data hypothesis understand why • Debugging should not be knee‐jerk it is a fix

4/7/2012 Perry Kivolowitz 9 4/7/2012 Perry Kivolowitz 10

What else is wrong with this? Kivolowitz Maxim #1 If you make a change Else, you are chasing a Program doesn’t work with no beneficial moving target result –back it out unless you are certain you are fixing a Make change different bug

Does Debugging is about the elimination of program work? unknowns, not their introduction.

4/7/2012 Perry Kivolowitz 11 4/7/2012 Perry Kivolowitz 12

2 4/12/2012

Maxwell (Mickey) Cohen’s Law Kivolowitz Maxim #2 Student: It worked yesterday. Then I wrote “Where there is one, X00 lines of code and now it doesn’t work. Can you help me? there are likely many” Master: Sigh. • Always be mindful that you may be seeing a Write in small units. cascade of errors • Revisited during “defensive programming” Test in small units. (Note to self: Even “personal” version control is useful) and “call stack” discussion (Note to self: Describe how debugging is different from looking for your phone)

4/7/2012 Perry Kivolowitz 13 4/7/2012 Perry Kivolowitz 14

Kivolowitz’s Maxim #3 Kivolowitz’s Maxim #3 Example

• Bugs want to be found Student: I get “Bus error. Core dumped.” • They typically announce themselves clearly if Can you help me? you make the effort to listen Master: What do you think the problem is? Student: How should I know? That’s all it says. Master: Sigh.

Observe / Formulate Test hypothesis Gather Data hypothesis

The Guru Meditation from arguably the most bug‐free non‐trivial OS in history. This contains a great deal of information.

4/7/2012 Perry Kivolowitz 15 4/7/2012 Perry Kivolowitz 16

Conan Doyle’s Law Defensive programming “When you have eliminated the • Cohen’s law and Conan Doyle’s law segue into defensive programming impossible, whatever remains, • Both suggest the use of however improbable, must be • If you “know” a condition to be true, assert it! the truth.” • Stops a cascade in its tracks • Helps to eliminate the impossible and identify the improbable The Sign of Four, 1890

4/7/2012 Perry Kivolowitz 17 4/7/2012 Perry Kivolowitz 18

3 4/12/2012

Defensive programming Defensive programming

• Code abstraction can be defensive programming • Comments • Code is more easily debugged when you… – Don’t just comment the code – Code in one place – Comment the thought process – Test in one place • As you write down your thoughts you get clarity – Fix in one place • Will you remember what you were thinking a year from • Aside: leverage STL, Boost, glm etc. now? Next week? Tomorrow? – Templates (“generics”) in general • Aside: code the “default” case and the impossible What about someone else reading your code?

“else” (Note to self: Mention value of writing the comments EVEN BEFORE writing the code!) (Note to self: Emphasize “fix in one place” with respect to cut and pasted code) (Note to self: “Improbable else” – testing floats and doubles – remember CS 252, CS 354 and CS 412)

4/7/2012 Perry Kivolowitz 19 4/7/2012 Perry Kivolowitz 20

Defensive programming Defensive programming

Student: I commented like you told me. Student: My code doesn’t work. // Increment j Can you help? Master: Sigh. Master: What does variable ineedsleep do? Student: I don’t remember, I was tired. Comment your thought process not minutia. Master: Sigh.

descriptive_variable_names_really_help Keystrokes are cheap. Debugging time isn’t. (Note to self: Explain … keystrokes are guaranteed to end, debugging hours are not.)

4/7/2012 Perry Kivolowitz 21 4/7/2012 Perry Kivolowitz 22

Role of testing Role of testing

Testing can only prove the presence • If you are fortunate to have a QA group of bugs, not their absence. – It is not their job to find bugs Edsger W. Dijkstra – That is your job Beware of bugs in the above code; I have – It is their job to confirm bugs you found are gone only proved it correct, not tried it. Donald Knuth • Don’t “throw your code over a wall” Beware of bugs in the above code; I have only written it, not tried it

4/7/2012 Perry Kivolowitz 23 4/7/2012 Perry Kivolowitz 24

4 4/12/2012