4/12/2012 Claim: You probably suck at debugging • Basis of claim: Debugging Years of helping students debug Art, Science: Neither, Either or Both? • It’s not entirely your fault Perry Kivolowitz Few teach how to debug University of Wisconsin –Madison Some say it is an art and cannot be taught Computer Sciences Department 4/7/2012 Perry Kivolowitz 2 My view The Science of Debugging • Certainly there is an art to debugging 1. The Scientific Method • Mastery matures over years of experience 4. Kivolowitz’s Corollary 5. Kivolowitz’s Maxim 1 6. Maxwell Cohen’s Law • But the art is based on science 7. Kivolowitz’s Maxim 2 – Understanding the science can accelerate 8. Kivolowitz’s Maxim 3 the growth of your artistry 9. Conan Doyle’s Law – Science can be taught 4/7/2012 Perry Kivolowitz 3 4/7/2012 Perry Kivolowitz 4 Case study from a CS class* Case study from a CS class • (Single threaded) code ran well on two • No, it was theirs different machines • Crashed on a third • In fact: • Third machine got OS patches – Symptoms tell you the type of bug it likely is • Bug went away – Symptoms explain why the bug went into hiding • Students asked: Was it an OS bug? (Heisenbug) (Note to self: Give other examples of Heisenbugs) * All scenarios in this talk happened … more or less 4/7/2012 Perry Kivolowitz 5 4/7/2012 Perry Kivolowitz 6 1 4/12/2012 Case study from a CS class Common Debugging Algorithm • The bug is likely a pointer problem Program doesn’t – Using an uninitialized pointer work – Using a pointer after what it pointed to was freed – Using a pointer clobbered by something else • Why did an OS patch “fix” the bug? Make change – Because memory shifted around – The bogus pointer now points to a less “crash inducing” value (or the clobbered location…) Does program • The bug is still there, waiting… work? 4/7/2012 Perry Kivolowitz 7 4/7/2012 Perry Kivolowitz 8 Principles 1 through 3 Kivolowitz Corollary • Missing from the preceding is application of the Scientific Method A fix is not a fix until you completely Observe / Formulate Test hypothesis Gather Data hypothesis understand why • Debugging should not be knee‐jerk it is a fix 4/7/2012 Perry Kivolowitz 9 4/7/2012 Perry Kivolowitz 10 What else is wrong with this? Kivolowitz Maxim #1 If you make a change Else, you are chasing a Program doesn’t work with no beneficial moving target result –back it out unless you are certain you are fixing a Make change different bug Does Debugging is about the elimination of program work? unknowns, not their introduction. 4/7/2012 Perry Kivolowitz 11 4/7/2012 Perry Kivolowitz 12 2 4/12/2012 Maxwell (Mickey) Cohen’s Law Kivolowitz Maxim #2 Student: It worked yesterday. Then I wrote “Where there is one, X00 lines of code and now it doesn’t work. Can you help me? there are likely many” Master: Sigh. • Always be mindful that you may be seeing a Write in small units. cascade of errors • Revisited during “defensive programming” Test in small units. (Note to self: Even “personal” version control is useful) and “call stack” discussion (Note to self: Describe how debugging is different from looking for your phone) 4/7/2012 Perry Kivolowitz 13 4/7/2012 Perry Kivolowitz 14 Kivolowitz’s Maxim #3 Kivolowitz’s Maxim #3 Example • Bugs want to be found Student: I get “Bus error. Core dumped.” • They typically announce themselves clearly if Can you help me? you make the effort to listen Master: What do you think the problem is? Student: How should I know? That’s all it says. Master: Sigh. Observe / Formulate Test hypothesis Gather Data hypothesis The Guru Meditation from arguably the most bug‐free non‐trivial OS in history. This contains a great deal of information. 4/7/2012 Perry Kivolowitz 15 4/7/2012 Perry Kivolowitz 16 Conan Doyle’s Law Defensive programming “When you have eliminated the • Cohen’s law and Conan Doyle’s law segue into defensive programming impossible, whatever remains, • Both suggest the use of <assert.h> however improbable, must be • If you “know” a condition to be true, assert it! the truth.” • Stops a cascade in its tracks • Helps to eliminate the impossible and identify the improbable The Sign of Four, 1890 4/7/2012 Perry Kivolowitz 17 4/7/2012 Perry Kivolowitz 18 3 4/12/2012 Defensive programming Defensive programming • Code abstraction can be defensive programming • Comments • Code is more easily debugged when you… – Don’t just comment the code – Code in one place – Comment the thought process – Test in one place • As you write down your thoughts you get clarity – Fix in one place • Will you remember what you were thinking a year from • Aside: leverage STL, Boost, glm etc. now? Next week? Tomorrow? – Templates (“generics”) in general • Aside: code the “default” case and the impossible What about someone else reading your code? “else” (Note to self: Mention value of writing the comments EVEN BEFORE writing the code!) (Note to self: Emphasize “fix in one place” with respect to cut and pasted code) (Note to self: “Improbable else” – testing floats and doubles – remember CS 252, CS 354 and CS 412) 4/7/2012 Perry Kivolowitz 19 4/7/2012 Perry Kivolowitz 20 Defensive programming Defensive programming Student: I commented like you told me. Student: My code doesn’t work. // Increment j Can you help? Master: Sigh. Master: What does variable ineedsleep do? Student: I don’t remember, I was tired. Comment your thought process not minutia. Master: Sigh. descriptive_variable_names_really_help Keystrokes are cheap. Debugging time isn’t. (Note to self: Explain … keystrokes are guaranteed to end, debugging hours are not.) 4/7/2012 Perry Kivolowitz 21 4/7/2012 Perry Kivolowitz 22 Role of testing Role of testing Testing can only prove the presence • If you are fortunate to have a QA group of bugs, not their absence. – It is not their job to find bugs Edsger W. Dijkstra – That is your job Beware of bugs in the above code; I have – It is their job to confirm bugs you found are gone only proved it correct, not tried it. Donald Knuth • Don’t “throw your code over a wall” Beware of bugs in the above code; I have only written it, not tried it 4/7/2012 Perry Kivolowitz 23 4/7/2012 Perry Kivolowitz 24 4 4/12/2012 Debugging tools and techniques Debugging tools and techniques • Language based debugging aids – Use strong typing whenever possible Observe / Formulate Test hypothesis Gather Data hypothesis – Pass structures and instances by reference via “&” instead of “*” where possible – (Use destructors to) munge pointers to The scientific method is your recognizable values • Historical reference to DEADBEEF strongest, most versatile tool – “Managed” languages and runtimes are sweet (Note to self: Mention void * is a fickle friend) (Note to self: don’t depend on side effects – remember register based param passing) 4/7/2012 Perry Kivolowitz 25 4/7/2012 Perry Kivolowitz 26 Debugging tools and techniques Debugging tools and techniques • Language based debugging aids • Single step • Re‐execution of – Use preprocessor for compile time / run time aid • Breakpoints code #if – Always • Immediate modes #ifdef – Conditional • Value inspection – Examples: • In debugger #ifdef _DEBUG • Value modification • In code And the verbose: #if DEBUG_VERBOSITY > 5 • Call stack – Non‐destructive experimentation (Note to self: Spend a good deal of time on this slide –it is really important) 4/7/2012 Perry Kivolowitz 27 4/7/2012 Perry Kivolowitz 28 Debugging tools and techniques Debugging tools and techniques Student: My code doesn’t work. Student: My code doesn’t work. Can you help me? Can you help me? Master: Have you checked all return values? Master: Have you checked all return values? Student: No. Student: No. Checking makes my code messy. Master: Sigh. Master: Sigh. Return values are meant to be checked. Exceptions (also) make handling errors “cleaner.” 4/7/2012 Perry Kivolowitz 29 4/7/2012 Perry Kivolowitz 30 5 4/12/2012 Debugging tools and techniques Debugging tools and techniques Student: My program stops responding to me. Student: My program worked great. Then it Can you help? crashed. Can you help me? Master: Do you have an infinite loop? Master: Have you watched memory usage? Student: I don’t know. Student: No. Master: Sigh. Master: Sigh. Monitor CPU usage to help uncover infinite loops. Check memory consumption. Asserts and console output can also help. Leave code running for days if need be. (Note to self: Oh how I miss my Altair and PDPs –Das Blinkenlights) Use leak detection tools. (Note to self: while (true) with no break, for / while i < x without changing i, unsigneds etc.) Use automated user “simulation” tools if appropriate. Fuzz testing (Prof. B Miller) 4/7/2012 Perry Kivolowitz 31 4/7/2012 Perry Kivolowitz 32 Debugging tools and techniques Debugging tools and techniques Student: This code doesn’t do anything. Student: My program crashes when I give it this input. Can you help me? Can you help me? Master: Have you traced execution with that Master: Do you know if it is even executed? input? Student: No. Student: No. Master: Sigh. Master: Sigh. Vet and clean (all) input data to degree affordable. Console output, breakpoints and Defensive programming! code coverage tools all help. (Note to self: Remind audience that idiots* [and bad guys] can be “smarter” than they are) (Note to self: Debugging the wrong executable is not a productive use of time) (*idiots as in the expression “Idiot proof”) 4/7/2012 Perry Kivolowitz 33 4/7/2012 Perry Kivolowitz 34 Debugging tools and techniques Debugging tools and techniques Student: I must have a compiler / OS bug.