Hands-on Workshop on How To Debug Codes at the Institute H. Birali Runesha, Shuxia Zhang and Ben Lynch (612) 626 0802 (help) [email protected] October 13, 2005 Outline • Debuggers at the Institute • Totalview – Starting TotalView – The TotalView GUI and basic windows – Data management and visualization – Breakpoints and Action points – Parallel debugging: MPI, OpenMP, etc. 1 Debuggers at the Institute • Write statements • CVD • dbx • Intel IDB • pdbx • DDD • Totalview • Xpdbx • Other tools: Compiler • xldb options, profilers, • Pedb hardware counters • GNU gdb dbx • dbx is a command line debugger for serial and parallel Fortran and C programs respectively • Most useful Commands: stop at linenumber # set a breakpoint at the given linenumber stop in myroutine # set a breakpoint at the given routine run < infile # run your program, reading from the file infile next # to execute the next step of your code print var # see the value of a given variable whatis var # determine the data type of a given variable cont # continue execution from where it stopped • Compile your program with –g option • Start dbx with or without the executable as an argument. • More info: man dbx, on dbx prompt use help • Available: unix systems: IBM, SGI and SUN 2 pdbx • Command-line parallel debugger built on dbx • IBM debugger for parallel programs on AIX systems and accepts all flags supported by POE • Supports C, C++, Fortran 77, Fortran 90 programs • Compile the code with -g flag • Setup POE environment variables, e.g.: setenv MP_BUFFER_MEM 28MB setenv MP_EAGER_LIMIT 4090kb • Run pdbx: pdbx a.out [program_options] [poe options] or pdbx -a poe process id [limited poe options] • Available: IBM Power3 and Power4 • More information: man pdbx xpdbx • Parallel debugger with graphical interface • Load executable after starting xpdbx. Use the File menu • Use X resources to customize the xpdbx window – The default resources for xpdbx are listed in /usr/lib/X11/app-defaults/Xpdbx • Needs the –rmpool option on the IBM SP/Power4 for an interactive run • Can set break/trace points 3 xldb • Serial and multi-thread debugger with graphical interface • xldb a.out – Window pops up with source, etc • Group of blue bars at the top right – Click on bar to open window – To minimize window, click on bar at top to get menu, click on “minimize” • To set breakpoint, click on source line • To navigate, see “commands” window pedb • Graphical user interface based parallel debugger for Fortran and C programs • Same as xpdbx • pedb a.out <poe options> – Window pops up with source, etc • To set breakpoint, double-click on source line. To delete breakpoint, right-click icon in left margin, select “ delete” • To navigate, use buttons below source listing Step over step into step return Continue halt play stop • “tasks”(processes for MPI) are chosen using buttons in “task” window • Use “Find” to find a specific variable 4 GNU gdb • gdb is a command line debugger for serial programs • Support C, C++, and Fortran77 programs • Available: linux and unix systems • Compile the code : g77 –g my.f or gcc –g my.c • Start debugging: gdb ./a.out or gdb ./a.out core # to include the core or gdb ./a.out 1234 # to attach GDB to process 1234 • Most useful Commands: quit: Exit from GDB. break [file:]function: Set a breakpoint at function in file. run: Start your program bt: Backtrace: display the program stack. print expr: Display the value of an expression. c: Continue running your program (after stopping) stop:set a breakpoint at line number or routine • Getting help: – (gdb) help – http://sources.redhat.com/gdb/onlinedocs/gdb _toc.html – man gdb 5 CVD • CVD - Invokes the WorkShop Debugger • a source-level debugging tool with GUI • set various types of breakpoints, watchpoints, and so on. • view variables, expressions, structures, arrays, call tree • debug Ada, C, C++, Fortran 77, and Fortran 90 programs. • Available on SGI system running Irix Operating systems (SGI workstations and Origin) • To invoke the Debugger for a Fortran 77 program: $ f77 -g -o myprog myprog.f $ cvd myprog # serial job or $ cvd myprog core # serial job with core dump or $ cvd mpirun -args -np 2 a.out # For an mpi job • More information: • Manpage: man cvd • http://techpubs.sgi.com/library/tpl/cgi-bin/init.cgi CVD and Other Workshop Tools: • Build Analyzer (cvbuild) - Invokes the Build Analyzer that provides a graphical display of build dependency information. • Parallel Analyzer (cvpav) - Invokes the Parallel Analyzer that presents information about MP Fortran programs. • Performance Analyzer (cvperf) - Invokes the Performance Analyzer that lets you specify the name of a performance task to enable proper data collection for your experiment. • Static Analyzer (cvstatic) - Invokes the Static Analyzer that lets you display source code information about your code and facilitates the setting of breakpoints. • Tester (cvxcov) - Invokes the test coverage tool that lets you perform dynamic test coverage over any set of tests. 6 Intel IDB • Intel debugger are part of the compiler installation • Support C, C++, Fortran 77/90 • Both dbx (default) and gdb like interfaces • Documentation: – http://www.intel.com/software/products/compilers/docs/linux/idb_m anual_l.html • Debugging of optimized code generated by intel compilers does not work • Does not support multithreaded/multiprocess applications. claims do support mpich programs (untested) • Available on the SGI Altix and systems with Intel compilers DDD • Graphical debugger • Part of Redhat distribution • Works best with gdb • Can hook in idb via: ddd --debugger “idb” -- dbx ./a.out • Multiple panes for – Data/array browsing – Source code – Assembler code – Command line interface • Use help menu for details 7 Other Debuggers • Portland Group PGDBG graphical debugger • LF95: Fujitsu debugger FDB • Write statements! • Editors Totalview 8 Introduction • Debugger for Unix/Linux platform • Supports Fortran, C/C++ programs • Serial, multi-threaded, parallel programs employing multiple processors • MPI, MPICH, OpenMP, PVM, SHMEM, Global arrays, and UPC • User interface and debugging style remain uniform and consistent across platforms and programming models. Starting TotalView • Compile with –g option • Starting Totalview on MSI machines: module load totalview • On a new process: % totalview a.out -a <arguments to a.out> • On a core file: % totalview a.out core • To attach to a running process: % totalview 9 o • File > New Program dialog Box • Short cut: ctrl-N • F1 function key tells Totalview to display help info about the window TotalView Windows Root Unattached Window Processes Window Process Data Window Windows Test drive: •Menu driven •Alternative keyboard shortcuts for those who prefer them •All 3 mouse buttons are used 10 Root Window Process name Process/thread status Number of threads Expand list Process Window Local variables Stack for the Trace pane selected frame Source pane Action Thread Points pane pane 11 Source Pane Gridded box is Current function and source file a possible site for a breakpoint Select to set oneCurrent point of execution • Dive on a source word to get more information • Select a line to use Run to selection command • Select or dive on a line number to set an action point Stack Trace Pane • Select a line to choose a new stack frame • Just dive on a routine and TotalView shows the routine in the Source Pane and its variables in the Stack frame value Stack Frame Pane. 12 Stack Frame Pane Names of arguments Names of function’s local variables Stack Frame pane in the process window contains current call stack Values of variables Dive on a variable to bring up a data pane (and follow a pointer) Starting, Stopping, & Restarting Your Program • Method 1 – Set a breakpoint – Select Go on the icon bar – The program starts executing and stop when it reaches the breakpoint • Method 2 – Select step – Totalview starts and then stops it immediately before the first statement in your main function • To stop a running program, select halt • To restart a program, select the Group > Restart command 13 • Stepping through a Program – Use the Step and Next commands. Step and Next both execute the current line. The difference between them is that if the line has a function call, Step dives into the function while Next executes it. – If you want to get to a line without individually stepping each intervening line, select the line, and then select Run To. • Stepping Out of a Function – If you've stepped into a function and want to pop out to the statement that called it, select the Out command. Data Management 14 Displaying Data Seeing a value of a variable • Method 1 Dive on a variable displayed in the Source Pane. The variable appears in a Variable Window. • Method 2 Scroll to the variable in the Stack Frame Pane. If you want to display a compound variable like an array or structure, dive on it. TotalView responds by displaying information in a Variable Window. • Method 3 Use the View > Lookup Variable command. The variable can be global or local. If you've defined the same variable in more than one routine, TotalView displays the variable contained in the current stack frame. Variable Window Array Data Window • Array has an actual type as well as a declaration type • Arrays have a slice field that you can edit to specify the dimensions to display • You can update data by Selecting (left mouse button) the value and editing it 15 Array Slice • An array slice indicates which portion of an array you want to see. I.e.: see elements 3 to 8 • Fortran example- TotalView displays a