TotalView Users Guide version 8.8 Copyright © 2007–2010 by TotalView Technologies. All rights reserved Copyright © 1998–2007 by Etnus LLC. All rights reserved. Copyright © 1996–1998 by Dolphin Interconnect Solutions, Inc. Copyright © 1993–1996 by BBN Systems and Technologies, a division of BBN Corporation. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, elec- tronic, mechanical, photocopying, recording, or otherwise without the prior written permission of TotalView Technologies. Use, duplication, or disclosure by the Government is subject to restrictions as set forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and Computer Software clause at DFARS 252.227-7013. TotalView Technologies has prepared this manual for the exclusive use of its customers, personnel, and licensees. The infor- mation in this manual is subject to change without notice, and should not be construed as a commitment by TotalView Tech- nologies. TotalView Technologies assumes no responsibility for any errors that appear in this document. TotalView and TotalView Technologies are registered trademarks of TotalView Technologies. TotalView uses a modified version of the Microline widget library. Under the terms of its license, you are entitled to use these modifications. The source code is available at: ftp://ftp.totalviewtech.com/support/toolworks/Microline_totalview.tar.Z. All other brand names are the trademarks of their respective holders. Book Overview part I - Introduction 1 Getting Started with TotalView ..............................................3 2 About Threads, Processes, and Groups..............................15 part II - Setting Up 3 Getting Started with Remote Display Client....................35 4 Setting Up a Debugging Session ..........................................53 5 Setting Up Remote Debugging Sessions ...........................85 6 Setting Up MPI Debugging Sessions .................................103 7 Setting Up Parallel Debugging Sessions..........................137 part III - Using the GUI 8 Using TotalView Windows.....................................................169 9 Visualizing Programs and Data ...........................................183 part IV - Using the CLI 10 Using the CLI...............................................................................201 11 Seeing the CLI at Work ...........................................................215 part V - Debugging 12 Debugging Programs ..............................................................225 13 Using Groups, Processes, and Threads.............................253 14 Examining and Changing Data............................................281 15 Examining Arrays ......................................................................335 16 Setting Action Points...............................................................351 17 Evaluating Expressions ...........................................................383 Glossary ........................................................................................399 Index ..............................................................................................413 TotalView Users Guide: version 8.8 iii iv Book Overview Contents About This Book TotalView Family Differences ..........................................................................................xxiii How to Use This Book ......................................................................................................xxv Using the CLI ...................................................................................................................xxvi Audience .........................................................................................................................xxvi Conventions ...................................................................................................................xxvii TotalView Documentation ............................................................................................ xxviii Contacting Us .................................................................................................................xxix part I - Introduction 1 Getting Started with TotalView Getting Started ................................................................................................................... 3 Starting TotalView ......................................................................................................... 4 What About Print Statements? ..................................................................................... 5 Examining Data ............................................................................................................ 6 Examining Arrays .......................................................................................................... 8 Seeing Groups of Variables .......................................................................................... 8 Setting Watchpoints ................................................................................................... 10 Debugging Multi-process and Multi-threaded Programs .................................................. 10 Program Using Almost Any Execution Model ............................................................. 11 Supporting Multi-process and Multi-threaded Programs ........................................... 11 Using Groups and Barriers ................................................................................................ 12 Memory Debugging ........................................................................................................... 13 Introducing the CLI ........................................................................................................... 14 What’s Next ....................................................................................................................... 14 2 About Threads, Processes, and Groups A Couple of Processes ...................................................................................................... 15 Threads ............................................................................................................................. 18 Complicated Programming Models ................................................................................... 19 Types of Threads ...............................................................................................................20 Organizing Chaos .............................................................................................................. 22 Creating Groups ................................................................................................................26 TotalView Users Guide: version 8.8 v Simplifying What You’re Debugging .................................................................................. 30 part II - Setting Up 3 Getting Started with Remote Display Client Using Remote Display ....................................................................................................... 35 Installing the Client ..................................................................................................... 36 Installing on Linux ................................................................................................... 36 Installing on Microsoft Windows ............................................................................ 37 Installing on Apple Mac OS X Intel ......................................................................... 37 Sample Session .......................................................................................................... 39 Naming Intermediate Hosts ....................................................................................... 41 Submitting a Job to a Batch Queuing System ........................................................ 41 Setting Up Your Systems and Security ....................................................................... 41 Session Profile Management ...................................................................................... 42 Sharing Profiles ....................................................................................................... 43 Remote Display Server and Viewer ............................................................................. 43 Batch Scripts .............................................................................................................. 44 tv_PBS.csh Script .................................................................................................... 45 tv_LoadLeveler.csh Script ...................................................................................... 45 Remote Display Commands .............................................................................................. 46 Session Profiles Area .................................................................................................. 46 Remote Host and Routing Area .............................................................................. 46 Product Area ........................................................................................................... 48 Using the Submit to Batch Queuing System Entries .............................................. 49 File > Profile > Save .............................................................................................. 50 File > Profile > Delete ..........................................................................................