Debugging Memory Problems with MemoryScape™ Version 2021.3 August, 2021 Copyright 2007-2021 by Rogue Wave Software, Inc., a Perforce company (“Rogue Wave”). All rights reserved. Copyright 1998–2007 by Etnus LLC. All rights reserved. Copyright 1996–1998 by Dolphin Interconnect Solutions, Inc. Copyright 1993–1996 by BBN Systems and Technologies, a division of BBN Corporation. All trademarks and registered trademarks are the property of their respective owners. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise without the prior written permission of Rogue Wave. Perforce has prepared this manual for the exclusive use of its customers, personnel, and licensees. The information in this manual is subject to change without notice, and should not be construed as a commitment by Perforce. Perforce assumes no responsibility for any errors that appear in this document. TotalView and TotalView Technologies are registered trademarks of Rogue Wave. TVD is a trademark of Rogue Wave. Perforce uses a modified version of the Microline widget library. Under the terms of its license, you are entitled to use these modifications. The source code is available at https://rwkbp.makekb.com/. All other brand names are the trademarks of their respective holders. ACKNOWLEDGMENTS Use of the Documentation and implementation of any of its processes or techniques are the sole responsibility of the client, and Perforce Software, Inc., assumes no responsibility and will not be liable for any errors, omissions, damage, or loss that might result from any use or misuse of the Documentation. ROGUE WAVE MAKES NO REPRESENTATION ABOUT THE SUITABILITY OF THE DOCUMENTATION. THE DOCUMENTATION IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. ROGUE WAVE HEREBY DISCLAIMS ALL WARRANTIES AND CON- DITIONS WITH REGARD TO THE DOCUMENTATION, WHETHER EXPRESS, IMPLIED, STATUTORY, OR OTHERWISE, INCLUDING WITHOUT LIMITATION ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PUR- POSE, OR NONINFRINGEMENT. IN NO EVENT SHALL PERFORCE SOFTWARE, INC. BE LIABLE, WHETHER IN CONTRACT, TORT, OR OTHERWISE, FOR ANY SPECIAL, CONSEQUENTIAL, INDIRECT, PUNITIVE, OR EXEMPLARY DAMAGES IN CONNEC- TION WITH THE USE OF THE DOCUMENTATION. The Documentation is subject to change at any time without notice. TotalView by Perforce http://totalview.io Batch Scripting Using tvscript 31 Contents Using the dheap Command. 31 dheap Example . 32 Notification When free Problems Occur . 32 Checking for Problems . 2 Showing Backtrace Information: dheap -backtrace: . 33 Programs and Memory . 4 Guarding Memory Blocks: dheap -guards . 33 Memory Reuse: dheap -hoard . 34 Behind the Scenes . 8 Writing Heap Information: dheap -export . 35 Your Program’s Data . 9 Filtering Heap Information: dheap -filter . 35 The Data Section . 9 Checking for Dangling Pointers: dheap -is_dangling: . 36 The Stack. 9 Detecting Leaks: dheap -leaks . 37 The Heap . 13 Block Painting: dheap -paint . 37 Finding Heap Allocation Problems . 13 Red Zones Bounds Checking: dheap -red_zones . 38 Finding Heap Deallocation Problems . 13 Deallocation Notification: dheap -tag_alloc . 42 realloc() Problems . 13 TVHEAP_ARGS . 42 Finding Memory Leaks . 14 Examining Memory . .45 Starting MemoryScape . 16 Block Properties . 47 Using MemoryScape Options . 17 Memory Contents Tab . 49 Preloading MemoryScape . 18 Additional Memory Block Information . 50 Filtering 51 Understanding How Your Program is Using Memory . 19 Using Guard Blocks 51 Finding free() and realloc() Problems . 21 Event and Error Notification 21 Using Red Zones . .53 Types of Problems . 23 Using Guard Blocks and Red Zones . .54 Freeing Stack Memory . 23 Block Painting 54 Freeing bss Data . 23 Freeing Data Section Memory . 23 Hoarding . .56 Freeing Memory That Is Already Freed . 23 Example 1: Finding a Multithreading Problem . 56 Tracking realloc() Problems . 24 Example 2: Finding Dangling Pointer References. 56 Freeing the Wrong Address . 24 Debugging with TotalView . .58 Finding Memory Leaks . 25 Starting MemoryScape. 61 Fixing Dangling Pointer Problems . 28 Adding Programs and Files to MemoryScape. 65 Attaching to Programs and Adding Core Files . 66 Dangling Pointers. 29 Stopping Before Finishing Execution . 66 Batch Scripting and Using the CLI . 31 Exporting Memory Data . 66 MemoryScape Information . 67 Using Saved State Information . 111 Where to Go Next . 67 Where to Go Next . 112 Basic Options. 71 Overview . 113 Advanced Options . 74 Obtaining a Comparison . 113 Halt execution at process exit (standalone MemoryScape only) 75 Memory Comparison Report. 114 Halt execution on memory event or error . 75 Where to Go Next . 115 Guard allocated memory . 77 Saving Report Information . 117 Use Red Zones to find memory access violations . 78 Using Remote Display 120 Restricting Red Zones . 79 Customizing Red Zones . 79 Compiling Programs . 122 Paint memory . 80 Linking with the dbfork Library . 123 Hoard deallocated memory . 80 dbfork on IBM AIX on RS/6000 Systems . 123 Where to Go Next . 81 Linking C++ Programs with dbfork . 123 Controlling Program Execution from the Home | Summary Screen 85 dbfork and Linux or Mac OS X . 124 Controlling Program Execution from the Manage Processes Screen 85 dbfork and SunOS 5 SPARC . 124 Controlling Program Execution from a Context Menu. 85 Where to Go Next . 85 Ways to Start MemoryScape . 125 Information Types . 86 Attaching to Programs 126 Process and Library Reports . 87 Chart Report. 87 Setting Up MPI Debugging Sessions . 127 Where to Go Next . 89 Debugging MPI Programs. 127 Error Notifications . 90 Debugging MPICH Applications . 129 Deallocation and Reuse Notifications. 92 Starting MemoryScape on an MPICH Job . 129 Where to Go Next . 93 Attaching to an MPICH Job . 130 Window Sections. ..