IBM™ C and C++ Maintenance and Test Toolsuite User’s Manual Release 1.0 The Toolsuite Incorporates χSuds® technology from Bellcore User’s Manual Release 1.0 First Edition (February 1998) This edition applies to Version 1 of IBM C and C++ Maintenance and Test Toolsuite for AIX, Solaris, Windows 95 and Windows NT and to all subsequent versions and releases until otherwise indicated in new editions. Click on [email protected] if you want to send us comments regarding this document. Notes to U.S. Government Users--Documentation related to restricted rights--Use, duplication or disclosure is subject to restrictions set forth in GSA ADP Schedule Contract with IBM Corp. Copyright© Bellcore 1989, 1998. All rights reserved. SPARC, SunOS, Solaris and SUN are trademarks of Sun Microsystems, Inc. Microsoft, Windows, Windows NT and the Windows logo are registered trademarks of Microsoft Corporation. UNIX is a registered trademark of The Open Group. χSuds is a registered trademark of Bellcore. AIX, OS/2, VisualAge, and IBM are trademarks of the IBM Corporation in the United States or other countries or both. Other company, product, or service names may be trademarks or service marks of others. ii User’s Manual Table of Contents Table of Contents Chapter 1 Introduction ................................................................................. 1−1 1.1 The Purpose of This Manual .........................................................1-2 1.2 The Contents of This Manual ........................................................1-2 1.3 How to Use This Manual...............................................................1-3 1.3.1 About Examples .................................................................1-4 1.3.2 Type Conventions ..............................................................1-4 1.4 Other Sources of Information........................................................1-4 Chapter 2 ATAC: A Tutorial ....................................................................... 2−1 Chapter 3 ATAC: Overview ........................................................................ 3−1 3.1 What is ATAC? .............................................................................3-2 3.2 What is Coverage Testing?............................................................3-2 3.3 What Does ATAC Do?..................................................................3-3 3.3.1 Function-Entry, Function-Return, Function-Call and Block Coverage .......................................................................................3-4 3.3.2 Decision Coverage .............................................................3-5 3.3.3 C-Use, P-use, and coverage criteria All-Uses Coverage ...3-6 3.4 How Does ATAC Work? ..............................................................3-8 3.5 What Will Using ATAC Cost You?..............................................3-9 3.6 How Does ATAC Fit into the Development Process? ................3-10 Chapter 4 ATAC: Setting Up Your Execution Environment .......................4-1 4.1 Common Environment Variables ..................................................4-2 4.1.1 ATAC_BLOCKONLY ......................................................4-2 4.1.2 ATAC_COMPRESS ..........................................................4-2 4.1.3 ATAC_COST .....................................................................4-2 4.1.4 ATAC_DIR ........................................................................4-3 4.1.5 ATAC_TEST .....................................................................4-3 4.1.6 ATAC_TMP .......................................................................4-3 4.1.7 ATAC_TRACE ..................................................................4-3 4.1.8 ATAC_UMASK ...............................................................4-4 4.2 UNIX Only Environment Information ..........................................4-4 4.2.1 ATAC_NOTRACE ............................................................4-4 4.2.2 ATAC_SIGNAL ................................................................4-4 4.2.3 ATAC_TEST_FILE ...........................................................4-5 4.2.4 PATH .................................................................................4-5 4.2.5 TERM .................................................................................4-5 4.2.6 ATACLIB ..........................................................................4-5 4.3 Windows Only Environment Information.....................................4-5 4.3.1 ATAC_CL ..........................................................................4-6 iii User’s Manual Table of Contents 4.3.2 ATAC_ICC .......................................................................4-6 4.3.3 ATAC_LIB ........................................................................4-6 4.3.4 ATAC_BIN ........................................................................4-6 4.3.5 ROOT .................................................................................4-6 4.3.6 DEFINE .............................................................................4-6 4.3.7 DEFINEPP .........................................................................4-7 4.3.8 VERSION ..........................................................................4-7 Chapter 5 ATAC: Instrumenting Your Software......................................... 5−1 5.1 Instrumenting on UNIX.................................................................5-2 5.1.1 Basic Instrumentation .......................................................5-2 5.1.2 Integrating with Makefiles .................................................5-2 5.1.3 Selectively Instrumenting Software ...................................5-2 5.1.4 Linking with ld ...................................................................5-3 5.1.5 Suppressing Instrumentation of Include Files ....................5-5 5.2 Instrumenting on Windows ...........................................................5-6 5.2.1 Basic Instrumentation ........................................................5-6 5.2.2 Integrating with Makefiles .................................................5-6 5.2.3 ................................... Selectively Instrumenting Software 5-7 5.2.4 Building Executables with Installed Linkers .....................5-8 5.2.5 Suppressing Instrumentation of Include Files ....................5-8 5.3 Common Instrumentation Options ................................................5-9 5.3.1 Code Inside Macros ...........................................................5-9 5.3.2 Marking Code for Selective Reporting ..............................5-9 5.4 Compilation and Link Errors.......................................................5-12 Chapter 6 ATAC: Executing Software Tests............................................... 6−1 6.1 Naming the Trace File ...................................................................6-2 6.2 Trace File Compression.................................................................6-2 6.2.1 Forcing Trace File Compression ........................................6-2 6.3 Temporary Trace Files...................................................................6-3 6.4 Trace File Locking.........................................................................6-4 6.5 Trace File Permission ....................................................................6-4 6.6 Parallel Test Execution..................................................................6-5 6.7 Improving Execution Speed and Saving Disk Space ....................6-5 6.8 Explaining Run-Time Errors .........................................................6-7 Chapter 7 ATAC: Managing Your Test Cases ............................................ 7−1 7.1 Listing Test Cases..........................................................................7-2 7.2 Selecting Test Cases ......................................................................7-3 7.3 Naming Test Cases ........................................................................7-4 7.3.1 Renaming Test Cases .........................................................7-5 7.3.2 What’s in a Name? .............................................................7-6 7.4 Extracting Test Cases and Merging Trace Files............................7-6 7.5 Deleting Test Cases .......................................................................7-7 iv User’s Manual Table of Contents 7.6 Assigning Cost to Test Cases ........................................................7-7 7.7 Dealing with Source Code Modifications .....................................7-8 7.8 Concerning Trace File Compression .............................................7-9 Chapter 8 ATAC: Generating Summary Reports ........................................ 8−1 8.1 Generating Coverage Summaries ..................................................8-2 8.2 Selecting What to Summarize .......................................................8-3 8.2.1 By File ................................................................................8-4 8.2.2 By Function ........................................................................8-5 8.2.3 By Test Case ......................................................................8-8 8.3 Restricting Summary Information...............................................8-10 8.3.1 By File ..............................................................................8-10 8.3.2 By Function ......................................................................8-10 8.3.3 By Coverage Criteria .......................................................8-11