IBM Spectrum Protect Version 8.1.0 Problem Determination Guide IBM IBM Spectrum Protect Version 8.1.0 Problem Determination Guide IBM Note: Before you use this information and the product it supports, read the information in “Notices” on page 211. This edition applies to version 8, release 1, modification 0 of IBM Spectrum Protect (product number 5725-W98, 5725-W99, 5725-X15), and to all subsequent releases and modifications until otherwise indicated in new editions. © Copyright IBM Corporation 1993, 2016. US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents About this publication ........ vii Resolving errors due to the incorrectly coded Who should read this guide......... vii include or exclude list .......... 25 Publications .............. vii Resolving Snapshot Difference problems..... 25 Resolving snapshot directory problems for Chapter 1. Help facilities ....... 1 NetApp or N-Series file system volumes.... 28 Resolving login problems when using the Backup-archive client help .......... 1 encrypted file system on AIX operating systems . 28 Access help for the Client Service Configuration Resolving image backup errors ........ 29 Utility (dsmcutil) ............ 2 Resolving Linux image backup errors..... 29 Server or storage agent help ......... 2 Resolving backup failures when you use Linux Accessing server or storage agent help for snapshot image backup ......... 30 commands .............. 2 Resolving errors during AIX JFS2 snapshot-based Accessing help for messages ........ 3 backup-archive and image backup...... 31 Command-line interface help for the client ... 3 Support solutions for the IBM Spectrum Protect API 33 Reporting a problem with a help topic...... 3 Gathering API information before calling IBM support............... 33 Chapter 2. Resolving client problems .. 5 Gathering API files before calling IBM Support 33 Examining error messages .......... 5 Determining whether data is sent to the storage Examining the server activity log messages .... 5 agent rather than the server ......... 35 Identifying when and where the problem can occur . 5 Running applications that use the API as a non-root Reproducing the problem .......... 6 user ID ................ 36 Collecting documentation to resolve problems with Journal Based Backup problem determination ... 38 the client application ........... 6 Determining if a backup will be journal-based.. 39 Determining why the dsmc, dsmadmc, dsm, or dsmj Running the journal daemon in the foreground 40 programs do not start ........... 7 The Journal Database Viewing utility ..... 40 Resolving problems with client option sets .... 9 Using Windows Volume Shadow Copy Services .. 41 Scenarios for resolving problems with client Defining VSS transient errors ....... 42 option sets .............. 9 Defining Windows VSS test flags ...... 42 Resolving password expiration problems .... 10 Volume Shadow Copy Services tuning .... 43 Resolving LDAP-authenticated password problems 10 Gathering VSS diagnostic information for Verifying your password authentication Microsoft assistance .......... 43 configuration ............. 11 Troubleshooting errors using a VSS trace ... 44 The IBM Spectrum Protect server does not accept Running VSS API calls with the vsreq.exe sample the LDAPPASSWORD .......... 12 program .............. 44 Resolving problems with the LDAP directory Comparing IBM Spectrum Protect and server ............... 12 Ntbackup.exe interaction with VSS ..... 45 Auditing the LDAP directory server to clean up SHOW commands for the backup-archive client ... 45 the server .............. 14 Resolving problems for recovery of individual Error messages for LDAP authenticated Microsoft SQL databases from a virtual machine passwords .............. 14 backup ................ 47 Resolving client scheduling problems...... 17 Resolving database access problems ..... 47 Determining the status of a scheduled event .. 18 Viewing active copies of Microsoft SQL databases 49 Checking for errors in the server activity log .. 18 Microsoft SQL databases with DBCS names .. 49 Starting and stopping the client service .... 19 Responses to messages for virtual machine Resolving errors when including or excluding client backups with application protection ..... 49 files during backup processing ........ 20 Saving VSS XML manifest files ....... 50 Identifying files that are included or excluded by Determining whether a virtual machine backup the server client option set ........ 21 might fail .............. 51 Excluding files automatically from backup processing .............. 21 Chapter 3. Resolving IBM Spectrum Excluding files with the EXCLUDE.DIR statement .............. 22 Protect server problems ....... 53 Determining whether compression, encryption, Recreating the problem .......... 53 and subfile backup statements include or exclude 24 Checking the server activity log file and other log Using delimiters to include or exclude files ... 25 files ................. 53 Checking system error log files for device errors .. 54 © Copyright IBM Corp. 1993, 2016 iii Reverting server options or settings ...... 54 Storage pool experiences high volume usage after Restarting the scheduling service ....... 54 increasing MAXSCRATCH value ........ 94 Resolving server space issues ........ 55 Storage pool is set to use collocation, but Allocating additional server memory ...... 55 volumes contain data that is not collocated ... 94 Configuring a server instance to use shared memory 56 Resolving storage problems for active data pools 95 Changing the copy frequency ........ 56 Resolving issues with cloud-container storage pools 95 Resolving RELABEL operation errors ...... 57 Avoiding communication errors during import Chapter 4. Resolving Operations processing ............... 57 Center problems .......... 99 Adding a self-signed certificate to the keystore .. 58 Log files overview ............ 99 Determining why summary records for a client Viewing the Operations Center log from within backup event are missing.......... 59 the Operations Center ......... 100 Resolving installation and upgrade problems ... 60 Alerts are not updated immediately ...... 100 Installation log files........... 60 Active tasks are not canceled immediately.... 101 Installation wizard fails to start ....... 60 Further known issues with the Operations Center 102 Resolving GSKit installation problems .... 60 Server instances are not created during an upgrade .............. 61 Chapter 5. Resolving communication Resolving a stopped uninstallation process ... 62 problems ............. 103 Client automatic deployment did not upgrade Resolving errors created when connecting to the the client software ........... 62 server ................ 103 Resolving server stoppages ......... 63 Resolving failed connections by clients or Resolving a stoppage or loop ....... 64 administrators ............. 103 Resolving wait state problems with external user Resolving Secure Sockets Layer errors ..... 104 repository servers ........... 65 Recovering the key database file password .. 106 Finding the server error file (dsmserv.err) ... 66 Troubleshooting the certificate key database .. 107 Finding the system image (core file) ..... 66 Retrieving library files for core analysis .... 67 Chapter 6. Resolving storage agent Retrieving system log files ........ 67 problems ............. 109 Retrieving the activity log ........ 68 Checking the server activity log for storage agent Detecting errors after a server service starts and information .............. 109 stops ............... 68 Resolving an error caused by reading or writing to sqllib/db2dump directory causes shutdown .. 68 a device ............... 109 Resolving issues with database page verification 69 Resolving problems caused by changing storage Resolving database errors.......... 70 agent options ............. 109 Resolving database manager starting problems 70 Resolving problems caused by changing server Tracing the user ID and Password plug-in ... 71 options or settings ............ 110 Limiting DB2 memory allocation ...... 72 Storage agent LAN-free setup ........ 110 Retrieving DB2 version information ..... 73 Resolving the issue of data being sent directly to Locating DB2 diagnostic log files ...... 73 the server.............. 110 DB2 upgrade log files .......... 74 Resolving a disqualified LAN-free-enabled Resolving a missing or incorrect database ID file storage pool ............. 111 problem .............. 75 Ensuring that data is transferred using a BACKUP DB Resolving problems with the and the LAN-free environment ......... 111 RESTORE DB commands.......... 76 Characteristics of the $$_TSMDBMGR_$$ user ID 80 Chapter 7. Using trace to resolve Resolving database reorganization problems .. 81 Analyzing the process symptoms to resolve problems ............. 113 problems ............... 81 Starting an extended trace of the Operations Center 113 Reviewing process messages to determine the Tracing the Operations Center by enabling state of server operations ......... 81 logging functions from within the Operations Analyzing the ANR1221E error message.... 87 Center ............... 113 Analyzing the ANR2317W error message ... 88 Tracing the Operations Center by enabling Analyzing error messages ANR1330E and functions in the logging configuration file ... 114 ANR1331E.............. 89 Enabling a trace for the server or storage agent .. 115 Files are not expired after reducing versions .. 92 Enabling a stack trace for messages for the Process