Assentor Operations Guide for Content Manager for Message Monitoring and Retention

Release 3.3.2 Installed for Content Manager for Message Monitoring and Retention

January 2005 NOTIFICATIONS

Release 3.3.2 (January 2005)

This document reflects the changes and enhancements for Assentor Compliance 3.3.1. Make certain that you are using the correct documentation for your product. This can be verified on the About page of Assentor Compliance.

The publisher of this document is iLumin Software Services, Inc., 1881 Campus Commons Drive, Suite 400, Reston, VA 20191. Printed in the U. S. A.

© Copyright 2005 by iLumin Software Services, Inc. All rights reserved. No part of this book, including its interior design, and icons, may be reproduced, stored in a retrieval system, or transmitted in any form by any means, electronic, mechanical, photocopying, recording or otherwise, without the express written permission of iLumin Software Services, Inc.

LIMITED LIABILITY; DISCLAIMER OF WARRANTY

ILUMIN SOFTWARE SERVICES, INC. MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE ACCURACY OR COMPLETENESS OF THE CONTENTS OF THIS DOCUMENT AND SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANBTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. NO WARRANTY IS TO BE MADE EITHER BY SALES REPRESENTATIVES OR THROUGH ANY WRITTEN SALES MATERIALS. ILUMIN SOFTWARE SERVICES, INC. SHALL NOT BE LIABLE FOR ANY LOSS OF PROFITS OR ANY OTHER COMMERCIAL DAMANGES, INCLUDING BUT NOT LIMITED TO SPECIAL, INCIDENTAL, CONSEQUENTIAL, OR OTHER DAMAGES.

Trademarks

• iLumin is a trademark of iLumin Software Services, Inc. • Server and Internet Explorer are trademarks of Microsoft. • Content Manager (CM) is a trademark of International Business Machines (IBM). • Chilkat Charset is a trademark of Chilkat Software, Inc.

All other trademarks, trade names, or product names used in this document are the property of their respective owners.

Terms denoted by an asterisk (*) used in this publication are trademarks or service marks of iLumin Software Services, Inc. in the United States or other countries. Terms denoted by a double asterisk (**) used in this publication are trademarks of other companies. Such terms are marked the first time they are used in each chapter in the book. iLumin Software Services Inc. 1881 Campus Commons Drive, Suite 400 Reston, VA 20191 Phone: (703) 481-8627 Fax: (703) 481-8672 Internet: www.ilumin.com Contents

LIST OF TABLES...... VI

LIST OF FIGURES ...... VII

ABOUT THIS GUIDE ...... VIII

Scope of This Guide...... viii

Who Should Use This Guide...... ix Skills You Need to Perform Tasks in Assentor Compliance...... ix

How This Guide Is Organized...... ix

Where to Find More Information...... x

Conventions Used in This Guide ...... xi

CHAPTER 1 DB2 LOGGING, BACKUPS, AND CONFIGURATION ...... 1 Architecture of DB Logging and Backup Alternatives...... 1 Archive Logging ...... 1 Circular Logging...... 1 Backup Alternatives ...... 1

Installation and Configuration of DB2...... 2

CHAPTER 2 PERFORMING PREVENTIVE MAINTENANCE...... 3 Performing Daily Preventive Maintenance...... 3 1. Monitor the File System Queues ...... 4 2. Monitor the Services ...... 4 3. Check That Current Messages Are Available...... 5

Assentor Operations Guide for Content Manager for Message Monitoring and Retention iii Contents

4. Review the Log Files ...... 5 Performing Weekly Preventive Maintenance...... 12 1. Delete Temporary Files...... 12 2. Delete Outdated Logs...... 12

Maintenance Checklist for Assentor Compliance Backups...... 12

CHAPTER 3 USING LOG FILES FOR PROBLEM ANALYSIS...... 14 Understanding Log Messages ...... 14 Managing Log Files ...... 16

Configuring the Log...... 17 Recommended Log Configuration...... 18

Viewing Log Files...... 19

Determining Which Log File Is Currently Active ...... 19 Understanding the Windows 2000 Event Viewer Messages...... 21

CHAPTER 4 TROUBLESHOOTING ASSENTOR COMPLIANCE ...... 24 Determining the Type of Problem...... 24

Understanding the Assentor Controller...... 24

Resolving Assentor Controller Problems...... 24 Assentor Controller Stopped on MP Server ...... 25 iLumin Delivery Service Not Started...... 25 W3srv not started ...... 25

Resolving Mail-Processing Problems ...... 26 Determine the Cause of Mail-Processing Problems ...... 26 Repeated Delivery of Same Mail Message...... 27 Problems with Quarantined/Released Messages ...... 27 Out of Space or Running Slow ...... 28 Monitor Queues...... 28 Backup in the Message_done Table ...... 29 Failure of Content Manager or DB2...... 29 Determine if Quarantined Mail is Lost ...... 29

Resolving User Interface Problems...... 30 Unable to Contact Web Server ...... 30 Unable to Authenticate with Assentor...... 31 IIS Authentication Problem...... 31 Assentor Compliance Authentication Problem...... 31 Users Cannot View Messages ...... 31 Browser Hangs after User Logs On to Assentor Compliance ...... 31

Assentor Operations Guide for Content Manager for Message Monitoring and Retention iv Contents

Users Cannot Approve or Reject Messages ...... 32 Resource Management Errors...... 32

Assentor Services Register/Unregister...... 33

GLOSSARY ...... 34

Assentor Operations Guide for Content Manager for Message Monitoring and Retention v List of Tables

Table 1. Assentor Compliance books, worksheets, and file names...... x

Table 2. Conventions used in this guide...... xi

Table 3. Maintenance checklist for Assentor Compliance ...... 12

Table 4. Predefined severity levels for log messages...... 15

Table 5. Logger registry settings ...... 17

Table 6. Recommended log configuration for Assentor...... 18

Table 7. Locations of log files ...... 20

Table 8. Assentor Address Processor messages ...... 21

Table 9. Assentor Controller messages ...... 22

Table 10. Assentor Services Register/Unregister ...... 33

Assentor Operations Guide for Content Manager for Message Monitoring and Retention vi List of Figures

Figure 1. Log message format...... 14

Figure 2. Example of initial log message ...... 15

Figure 3. Syntax of the LogUtil command...... 19

Assentor Operations Guide for Content Manager for Message Monitoring and Retention vii About This Guide

Assentor* Enterprise, a product suite from iLumin* Software Services, Inc., is an enterprise message archival and management solution. It consists of several modules, but the centerpiece of the Enterprise suite is Assentor Compliance.

The purpose of Assentor Compliance is to recognize, flag, and quarantine inappropriate content in inbound, internal, and outbound email messages in your firm for possible compliance violations with regulatory agencies and with the policies of your firm. In addition, Assentor Compliance provides support for search, discovery, and retrieval of messages that may entail searching through archives containing terabytes of data.

Assentor Compliance operates in the SQL environment and is also fully integrated with IBM DB2 Content Manager (CM). The joint solution between IBM and iLumin is known as Content Management for Message Monitoring and Retention (CM for MMR).

CM for MMR is IBM’s solution for managing message monitoring and the file archives of the messages that Assentor Compliance processes.

Scope of This Guide

This guide is one of a set of guides that is used in support of Assentor*1 Compliance and its interfacing IBM**Content Management for Message Monitoring and Retention (CM for MMR) product.

The primary purpose of the Assentor Operations Guide for Content Manager for Message Monitoring and Retention is to explain database setups, operations tasks, and troubleshooting methods for Assentor Compliance. This guide deals solely with the operations of a CM for MMR system and is not applicable for SQL customers.

1 Terms denoted by an asterisk (*) used in this publication are trademarks or service marks of iLumin Software Services, Inc. in the United States or other countries. Terms denoted by a double asterisk (**) used in this publi- cation are trademarks of other companies. Such terms are marked the first time they are used in each chapter in the book.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention viii About This Guide

Who Should Use This Guide

Depending on the size and organization of your firm, the audience for this guide may include the following: • Someone, often a system administrator, who performs all of the tasks described in this guide • One or more of the following individuals: > A product administrator who sets up users, groups, and threshold classes; customizes the user interface; manages email messages; and backs up the system > A database administrator who is responsible for maintaining and backing up the database > A system operator who is responsible for performing operations maintenance

Since one or more people may use this guide, the guide covers related tasks that are performed on a daily or frequent basis and clearly separates these tasks so that you know which portion of the guide to use.

Skills You Need to Perform Tasks in Assentor Compliance

Whether you are a system administrator, product administrator, database administrator, or system operator, you should have the following: • Experience in using computer applications running under Windows 2000** • Familiarity in using the Microsoft** Internet Explorer browser

You should feel comfortable in performing the basic functions that are common to a Windows environment.

How This Guide Is Organized

The Assentor Operations Guide for Content Manager for Message Monitoring and Retention provides information for database administrators, system operators, and troubleshooting personnel.

The following chapters are designed for database administrators and provide information on configuring and backing up the Assentor database: • Chapter 1, “DB2 Logging, Backups, and Configuration,” provides information on configuring the Assentor database and on performing scheduled database tasks. • Chapter 2, “Performing Preventive Maintenance,” describes scheduled database tasks.

The following chapter is designed for system operators and provides information on the log files:

Assentor Operations Guide for Content Manager for Message Monitoring and Retention ix About This Guide

• Chapter 3, “Using Log Files For Problem Analysis,” explains the format of Assentor Compliance log messages and discusses how to manage, configure, and view log files. It also lists the messages that you may see for the Assentor Address Processor and Assentor Controller.

The following chapter is designed for any users who are tasked to troubleshoot problems when they occur in Assentor Compliance. It provides information for determining and resolving problems: • Chapter 4, “Troubleshooting Assentor Compliance,” offers suggestions for troubleshooting problems in Assentor Compliance.

The glossary includes a list of terms and acronyms commonly used in Assentor Compliance.

Where to Find More Information

If you would like to read more about what Assentor Compliance does and how it works, see a list of its components, and get a functional overview of the Assentor Compliance system, refer to Chapter 1, “Introducing the Assentor Compliance System,” in the v. 3.3.1 Assentor Compliance Administration Guide.

For more information about this release of Assentor Compliance, refer to the documents listed in Table 1. The table indicates the path and file names for the Adobe Acrobat Reader (.pdf) document files that are shipped to you on the Assentor CD-ROM. This table also lists worksheets in Word that you can use in developing a compliance strategy for your firm. (These are the same worksheets that are shown in the Assentor Compliance Planning Guide for Threshold Management.)

The cover and copyright pages of each guide state the release date and version of the documentation. You can click the About menu option in the Assentor Compliance interface to verify that the version of the software you are using corresponds to the documentation release.

To read about the new features and enhancements that have been made to each new release of Assentor Compliance, refer to the “What’s New” document.

Table 1. Assentor Compliance books, worksheets, and file names

Path and File Name for Book and Worksheets Adobe Acrobat Reader .pdf File or Microsoft Word File

Books

Assentor Compliance Administration Guide Documents\ComplianceAdminGuide.cm.pdf

Assentor Compliance Configuration Guide Documents\ComplianceConfigurationGuide.cm.pdf

Assentor Compliance Planning Guide for Threshold Management Documents\ComplianceThresholdPlanningGuide.cm.pdf

Assentor Compliance Quick Reference Guide Documents\ComplianceQuickRefGuide.cm.pdf

Assentor Compliance Reviewing Guide Documents\ComplianceReviewingGuide.cm.pdf

What’s New in Assentor Compliance [current version] Documents\ComplianceWhatsNew.cm.pdf

Assentor Operations Guide for Content Manager for Message Monitoring and Retention x About This Guide

Table 1. Assentor Compliance books, worksheets, and file names (Continued)

Path and File Name for Book and Worksheets Adobe Acrobat Reader .pdf File or Microsoft Word File

Worksheets

Worksheet for Creating Threshold Classes Documents\CreatingThresholdClasses.cm.doc

Worksheet for Customized Threshold Class Document\CustomizedThresholdClass.cm.doc

Conventions Used in This Guide

The following conventions are used: Table 2. Conventions used in this guide

This convention . . . Stands for . . . Italics • Names of books as references • New terms or emphasis • Names of executable files • Labels on graphics Bold • User input • Names of commands Upper- and lowercase Names of keys ALL UPPERCASE • Environment variables • Registers ALL UPPERCASE ITALIC Parameter names BOLD UPPERCASE Registry keys [email protected] An email address Hyperlink An Internet link (position the cursor on this word and click the left mouse button) Monospace • Paths • Directories and subdirectories • File names and extensions • Specified user account names • Code sample, including keywords and variables within text and as separate paragraphs, and user-defined program elements within text

Assentor Operations Guide for Content Manager for Message Monitoring and Retention xi CHAPTER 1 DB2 Logging, Backups, and Configuration

This chapter explains how to provide maintenance and recoverability of the Assentor DB2 database for an iLumin-based CM for MMR solution. This database is a component that maintains users, groups, content analysis instructions, and mail processor workflow.

Architecture of DB Logging and Backup Alternatives

Logging and backup in DB2 go hand in hand. The two choices for logging are circular logging and archive logging.

Archive Logging

With archive logging, new log files are created as required and are usually archived to TSM or another location for safekeeping and to avoid filling up the log file system. Archive logging allows all recovery options supported by DB2.

Circular Logging

With circular logging, a fixed set of log files are used on a wraparound basis. Because circular logging causes old log records to be overwritten as the log wraps around, it cannot be used to bring the database up to date after a database Restore is done. Circular logging allows recovery only from such events as power failures, which require only the current log contents.

Backup Alternatives

The choices for DB2 backups are: • Offline vs. Online • Database vs. Tablespace • Full vs. Incremental vs. Incremental Delta

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 1 Configuring the Assentor Compliance Database

Installation and Configuration of DB2

To install and configure a DB2 CM for MMR system, refer to the guide, IBM CM for MMR Installation and Configuration Assistance, which summarizes the procedures for setting up the system. That guide: • Assists you in defining, deploying, and implementing the system • Instructs you on the order in which to install the software components • Provides configuration hints

It is recommended that you completely review the guide before starting installation and operation of the DB2 CM for MMR solution.

The IBM CM for MMR Installation and Configuration Assistant Guide is intended as a reference for an experienced Content Management installer. It is also intended to be used in conjunction with the installation guides provided with each of the referenced software components. The information that the guide provides represents one way to install the products and should not be interpreted to represent the only way to install the CM for MMR solution. At each installation you must consider the customer requirements and environment, as well as the hardware available at the site, in determining how best to set up and configure the DB2 database to interface with the Assentor Compliance system.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 2 CHAPTER 2 Performing Preventive Maintenance

This chapter discusses the maintenance responsibilities for Assentor* Compliance. The maintenance tasks are discussed according to how frequently they should be performed: daily, weekly, and monthly. See Table 3 for a checklist of all of the maintenance tasks that are discussed.

To ensure that Assentor Compliance continues to perform accurately and efficiently, use this chapter as a guide for preventive maintenance.

Performing Daily Preventive Maintenance

iLumin suggests that you perform the following tasks on a daily basis to ensure that Assentor Compliance is running properly: • Monitor the file system queues • Monitor the services • Check that current messages are available • Review the log files

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 3 Performing Preventive Maintenance

1. Monitor the File System Queues

Assentor Compliance stores mail messages in the file system. As messages move through the system, they appear in different folders or queues.

Review the contents of the following queues for an important indication of system operation. Monitor the queues frequently to ensure that they are not backing up. At least 600 files in any directory should be investigated. • %DATALOCATION%msw\internet\Addressing • %DATALOCATION%\msw\internet\Pending • %DATALOCATION%\msw\internet\Holding • %DATALOCATION%\msw\internet\Incoming

You may wish to automate some monitoring tasks. Tools such as Queue Watcher can be configured to execute batch files or command line statements to notify you which queues have reached specified thresholds. If you are using Queue Watcher, verify that the queues are below 100.

2. Monitor the Services

Monitor the Services periodically. No service should be restarted by a monitoring tool, and if a service remains stopped for more than 10 minutes, an alert should go out. Monitor the following: • User Interface > IIS Admin Service > Simple Mail Transport Protocol (SMTP) • CMService • CMSearch • CMQueue • CMIndex • CMComplianceCOM • Mail Processor Services > Assentor Address Processor > Assentor Bloomberg Processors, optional > Assentor Collator > Assentor Controller > Assentor IM Link, optional > Assentor Mail Engine, multiple instances are generally started > Assentor Quarantine Manager > iLumin Delivery Service

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 4 Performing Preventive Maintenance

> AssentorMIME > CMStore > CMAdd > Simple Mail Transport Protocol (SMTP)

It is important to be aware of any tools that may cause conflicts with other services. For example, DameWare, PCAnywhere or any remote administration tools may cause a conflict and should be stopped and disabled during software upgrades and installations.

3. Check That Current Messages Are Available

Perform the following checks to ensure that current messages appear in the Assentor Compliance interface each day:

Check for messages in Assentor Compliance: 1. Logon to Assentor Compliance. 2. Select All Messages in the Browse Messages page. 3. Go to the end of the list and verify that the process date is current.

4. Review the Log Files

Review the following logs: • Check the MIMESweeper log. Review: d:\msw\log\error.log for current errors. Verify that there are no consistent errors. The errors that appear in this log are varied and usually do not indicate a problem. However, if you note repeated errors in close time proximity to each other or if the message delivery queue (\msw\internet\domains) continues to grow, there may be a network or configuration problem. If the file does not exist, that means no errors were reported. • Check the current d:\program files\assentor\logs\archivesubsystem_x.log to verify that archiving, migrating and purging were all performed without errors. • Review the Assentor Compliance Log Files. You can configure Assentor logging to provide a log for a specific component or set of components. Further, you can also configure the number of files per component set and the size of the log file.

Assentor is configured to store all log information in the \Program Files\Assentor\Logs directory. This directory contains different collections of logs, or log file groups. Files within a specific log file group contain the log name and log file number. For example, the names of the logs for the Assentor Controller are AssentorController_1.log, AssentorController_2.log, etc. Each log group contains a .lgc file. This file specifies the current or active log in the group. For example, if the AssentorController.lgc file specifies AssentorController_2.log, then this is the file that contains the most recent information.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 5 Performing Preventive Maintenance

Review each of the log file groups for errors that may be occurring on the system. Messages that indicate errors contain the string “EROR”. • Check Application and System Events. Using the Windows** 2000 Event Viewer, review the Application and System log for messages. In particular, note any messages from SQL Server, Assentor Controller, and Assentor Decomposition Engine. • Check Content Manager Logs. The following is an example of the Content Manager log: 20050120.11:33:51.965|DBUG|CM-PM|ContentManager|003100|003104|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:33:52.184|DBUG|CM-PM|ContentManager|003048|003056|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:33:52.324|INFO|CM-PM|ContentManager|003100|002984|CM DataStore = ICMNLSDB, user = icmadmin, password = Password1 20050120.11:33:52.324|INFO|CM-PM|ContentManager|003100|002984|CM ParType = single, ParName = iMail 20050120.11:33:52.324|INFO|CM-PM|ContentManager|003100|002984|CM IndexType = 0 20050120.11:33:52.340|DBUG|CM-PM|ContentManager|002960|003260|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:33:52.449|DBUG|CM-PM|ContentManager|003032|003028|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:33:52.637|DBUG|CM-PM|ContentManager|002940|000840|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:33:52.746|INFO|CM-PM|ContentManager|003032|003004|CMSTORE Get record from message_done -- external Id = 20050120.11:33:52.887|DBUG|CM-PM|ContentManager|003156|002944|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:33:53.121|DBUG|CM-PM|ContentManager|002952|002508|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:33:53.309|DBUG|CM-PM|ContentManager|002300|002488|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:33:53.449|INFO|CM-PM|ContentManager|002300|002452|CM DataStore = ICMNLSDB, user = icmadmin, password = Password1 20050120.11:33:53.449|INFO|CM-PM|ContentManager|002300|002452|CM ParType = single, ParName = iMail 20050120.11:33:53.449|INFO|CM-PM|ContentManager|002300|002452|CM IndexType = 0 20050120.11:33:53.527|DBUG|CM-PM|ContentManager|002324|002384|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:33:53.543|INFO|CM-PM|ContentManager|002324|002472|CM DataStore = ICMNLSDB, user = icmadmin, password = Password1 20050120.11:33:53.543|INFO|CM-PM|ContentManager|002324|002472|CM ParType = single, ParName = iMail 20050120.11:33:53.762|INFO|CM-PM|ContentManager|003032|003004|CMSTORE Get record from message_done -- external Id = 20050120.11:33:53.793|INFO|CM-PM|ContentManager|003032|003004|CMSTORE Get record from message_done -- external Id = 20050120.11:33:54.574|DBUG|CM-PM|ContentManager|002320|003860|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:33:55.043|DBUG|CM-PM|ContentManager|002176|002260|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:33:59.074|DBUG|CM-PM|ContentManager|002220|002212|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:01.058|DBUG|CM-PM|ContentManager|002112|002088|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:03.074|DBUG|CM-PM|ContentManager|003068|003000|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:05.105|DBUG|CM-PM|ContentManager|002948|002308|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:05.230|DBUG|CM-PM|ContentManager|002052|002096|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:05.668|DBUG|CM-PM|ContentManager|002060|002048|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:06.183|DBUG|CM-PM|ContentManager|003484|001492|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:06.668|DBUG|CM-PM|ContentManager|001472|001744|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<*****

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 6 Performing Preventive Maintenance

20050120.11:34:07.105|DBUG|CM-PM|ContentManager|003748|002672|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:07.230|DBUG|CM-PM|ContentManager|002824|003772|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:07.683|DBUG|CM-PM|ContentManager|003324|002328|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:08.152|DBUG|CM-PM|ContentManager|001716|001944|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:08.652|DBUG|CM-PM|ContentManager|001980|002008|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:09.043|DBUG|CM-PM|ContentManager|001464|002676|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:09.168|DBUG|CM-PM|ContentManager|003712|001652|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:09.652|DBUG|CM-PM|ContentManager|001580|002688|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:11.058|DBUG|CM-PM|ContentManager|001484|001628|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:13.043|DBUG|CM-PM|ContentManager|002796|002816|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:15.043|DBUG|CM-PM|ContentManager|002020|002912|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:17.058|DBUG|CM-PM|ContentManager|002664|003268|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:19.042|DBUG|CM-PM|ContentManager|001952|003340|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:21.058|DBUG|CM-PM|ContentManager|001644|001920|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:34:23.058|DBUG|CM-PM|ContentManager|001684|001892|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:38:33.303|INFO|CM-PM|ContentManager|003100|003052|CMCOMPLIANCE Compliance Application 20050120.11:38:41.022|INFO|CM-PM|ContentManager|003100|003052|> >: getMessage: ---End ---: P0000172605@FUN2MP 20050120.11:38:41.069|INFO|CM-PM|ContentManager|003100|003052|CMCOMPLIANCE Message file = [email protected], Blob Type = 3 20050120.11:38:51.068|DBUG|CM-PM|ContentManager|003856|003424|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.11:38:52.881|INFO|CM-PM|ContentManager|003100|003052|> >: getMessage: ---End ---: P0000000010@CM-PM 20050120.11:38:52.928|INFO|CM-PM|ContentManager|003100|003052|CMCOMPLIANCE Message file = [email protected], Blob Type = 1 20050120.11:38:58.850|INFO|CM-PM|ContentManager|003100|001988|> >: getMessage: ---End ---: P0000000009@CM-PM 20050120.11:38:58.896|INFO|CM-PM|ContentManager|003100|001988|CMCOMPLIANCE Message file = [email protected], Blob Type = 1 20050120.11:39:05.740|INFO|CM-PM|ContentManager|003100|001636|> >: getMessage: ---End ---: P0000000007@CM-PM 20050120.11:39:05.787|INFO|CM-PM|ContentManager|003100|001636|CMCOMPLIANCE Message file = [email protected], Blob Type = 1 20050120.14:42:32.747|INFO|CM-PM|ContentManager|003032|003004|CMSTORE Get record from message_done -- external Id = 20050120.14:44:45.542|DBUG|CM-PM|ContentManager|001980|003804|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.14:44:45.651|INFO|CM-PM|ContentManager|001980|003256|CMSTORE Get record from message_done -- external Id = 20050120.14:45:48.369|DBUG|CM-PM|ContentManager|002328|003324|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.14:45:58.447|DBUG|CM-PM|ContentManager|003772|003360|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.14:45:58.947|DBUG|CM-PM|ContentManager|001744|001472|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.14:45:59.447|DBUG|CM-PM|ContentManager|001492|003484|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.14:45:59.634|DBUG|CM-PM|ContentManager|002044|002096|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.14:45:59.947|DBUG|CM-PM|ContentManager|002840|003028|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<*****

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 7 Performing Preventive Maintenance

20050120.14:46:00.447|DBUG|CM-PM|ContentManager|003004|003580|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.14:46:00.978|DBUG|CM-PM|ContentManager|003016|002968|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.14:46:01.447|DBUG|CM-PM|ContentManager|003860|002320|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.14:46:01.962|DBUG|CM-PM|ContentManager|002264|002148|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.14:46:02.462|DBUG|CM-PM|ContentManager|003696|002240|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.14:46:02.962|DBUG|CM-PM|ContentManager|003820|001652|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.14:50:06.645|INFO|CM-PM|ContentManager|003100|003052|CMCOMPLIANCE Compliance Application 20050120.14:50:12.442|INFO|CM-PM|ContentManager|003100|003052|> >: getMessage: ---End ---: P0000172605@FUN2MP 20050120.14:50:12.489|INFO|CM-PM|ContentManager|003100|003052|CMCOMPLIANCE Message file = [email protected], Blob Type = 3 20050120.14:50:16.848|DBUG|CM-PM|ContentManager|001984|002652|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<***** 20050120.14:50:22.957|INFO|CM-PM|ContentManager|003100|003052|> >: getMessage: ---End ---: P0000000012@CM-PM 20050120.14:50:23.004|INFO|CM-PM|ContentManager|003100|003052|CMCOMPLIANCE Message file = [email protected], Blob Type = 1 20050120.14:50:30.910|INFO|CM-PM|ContentManager|003100|000508|> >: getMessage: ---End ---: P0000000011@CM-PM 20050120.14:50:30.957|INFO|CM-PM|ContentManager|003100|000508|CMCOMPLIANCE Message file = [email protected], Blob Type = 1 20050120.14:50:41.347|INFO|CM-PM|ContentManager|003100|003052|> >: getMessage: ---End ---: P0000000012@CM-PM 20050120.14:50:41.379|INFO|CM-PM|ContentManager|003100|003052|CMCOMPLIANCE Message file = [email protected], Blob Type = 1 20050120.14:59:41.993|INFO|CM-PM|ContentManager|003100|003052|CMCOMPLIANCE Compliance Application 20050120.14:59:47.978|INFO|CM-PM|ContentManager|003100|003052|> >: getMessage: ---End ---: P0000172605@FUN2MP 20050120.14:59:48.056|INFO|CM-PM|ContentManager|003100|003052|CMCOMPLIANCE Message file = [email protected], Blob Type = 3 20050120.14:59:57.852|INFO|CM-PM|ContentManager|003100|003052|> >: getMessage: ---End ---: P0000000012@CM-PM 20050120.14:59:57.899|INFO|CM-PM|ContentManager|003100|003052|CMCOMPLIANCE Message file = [email protected], Blob Type = 1 20050120.15:14:53.382|INFO|CM-PM|ContentManager|003100|000508|> >: getMessage: ---End ---: P0000000012@CM-PM 20050120.15:14:53.476|INFO|CM-PM|ContentManager|003100|000508|CMCOMPLIANCE Message file = [email protected], Blob Type = 1 20050120.15:15:03.976|INFO|CM-PM|ContentManager|003100|002236|> >: getMessage: ---End ---: P0000000011@CM-PM 20050120.15:15:04.007|INFO|CM-PM|ContentManager|003100|002236|CMCOMPLIANCE Message file = [email protected], Blob Type = 1 20050120.15:15:08.163|INFO|CM-PM|ContentManager|003100|000508|> >: getMessage: ---End ---: P0000000009@CM-PM 20050120.15:15:08.210|INFO|CM-PM|ContentManager|003100|000508|CMCOMPLIANCE Message file = [email protected], Blob Type = 1 20050120.15:59:07.831|----|CM-PM|ContentManager|003048|003060|> >: : 20050120.15:59:07.831|----|CM-PM|ContentManager|003048|003060|> >: : execute getFromCM 20050120.15:59:07.831|----|CM-PM|ContentManager|003048|003060|> >: : XXXXXXXXXXXXXXXXXXXXXXXXXXXX 20050120.15:59:07.831|----|CM-PM|ContentManager|003048|003060|> >: : X !!! Exception !!! X 20050120.15:59:07.831|----|CM-PM|ContentManager|003048|003060|> >: : XXXXXXXXXXXXXXXXXXXXXXXXXXXX 20050120.15:59:07.831|----|CM-PM|ContentManager|003048|003060|> >: : Name: DKLogonFailure 20050120.15:59:07.831|----|CM-PM|ContentManager|003048|003060|> >: : Message ID: 394 20050120.15:59:07.831|----|CM-PM|ContentManager|003048|003060|> >: : Error State: 08001 20050120.15:59:07.831|----|CM-PM|ContentManager|003048|003060|> >: : Error Code: -30081 20050120.15:59:07.831|----|CM-PM|ContentManager|003048|003060|> >: : Location 0: DKDatastoreICM.cpp::[8067]

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 8 Performing Preventive Maintenance

20050120.15:59:07.831|----|CM-PM|ContentManager|003048|003060|> >: : Message 0: [IBM][CLI Driver] SQL30081N A communication error has been detected. being used: "TCP/IP". Communication API being used: "SOCKETS". Location where the error was detected: "". Communication function detecting the error: "connect". Protocol specific error code(s): "10060", "*", "*". SQLSTATE=08001 [SQLDriverConnect] [SQLCODE=-30081,SQLSTATE=08001][SERVER=ICMNLSDB:USERID=icmadmin] (STATE) : 08001 20050120.15:59:07.831|----|CM-PM|ContentManager|003048|003060|> >: : ------20050120.15:59:07.831|EROR|CM-PM|ContentManager|003048|003060|> >: execute getFromCM: DKException. query='/iFolder[@aHost="CM-PM" AND @aStatus="INDEXING"]' 20050120.15:59:53.768|----|CM-PM|ContentManager|003048|003060|> >: : 20050120.15:59:53.768|----|CM-PM|ContentManager|003048|003060|> >: : disconnect 20050120.15:59:53.768|----|CM-PM|ContentManager|003048|003060|> >: : XXXXXXXXXXXXXXXXXXXXXXXXXXXX 20050120.15:59:53.768|----|CM-PM|ContentManager|003048|003060|> >: : X !!! Exception !!! X 20050120.15:59:53.768|----|CM-PM|ContentManager|003048|003060|> >: : XXXXXXXXXXXXXXXXXXXXXXXXXXXX 20050120.15:59:53.768|----|CM-PM|ContentManager|003048|003060|> >: : Name: DKLogonFailure 20050120.15:59:53.768|----|CM-PM|ContentManager|003048|003060|> >: : Message ID: 394 20050120.15:59:53.768|----|CM-PM|ContentManager|003048|003060|> >: : Error State: 08001 20050120.15:59:53.768|----|CM-PM|ContentManager|003048|003060|> >: : Error Code: -30081 20050120.15:59:53.768|----|CM-PM|ContentManager|003048|003060|> >: : Location 0: DKDatastoreICM.cpp::[8067] 20050120.15:59:53.768|----|CM-PM|ContentManager|003048|003060|> >: : Message 0: [IBM][CLI Driver] SQL30081N A communication error has been detected. Communication protocol being used: "TCP/IP". Communication API being used: "SOCKETS". Location where the error was detected: "". Communication function detecting the error: "connect". Protocol specific error code(s): "10060", "*", "*". SQLSTATE=08001 [SQLDriverConnect] [SQLCODE=-30081,SQLSTATE=08001][SERVER=ICMNLSDB:USERID=icmadmin] (STATE) : 08001 20050120.15:59:53.768|----|CM-PM|ContentManager|003048|003060|> >: : ------20050120.15:59:53.768|EROR|CM-PM|ContentManager|003048|003060|> >: disconnect: DKException 20050120.16:01:06.501|----|CM-PM|ContentManager|003048|003060|> >: : 20050120.16:01:06.501|----|CM-PM|ContentManager|003048|003060|> >: : 20050120.16:01:06.501|----|CM-PM|ContentManager|003048|003060|> >: : XXXXXXXXXXXXXXXXXXXXXXXXXXXX 20050120.16:01:06.501|----|CM-PM|ContentManager|003048|003060|> >: : X !!! Exception !!! X 20050120.16:01:06.501|----|CM-PM|ContentManager|003048|003060|> >: : XXXXXXXXXXXXXXXXXXXXXXXXXXXX 20050120.16:01:06.501|----|CM-PM|ContentManager|003048|003060|> >: : Name: DKDatabaseLogonFailure 20050120.16:01:06.501|----|CM-PM|ContentManager|003048|003060|> >: : Message ID: 394 20050120.16:01:06.501|----|CM-PM|ContentManager|003048|003060|> >: : Error State: 08001 20050120.16:01:06.501|----|CM-PM|ContentManager|003048|003060|> >: : Error Code: -30081 20050120.16:01:06.501|----|CM-PM|ContentManager|003048|003060|> >: : Location 0: DKDatastoreICM.cpp::[2171] 20050120.16:01:06.501|----|CM-PM|ContentManager|003048|003060|> >: : Message 0: [IBM][CLI Driver] SQL30081N A communication error has been detected. Communication protocol being used: "TCP/IP". Communication API being used: "SOCKETS". Location where the error was detected: "". Communication function detecting the error: "connect". Protocol specific error code(s): "10061", "*", "*". SQLSTATE=08001 [SQLDriverConnect] [SQLCODE=-30081,SQLSTATE=08001][SERVER=ICMNLSDB:USERID=icmconct] (STATE) : 08001 20050120.16:01:06.501|----|CM-PM|ContentManager|003048|003060|> >: : ------20050120.16:01:06.501|EROR|CM-PM|ContentManager|003048|003060|> >: connect: DKException: ICMNLSDB icmadmin Password1 20050120.16:01:07.501|----|CM-PM|ContentManager|003048|003060|> >: : 20050120.16:01:07.501|----|CM-PM|ContentManager|003048|003060|> >: : execute getFromCM 20050120.16:01:07.501|----|CM-PM|ContentManager|003048|003060|> >: : XXXXXXXXXXXXXXXXXXXXXXXXXXXX 20050120.16:01:07.501|----|CM-PM|ContentManager|003048|003060|> >: : X !!! Exception !!! X 20050120.16:01:07.501|----|CM-PM|ContentManager|003048|003060|> >: : XXXXXXXXXXXXXXXXXXXXXXXXXXXX 20050120.16:01:07.501|----|CM-PM|ContentManager|003048|003060|> >: : Name: DKDatastoreError 20050120.16:01:07.501|----|CM-PM|ContentManager|003048|003060|> >: : Message ID: 310 20050120.16:01:07.501|----|CM-PM|ContentManager|003048|003060|> >: : Error State: 20050120.16:01:07.501|----|CM-PM|ContentManager|003048|003060|> >: : Error Code: 0 20050120.16:01:07.501|----|CM-PM|ContentManager|003048|003060|> >: : Location 0: DKDatastoreICM.cpp::[3224]

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 9 Performing Preventive Maintenance

20050120.16:01:07.501|----|CM-PM|ContentManager|003048|003060|> >: : Message 0: DGL0310A: Connection not found 20050120.16:01:07.501|----|CM-PM|ContentManager|003048|003060|> >: : ------20050120.16:01:07.501|EROR|CM-PM|ContentManager|003048|003060|> >: execute getFromCM: DKException. query='/iFolder[@aHost="CM-PM" AND @aStatus IN ("UPDATING")]' 20050120.16:01:08.798|----|CM-PM|ContentManager|002324|002472|> >: : 20050120.16:01:08.798|----|CM-PM|ContentManager|002324|002472|> >: : execute getDiscoveryWork 20050120.16:01:08.798|----|CM-PM|ContentManager|002324|002472|> >: : XXXXXXXXXXXXXXXXXXXXXXXXXXXX 20050120.16:01:08.798|----|CM-PM|ContentManager|002324|002472|> >: : X !!! Exception !!! X 20050120.16:01:08.798|----|CM-PM|ContentManager|002324|002472|> >: : XXXXXXXXXXXXXXXXXXXXXXXXXXXX 20050120.16:01:08.798|----|CM-PM|ContentManager|002324|002472|> >: : Name: DKLogonFailure 20050120.16:01:08.798|----|CM-PM|ContentManager|002324|002472|> >: : Message ID: 394 20050120.16:01:08.798|----|CM-PM|ContentManager|002324|002472|> >: : Error State: 08001 20050120.16:01:08.798|----|CM-PM|ContentManager|002324|002472|> >: : Error Code: -30081 20050120.16:01:08.798|----|CM-PM|ContentManager|002324|002472|> >: : Location 0: DKDatastoreICM.cpp::[8067] 20050120.16:01:08.798|----|CM-PM|ContentManager|002324|002472|> >: : Message 0: [IBM][CLI Driver] SQL30081N A communication error has been detected. Communication protocol being used: "TCP/IP". Communication API being used: "SOCKETS". Location where the error was detected: "". Communication function detecting the error: "connect". Protocol specific error code(s): "10060", "*", "*". SQLSTATE=08001 [SQLDriverConnect] [SQLCODE=-30081,SQLSTATE=08001][SERVER=ICMNLSDB:USERID=icmadmin] (STATE) : 08001 20050120.16:01:08.798|----|CM-PM|ContentManager|002324|002472|> >: : ------20050120.16:01:08.798|EROR|CM-PM|ContentManager|002324|002472|> >: execute getDiscoveryWork: DKException. query='/iDisWork[@aHost="CM-PM" AND @aWorkType=2]' 20050120.16:01:10.579|----|CM-PM|ContentManager|003048|003060|> >: : 20050120.16:01:10.579|----|CM-PM|ContentManager|003048|003060|> >: : 20050120.16:01:10.579|----|CM-PM|ContentManager|003048|003060|> >: : XXXXXXXXXXXXXXXXXXXXXXXXXXXX 20050120.16:01:10.579|----|CM-PM|ContentManager|003048|003060|> >: : X !!! Exception !!! X 20050120.16:01:10.579|----|CM-PM|ContentManager|003048|003060|> >: : XXXXXXXXXXXXXXXXXXXXXXXXXXXX 20050120.16:01:10.579|----|CM-PM|ContentManager|003048|003060|> >: : Name: DKDatabaseLogonFailure 20050120.16:01:10.579|----|CM-PM|ContentManager|003048|003060|> >: : Message ID: 394 20050120.16:01:10.579|----|CM-PM|ContentManager|003048|003060|> >: : Error State: 08001 20050120.16:01:10.579|----|CM-PM|ContentManager|003048|003060|> >: : Error Code: -30082 20050120.16:01:10.579|----|CM-PM|ContentManager|003048|003060|> >: : Location 0: DKDatastoreICM.cpp::[2171] 20050120.16:01:10.579|----|CM-PM|ContentManager|003048|003060|> >: : Message 0: [IBM][CLI Driver] SQL30082N Attempt to establish connection failed with security reason "24" ("USERNAME AND/OR PASSWORD INVALID"). SQLSTATE=08001 [SQLDriverConnect] [SQLCODE=-30082,SQLSTATE=08001][SERVER=ICMNLSDB:USERID=icmconct] (STATE) : 08001 20050120.16:01:10.579|----|CM-PM|ContentManager|003048|003060|> >: : ------20050120.16:01:10.579|EROR|CM-PM|ContentManager|003048|003060|> >: connect: DKException: ICMNLSDB icmadmin Password1 20050120.16:01:16.282|----|CM-PM|ContentManager|002324|002472|> >: : 20050120.16:01:16.282|----|CM-PM|ContentManager|002324|002472|> >: : disconnect 20050120.16:01:16.282|----|CM-PM|ContentManager|002324|002472|> >: : XXXXXXXXXXXXXXXXXXXXXXXXXXXX 20050120.16:01:16.282|----|CM-PM|ContentManager|002324|002472|> >: : X !!! Exception !!! X 20050120.16:01:16.282|----|CM-PM|ContentManager|002324|002472|> >: : XXXXXXXXXXXXXXXXXXXXXXXXXXXX 20050120.16:01:16.282|----|CM-PM|ContentManager|002324|002472|> >: : Name: DKDatastoreAccessError 20050120.16:01:16.282|----|CM-PM|ContentManager|002324|002472|> >: : Message ID: 394 20050120.16:01:16.282|----|CM-PM|ContentManager|002324|002472|> >: : Error State: HY010 20050120.16:01:16.282|----|CM-PM|ContentManager|002324|002472|> >: : Error Code: -99999 20050120.16:01:16.282|----|CM-PM|ContentManager|002324|002472|> >: : Location 0: DKDatastoreICM.cpp::[2662] 20050120.16:01:16.282|----|CM-PM|ContentManager|002324|002472|> >: : Message 0: [IBM][CLI Driver] CLI0125E Function sequence error. SQLSTATE=HY010[SQLFreeHandle] [SQLCODE=-99999,SQLSTATE=HY010] (STATE) : HY010 20050120.16:01:16.282|----|CM-PM|ContentManager|002324|002472|> >: : ------20050120.16:01:16.282|EROR|CM-PM|ContentManager|002324|002472|> >: disconnect: DKException

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 10 Performing Preventive Maintenance

20050120.16:01:30.578|----|CM-PM|ContentManager|003048|003060|> >: : 20050120.16:01:30.578|----|CM-PM|ContentManager|003048|003060|> >: : execute getFromCM 20050120.16:01:30.578|----|CM-PM|ContentManager|003048|003060|> >: : XXXXXXXXXXXXXXXXXXXXXXXXXXXX 20050120.16:01:30.578|----|CM-PM|ContentManager|003048|003060|> >: : X !!! Exception !!! X 20050120.16:01:30.578|----|CM-PM|ContentManager|003048|003060|> >: : XXXXXXXXXXXXXXXXXXXXXXXXXXXX 20050120.16:01:30.578|----|CM-PM|ContentManager|003048|003060|> >: : Name: DKDatastoreError 20050120.16:01:30.578|----|CM-PM|ContentManager|003048|003060|> >: : Message ID: 310 20050120.16:01:30.578|----|CM-PM|ContentManager|003048|003060|> >: : Error State: 20050120.16:01:30.578|----|CM-PM|ContentManager|003048|003060|> >: : Error Code: 0 20050120.16:01:30.578|----|CM-PM|ContentManager|003048|003060|> >: : Location 0: DKDatastoreICM.cpp::[3224] 20050120.16:01:30.578|----|CM-PM|ContentManager|003048|003060|> >: : Message 0: DGL0310A: Connection not found 20050120.16:01:30.578|----|CM-PM|ContentManager|003048|003060|> >: : ------20050120.16:01:30.578|EROR|CM-PM|ContentManager|003048|003060|> >: execute getFromCM: DKException. query='/iFolder[@aHost="CM-PM" AND @aStatus="INDEXING"]' 20050120.16:06:58.900|INFO|CM-PM|ContentManager|002300|002440|CMDISCOVERY Discovery Application 20050120.16:08:22.867|INFO|CM-PM|ContentManager|003100|002704|CMCOMPLIANCE Compliance Application 20050120.16:08:29.508|INFO|CM-PM|ContentManager|003100|002704|> >: getMessage: ---End ---: P0000172605@FUN2MP 20050120.16:08:29.602|INFO|CM-PM|ContentManager|003100|002704|CMCOMPLIANCE Message file = [email protected], Blob Type = 3 20050120.16:12:02.051|INFO|CM-PM|ContentManager|002300|002688|CMDISCOVERY Discovery Application 20050120.16:12:19.847|INFO|CM-PM|ContentManager|002300|002688|CMDISCOVERY Discovery Application 20050120.16:21:09.196|INFO|CM-PM|ContentManager|003100|002704|> >: getMessage: ---End ---: P0000172566@FUN2MP 20050120.16:21:09.243|INFO|CM-PM|ContentManager|003100|002704|CMCOMPLIANCE Message file = [email protected], Blob Type = 1 20050120.16:21:16.056|INFO|CM-PM|ContentManager|003100|002704|> >: getMessage: ---End ---: P0000172502@FUN2MP 20050120.16:21:16.118|INFO|CM-PM|ContentManager|003100|002704|CMCOMPLIANCE Message file = [email protected], Blob Type = 1 20050120.16:21:22.040|INFO|CM-PM|ContentManager|003100|002048|> >: getMessage: ---End ---: P0000172605@FUN2MP 20050120.16:21:22.087|INFO|CM-PM|ContentManager|003100|002048|CMCOMPLIANCE Message file = [email protected], Blob Type = 3 20050120.16:21:28.134|INFO|CM-PM|ContentManager|003100|002704|> >: getMessage: ---End ---: P0000172566@FUN2MP 20050120.16:21:28.212|INFO|CM-PM|ContentManager|003100|002704|CMCOMPLIANCE Message file = [email protected], Blob Type = 1 20050120.16:21:38.836|INFO|CM-PM|ContentManager|003100|002704|> >: getMessage: ---End ---: P0000172605@FUN2MP 20050120.16:21:38.899|INFO|CM-PM|ContentManager|003100|002704|CMCOMPLIANCE Message file = [email protected], Blob Type = 3 20050120.16:25:34.754|INFO|CM-PM|ContentManager|002300|002244|> >: getMessage: ---End ---: P0000000006@CM-PM 20050120.16:25:34.801|INFO|CM-PM|ContentManager|002300|002244|CMCOMPLIANCE Message file = [email protected], Blob Type = 1

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 11 Performing Preventive Maintenance

Performing Weekly Preventive Maintenance

iLumin suggests that you perform the following tasks weekly: • Check the disk and log space used • Check scheduled tasks • Delete temporary files

1. Delete Temporary Files

Assentor uses the INSO Outside In** product for conversion of attachments to plain text. Temporary files are created by Outside In during the conversion process.

If Outside In encounters a problem during conversion, it may not delete temporary files from the \Winnt directory. Over time, the number of temporary files may increase and could fill the system partition. To avoid this situation, stop the Assentor Controller, and then delete all TMP*.TMP files from the \Winnt directory.

2. Delete Outdated Logs

iLumin suggests that you should periodically delete the outdated logs to ensure that the system does not become overburdened with files. The firm may wish to set a policy for how often these logs should be deleted.

Maintenance Checklist for Assentor Compliance Backups

Table 3 provides a check list of the tasks that should be performed in Assentor Compliance.

Table 3. Maintenance checklist for Assentor Compliance

Frequency Task

Daily Monitor the file system queues: • %DATALOCATION%msw\internet\Addressing • %DATALOCATION%\msw\internet\Pending • %DATALOCATION%\msw\internet\Holding • %DATALOCATION%\msw\internet\Incoming

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 12 Performing Preventive Maintenance

Table 3. Maintenance checklist for Assentor Compliance (Continued)

Frequency Task

Monitor the services: • User Interface • IIS Admin Service • Simple Mail Transport Protocol (SMTP) • Archiver Server Services • Assentor Controller, optional • LDAP Dump, optional • Simple Mail Transport Protocol (SMTP) • Mail Processor Services • Assentor Address Processor • Assentor Bloomberg Processor, optional • Assentor Bond Desk Processor, optional • Assentor Collator • Assentor Controller • Assentor IM Link, optional • Assentor Mail Engine, multiple instances are generally started • Assentor Quarantine Manager • iLumin Delivery Service • AssentorMIME • Simple Mail Transport Protocol (SMTP)

Check that current messages are available: • Check in Browse Messages for current date • Review the Archive Manager log: \Program Files\Assentor\Logs

Check for free space

Review the log files • Check the Assentor Compliance logs: \Program Files\Assentor\Logs • Check application and system files

Weekly Check the disk and log space used and check available disk space on the system

Check scheduled tasks to see if Assentor update statistics tasks completed successfully

Delete temporary files from \Winnt directory

Periodically delete the log files based on the firm’s policy.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 13 CHAPTER 3 Using Log Files For Problem Analysis

To provide information about message processing and about errors that may occur, Assentor* Compliance includes a facility for logging diagnostic messages. This chapter describes the format of Assentor Compliance log messages and discusses how to manage, configure, and view log files.

Understanding Log Messages

Each component of Assentor Compliance has a unique name that appears in the log file to easily identify the origin of a log message. Components may log files, or they may be assigned their own log file as needed.

Assentor Compliance adds header information to each log message as it is produced. Figure 1 describes the format of a log message. Each log file contains one or more log messages, and each log message ends with a new line character.

Figure 1. Log message format

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 14 Using Log Files for Problem Analysis

Assentor Compliance assigns each log message a severity level that indicates the relative importance of the message. To allow filtering of log messages based on the message severity levels, you can configure a severity threshold for log messages. Table 4 lists the predefined severity levels for the log messages.

Table 4. Predefined severity levels for log messages

Log Level Rank Text Description

none 0 -- Use as threshold value to record no log messages

emergency 100 EMRG Indicates a panic condition; the system is unusable.

alert 200 ALRT Indicates a condition that should be corrected immediately; for example a corrupted database.

critical 300 CRIT Indicates a critical system condition, a failure that causes termination of normal processing and prevents future processing as well; for example: disk full.

error 400 EROR Indicates error condition. For example: file not found.

warning 500 WARN Indicates a warning message. Does not cause termination of normal processing. For example: File larger than expected.

notice 600 NOTE Indicates a condition requiring special handling, but not an error condition. Used when something special must be done, such as an SNMP trap set or email message sent.

info 700 INFO Indicates general information message. Used to output information that may be helpful to determine the status of a running system. For example: File Processed.

debugged 1000 DBGL Provides brief debug information. For example: Command line read.

debug 2000 DBUG Used to output more detail than “debug_low.” For example: Data line read.

debug_high 3000 DBGH Very detailed debug information, such as buffer dumps, etc.

all 4000 –– Use as threshold value to record all log messages

The logger writes an initial message to each log file after the file is created or opened. Figure 2 shows an example initial message.

20050208.13:56:28.123|INFO|TOBOLD|LogTester|000285|000123|*****>> Log Opened, threshold = 700, #Generations = 5, MaxLogSize = 2097152 <<*****

Figure 2. Example of initial log message

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 15 Using Log Files for Problem Analysis

Managing Log Files

The logging facility automatically manages log files based on configuration parameters.

A set of log files is associated with each configured log. Each component can have its own log file set, or you can configure multiple components to write to the same log file set concurrently. You can also specify both the number of files in a set and the maximum size for each file in the registry. If not configured in the registry, the default is 5 files in a set, each with a maximum size of 2 Mb.

Log files are named “_n.log,” where is the name of the file set, and n is the log file generation number. You can specify the file set name in the registry. If no registry entry is found, the component name is used for the file set name.

The first time a log file set is opened, the log file name created is _1.log. The next time a log file in the set is created, the generation number is incremented, giving the name _2.log. When a log file reaches its configured maximum size, the system closes the log file and opens the next log file generation in the set. When the last log file generation in the set reaches its configured maximum size, the system opens and reuses the first log file generation in the set.

In addition to the set of log files, a control file is maintained for each log file set. The name of the control file is of the form .lgc. Although this file contains mostly binary data, the path of the current log file in the set is visible when the control file is viewed (such as with Microsoft** Notepad).

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 16 Using Log Files for Problem Analysis

Configuring the Log

Use the Microsoft Registry Editor to configure logging. Each Assentor Compliance component obtains the settings from the registry during component initialization. You may store the registry settings by component name or in the LogDefault settings as described in the footing of Table 5. Component-level registry settings override the LogDefault settings.

Note that you can calculate the amount of disk space used by logging by multiplying the value for LogGenerations by the value for LogMaxSize. The minimum log file size is 10 Kb. The maximum size for a single log file is 2 Mb. Table 5. Logger registry settings

Registry Value Name Description

LogGenerations (DWORD) The number of log files in the set. (Example: “5”)

LogMaxSize (DWORD) The maximum size, in bytes, for each log file in the set. (Example: “10485760” or total 10 Mb for a set of 5)

LogPath (string) The path for the log file set. This path does not include the generation number, but it does include the log file set name. (Example: [Common\LogLocation]\Program Files\Assentor\Logs\MailProcessing.log)

LogThreshold (DWORD) The default log message threshold; for filtering log messages, see table F-1. (Example: “700”)

LogType (DWORD) The type of log message container. Currently only the “File” type is supported. (Example: “1” for File)

Store Registry settings by either of the following: LogDefault key: SOFTWARE\iLumin\ASSENTOR\LogDefault Component key: SOFTWARE\iLumin\ASSENTOR\

Note: Use caution when changing the value of the LogThreshold setting. Setting the value below 700 (“INFO”) makes it difficult to analyze problems since the amount of information collected in the log is greatly reduced. Setting the value above 700 causes much more information to be logged, which can have a performance impact.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 17 Using Log Files for Problem Analysis

Recommended Log Configuration

The following configuration is recommended. Table 6. Recommended log configuration for Assentor

Assentor Component Registry Value Name Name Registry Setting

LogDefault Key: HKEY_LOCAL_MACHINE\SOFTWARE\SRA International\ASSENTOR\LogDefault

LogGenerations 5

LogMaxSize 2097152

LogPath [Common\LogLocation]\Program Files\Assentor\Logs\MailProcessing.log

LogThreshold 700

LogType 1

AddressProcessor Key: HKEY_LOCAL_MACHINE\SOFTWARE\SRA International\ASSENTOR\AddressProcessor

LogPath [Common\LogLocation]\Program Files\Assentor\Logs\AddressProcessor.log

AssentorController Key: HKEY_LOCAL_MACHINE\SOFTWARE\SRA International\ASSENTOR\AssentorController

LogPath [Common\LogLocation]\ProgramFiles\Assentor\Logs\AssentorController.log

BloombergProcessor Key: HKEY_LOCAL_MACHINE\SOFTWARE\SRA International\ASSENTOR\BloombergProcessor

LogPath [Common\LogLocation]\ProgramFiles\Assentor\Logs\BloombergProcessor.log

WebUI Key: HKEY_LOCAL_MACHINE\SOFTWARE\SRA International\ASSENTOR\WebUI

LogPath [Common\LogLocation]\ProgramFiles\Assentor\Logs\WebUI.log

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 18 Using Log Files for Problem Analysis

Viewing Log Files

You can view log files using either Microsoft Notepad or some other editor that can read ASCII text files.

Determining Which Log File Is Currently Active

To determine which log file in the set is the currently active log file, open the log control file (which has the .lgc suffix) in Notepad to display the path of the current log file.

You can also view the currently active log file with Notepad, but there are some considerations. The logging facility uses the NT file memory-mapping features to manage the currently active log file. This requires that the log file be created at its maximum size; the log file remains at its maximum size until it is closed. When the last process connected to the log file closes that log file, the system truncates the log file to the actual size of the data contained within. Sometimes this truncation operation is not performed (for example, due to a process crash) and the log file stays at its maximum configured size.

When a log file is reused (because all of the files in the set were used), residual data from the previous use of the file can appear after the new log data. This can be confusing when viewing the active log file (or a log file that has not been truncated), but you can use the date/time stamp to determine where the old data starts.

The LogUtil program provides the capability to manipulate the log files and to display log information from active log files. Figure 3 describes the syntax of the LogUtil command.

Usage: LogUtil [args]

Command Description ------Display [n] Display log control information [every n seconds] Help Display this information Monitor [n] Monitor the current log file [sleep n seconds] Roll Close current log file and roll to next Snapshot Display the current log file and quit Threshold n Set log threshold to 'n' ThresholdAll n Set log threshold to ‘n’ for all components Truncate Truncate log files to actual size

Figure 3. Syntax of the LogUtil command

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 19 Using Log Files for Problem Analysis

Table 7 lists Assentor Compliance-related log files, their locations, and gives brief descriptions of them. Table 7. Locations of log files

Application Log File Location Description

Address Processor [drive letter]:\Program Logs information about the Address Processor Files\Assentor\Logs\AddressProcessor.log service

Assentor Controller [drive letter]:\Program Assentor Controller error log Files\Assentor\Logs\AssentorController.log

Mail Engine Services [drive letter]:\Program Files\Assentor\logs\mail Mail engine logs processing.log

Collator Service [drive letter]:\Mail processing.log Collator service log

Assentor User [drive letter]:\Program Errors returned from the User Interface web Interface Files\Assentor\Logs\WebUI.log pages

Assentor Validator [drive letter]:\Program Files\Assentor\Logs\mail Errors with the validator DLL processing.log

Assentor Quarantine [drive letter]:\Program Files\Assentor\Logs\mail Assentor Quarantine Manager service errors Manager Service processing.log

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 20 Using Log Files for Problem Analysis

Understanding the Windows 2000 Event Viewer Messages

Table 8 lists messages that are generated by the Assentor Address processor. These messages appear in the Windows 2000 Event log and can be viewed using the Event Viewer.

Table 8. Assentor Address Processor messages

Message ID Severity Symbolic Name Message*

2001 Informational AdminEventAPStarted Assentor Address Processor is starting.

2002 Informational AdminEventAPStopped Assentor Address Processor is stopping.

2003 Warning AdminEventMoveError Error while moving file, see Address Processor log for details.

2004 Warning AdminEventCopyError Error while copying file, see Address Processor log for details.

2005 Error AdminEventMissingINI Bad or missing AssentorAddressProcessor.ini file.

2006 Error AdminEventMissingQueue Defined queue not present: %1.

2007 Warning AdminEventCommitError Error while committing file, see Address Processor log for details.

2008 Warning AdminEventMessageAbandoned Error, abandoning message %1, see Address Processor log for details.

2009 Error AdminEventMissingRequiredQueue Required Queue not present: %1

2010 Error AdminEventMissingRequiredAction Required Action not present: %1.

2012 Informational AdminEventAPPaused Assentor Address Processor is paused.

2013 Informational AdminEventAPResumed Assentor Address Processor is resuming.

2014 Error AdminEventActionMissingOutputQueue Required Output Queue not present for Action %1.

1012 Informational AdminEventPauseRequested Assentor Address Processor reports a pause was requested. Informational only.

1013 Informational AdminEventContinueRequested Assentor Address Processor reports a continue was requested. Informational only.

1026 Informational AdminEventPaused Assentor Address Processor reports the Assentor Address Processor is paused. Informational only.

*Note that %1 is a variable that is filled in by the service generating the event.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 21 Using Log Files for Problem Analysis

Table 9 lists messages that you might receive from the Assentor controller. These messages also appear in the Windows 2000 Event log which you can view using the Event Viewer.

Table 9. Assentor Controller messages

Message ID Severity Symbolic Name Message*

1001 Error AdminEventServiceFailure Assentor Controller reports a service failure. Please investigate service: %1.

1003 Error AdminEventStartFailure Assentor Controller reports it is unable to start a service. Please investigate service: %1.

1004 Error AdminEventStopFailure Assentor Controller reports it is unable to stop a service. Please investigate service: %1.

1005 Warning AdminEventRecoveryAttempt Assentor Controller reports a service recovery attempt. Informational only.

1006 Informational AdminEventManualOverideDet Assentor Controller reports a manual override detection of a service. Informational only.

1007 Error SymbolicName=AdminEventServiceMgrError Assentor Controller reports an NT Service Manager Error. Please investigate: %1.

1008 Error SymbolicName=AdminEventAssentorNstarted Assentor Controller reports Assentor will not start. Please investigate.

1009 Error AdminEventAssentorNstopped Assentor Controller reports Assentor will not stop. Please investigate.

1010 Informational AdminEventMissingINI Assentor Controller reports a bad/missing AssentorController.ini file. SHB in C:/WINNT.

1011 Error AdminEventAssentorMailNstopped Assentor Controller reports mail processing services will not stop. Please investigate.

1012 Informational AdminEventPauseRequested Assentor Controller reports a pause was requested. Informational only.

1013 Informational AdminEventContinueRequested Assentor Controller reports a continue was requested. Informational only.

1014 Informational AdminEventMonitoringServices Assentor Controller reports it is monitoring services. Informational only.

1015 Informational AdminEventStoppingAllServices Assentor Controller reports it is stopping all services. Informational only.

1016 Informational AdminEventStoppingMailServices Assentor Controller reports it is stopping mail services. Informational only.

1017 Informational AdminEventStartingAllServices Assentor Controller reports it is starting all services. Informational only.

1018 Informational AdminEventStartingMailServices Assentor Controller reports it is starting mail services. Informational only.

1019 Informational AdminEventACStarted Assentor Controller reports Assentor Controller is starting. Informational only.

1020 Informational AdminEventACStopped Assentor Controller reports Assentor Controller is stopping. Informational only.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 22 Using Log Files for Problem Analysis

Table 9. Assentor Controller messages (Continued)

Message ID Severity Symbolic Name Message*

1021 Informational AdminEventStartService Assentor Controller reports it is starting service: %1.

1022 Informational AdminEventStopService Assentor Controller reports it is stopping service:%1.

1023 Error AdminEventAssentorMailNstarted Assentor Controller reports mail services will not start. Please investigate.

1024 Error AdminEventSCMOpenFailed Assentor Controller reports an SCM open failure. Please investigate.

1025 Error AdminEventServiceOpenFailed Assentor Controller reports a service open failure. Please investigate.

1026 Informational AdminEventPaused Assentor Controller reports the Assentor Controller is paused. Informational only.

1027 Warning AdminEventServiceNotResponding Assentor Controller reports a service not responding. Please investigate service %1.

1028 Warning AdminEventQueueThresholdExceeded Assentor Controller reports a queue depth has exceeded threshold. Please investigate queue:%1.

1029 Warning AdminEventQueueDelayExceeded Assentor Controller reports a queue delay exceeded threshold. Please investigate queue: %1.

1030 Warning AdminEventBounceService Assentor Controller reports a bounce of service: %1.

*Note that %1 is a variable that is filled in by the service generating the event.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 23 CHAPTER 4 Troubleshooting Assentor Compliance

This chapter explains how to determine the type of problem that Assentor* Compliance is having. It provides troubleshooting information to help you resolve mail processing and user interface problems.

Determining the Type of Problem

Whenever a problem with an Assentor Compliance server occurs, take the following actions: 1. Verify that all the services are running. 2. Review the Microsoft** Windows 2000** Event Viewer logs. 3. Retrieve and review all Assentor Compliance logs. 4. Monitor the Windows 2000 Task Manager and determine which tasks are consuming a large amount of CPU or memory. 5. Determine if there have been recent changes made to the server.

Understanding the Assentor Controller

The Assentor Controller is the controlling service for Assentor Compliance. It is the service that starts all the dependent services for Assentor Compliance to function. It monitors the services and restarts services as necessary. It is also responsible for sending out notifications when services have stopped.

Resolving Assentor Controller Problems

When checking Assentor Compliance to troubleshoot a problem, always verify that the services are started. If one of the services is not started, check the any corresponding log for more information on the stopped service.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 24 Troubleshooting Assentor Compliance

Assentor Controller Stopped on MP Server

If Assentor Controller stopped on the Mail Processing server, do the following: 1. Restart the service. If it returns an error instantly and does not attempt to start any services, check AssentorController.ini and verify that all services are installed that are trying to be started and that lines are properly punctuated. 2. Find out what service does not start by looking in the AssentorController_x.log file: %ProgramLocation%\program files\assentor\logs\AssentorController_x.log iLumin Delivery Service Not Started

If the iLumin Delivery Service is not started, do the following: 1. Attempt to restart service. Verify that service stays up. 2. Check %DataLocation%\msw\log\error.log for errors. 3. Check %DataLocation%\msw\log\dtYYMMDD.log for errors.

W3srv not started

If W3srv is not started, do the following: 1. Attempt to restart W3srv. 2. Reboot the server. Then restart AssentorController on each Mail Processing server, one at a time.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 25 Troubleshooting Assentor Compliance

Resolving Mail-Processing Problems

Mail processing is independent of most other Assentor Compliance processing. Therefore, troubleshooting a mail-processing problem is relatively isolated.

The Assentor Compliance mail engine services perform most of the mail processing. They read a shared file named A_Engine., to determine the message components to work on. This file resides on the root of the system partition.

Mail engines can serve the following two roles: • Primary mail processors, which are odd-numbered and perform the primary processing of incoming mail • Recovery mail engines, which are even-numbered and perform recovery processing on components that either could not be processed correctly the first time or that exceed a configured size

The wip_show utility displays the contents of the shared file A_Engine.DAT. The contents of this file may be cryptic. In general, if there are many items in this shared file for a long period of time, there may be a problem that needs attention. The Assentor Collator service issues log messages if a message resides in the shared file for more than 10 minutes.

The following subsections address potential email processing issues.

Determine the Cause of Mail-Processing Problems

When mail-processing problems stop messages from being processed, try to determine the cause of the problem. Do the following: 1. Answer the following questions: • Is the number of messages in the directories below continuing to grow and never decreasing?

\MSW\Internet\Pending \MSW\Internet\Holding This situation may indicate that: the AssentorMIME service is too busy to service its input queue, there is a problem with the system that Assentor is trying to send the email to, or the routing is not configured correctly.

• Has the oldest message in the \MSW\Internet\Pending directory remained there for more than 3 minutes? This situation may indicate that the AssentorMIME service is not processing new messages.

• Is AssentorMIME (AssentorMIME.exe) consuming most of the CPU? This situation may indicate that AssentorMIME is having difficulty processing a single message. If you answered yes to any of these questions, go to step •. If you answered no, go to step 8. 2. Check the following logs for errors relative to the Pending queue: • %DataLocation%\msw\log\error.log for errors

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 26 Troubleshooting Assentor Compliance

• %DataLocation%\mssql\log\errorlog for any errors • %DataLocation%\program files\assentor\logs\mailprocessing_x.log for any errors. 3. Check the %DataLocation%\msw\log\error.log for errors relative to the Holding queue. 4. Remove the oldest message from the following:

\MSW\Internet\Pending directory 5. Stop the Assentor Controller. 6. Ensure that all of the Assentor Compliance services stop successfully. 7. Restart the Assentor Controller. If the problem persists, go to step 8. 8. Do the following and answer the questions:

• Is a particular mail engine (mengine.exe) consuming most of the system’s CPU? If so, this mail engine is probably having trouble processing a message component. • Review the Assentor Controller logs. Has the Assentor Controller restarted more than one mail engine recently? If so, this may mean that a problem message is being retried by the mail engine services. • Review the mail-processing logs. Does the Assentor Collator recommend running the collator panic command?

• Run wip_show –v –l > wip.txt from the command line. Review the contents of wip.txt. Are there more than 100 entries in the WIP? If you answer yes to these questions and the problem has persisted for an extended period of time, it may be necessary to remove entries from the shared file A_Engine.DAT. In this situation, you may need to use the collator panic command on the system to force all messages in the A_Engine.DAT file into quarantine.

Warning: Because the A_Engine.DAT file can contain up to 1,000 messages, iLumin discourages using the collator panic command. Before using this command, contact the Assentor Customer Service Center. Never rename or remove the A_Engine.DAT file since this results in lost mail.

Repeated Delivery of Same Mail Message

Content Technologies has made iLumin aware of an issue where leaving Windows 2000 Explorer open on the Domains directory can result in the repeated delivery of the same mail message. For this reason, please avoid leaving Windows 2000 Explorer open on the Domains directory.

Problems with Quarantined/Released Messages

Assentor stores the Windows 2000 computer name of the Mail Processing server that processes a particular message. The Quarantine Manager uses this computer name when a compliance reviewer tries to approve or reject a message from quarantine.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 27 Troubleshooting Assentor Compliance

Therefore, changing the computer name of the Mail Processing server halts mail processing and prevent compliance reviewers from approving or rejecting any messages that were processed under the previous computer name.

If you want to rename the Mail Processing server and want to avoid the problem noted above, do the following: 1. First, ensure that all messages are released from quarantine. 2. Then rename the Mail Processing server.

If the server was not renamed and Compliance Reviewers are unable to approve or reject messages, verify that permissions have not change on the Compliance Database server and Mail Processing server. The Assentor Compliance Users' group should have read and write permissions to the documents directory and w_qtine2 directory on the Compliance Database server and Mail Processing servers.

Out of Space or Running Slow

Check the following if you think there is a space problem on the Mail Processing Server. • If processes appear to be running slowly, check the sizes of: %Program Drive%:\Inetpub\Mailroot\Queue %Data Drive%:\MSW\Internet\Addressing %Data Drive%:\MSW\Internet\Pending %Data Drive%:\MSW\Internet\Incoming • If there are backups in the Queue, there could be iLuminSRS problems. A high percentage of the cause for these backups is due to registration issues. • Make sure that the iLuminSRS is installed correctly. Open the Registry and go to HKLM\Software and search for iluminSRS.dll. The result should be: C:\Program Files\Assentor\Run\Release\bin\iLuminSRS.dll Otherwise, refer to the Installation Guide for a reinstall of the sink.

Monitor Queues

There is a file queue monitoring tool for CM for MMR. It was designed for troubleshooting the Mail Processor mail flow and it is provided on the Assentor Installation CD. It takes one parameter, which is simply a number that determines how many minutes to wait between doing each round of monitoring.

It is not recommended keeping this running as a long-term diagnostic tool (it might add too much overhead), but it should be useful in certain cases where you are trying to monitor the system during a short period of time. If you specify a parameter of "0" it runs the monitoring only once. For example running this as "monitor 6" would give counts of the queue sizes every 6 minutes monitoring the pending, addressing, and CMStore queues.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 28 Troubleshooting Assentor Compliance

Backup in the Message_done Table

On a healthy system, the CMStore process should be able to keep the message done table fairly clean, with only a few entries in it. If there is a back-up in the message_done table, this could indicate a problem with CM or DB2. First, try to re-cycle the service. If CMStore encounters a message-level error that it cannot handle, it will move the message to the "bad" folder in the Documents directory. It also logs the error. CMStore logs to the ContentManager log that is located where specified in the CM key.

If the CMAdd encounters a message-level error that it cannot recover from, it places the reference file for that message in the "storeerror" directory (under the AssentorOps folder).

Failure of Content Manager or DB2

If Content Manager fails, the CMAdds should automatically reconnect and continue processing when CM is brought back up. However, the CMAdds will sleep between retries for up to 5 minutes, so be cautious in assuming there is a problem immediately after bringing CM back up. There could also be occasions where the CMAdds do not automatically restart. In those cases, they may need to be manually recycled.

If Content Manager is taken down for an extended period of time (for maintenance, etc.), you may have to be concerned about the number of files backing up in the CMStore directories (from both a disk-space perspective and from a "number of files in a single directory" perspective). The work-around to this is similar to work-around for backed-up pending folders. If necessary, you can copy off the files and put them back in manually later on.

A healthy system should back-up in the pending folders long before you see any substantial back-up in the CMStore directories, although the rate of storage to CM may drop during certain periods of usage on the CM box (during back-ups, etc.). Once the CM condition is removed, the CMAdds should be able to reduce the backlog while keeping up with new messages coming in.

Determine if Quarantined Mail is Lost

Do the following to determine whether pre-review quarantined email is present on the lost server:

Select count(*) from doc2 where external_identification like '%hostname%' and status_cd = 'BAD' and review_type_cd = 'PRE'. If the count is zero, then there is no need for further action because there is no pre-review quarantined mail on the lost server. However, if the count is > 0, then there is pre-review quarantined email on the lost server and you will need to take further action.

Do the following to recover the pre-review quarantined email: 1. Build a mail processor with the host name of the lost mail processor. 2. Copy the messages that you identified as being on the lost server to the %Program Drive%:\MSW\Internet\W-Qtine2 directory. The algorithm is demonstrated below. B0000123456@TE4MP D:\MSW\Interent\W_Qtine2\6\5\4\

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 29 Troubleshooting Assentor Compliance

3. Copy both the .msg and .rcp files to the server that you built. Using the parameters in the above example the result would be: D:\MSW\Interent\W_Qtine2\6\5\4\B0000123456.MSG D:\MSW\Interent\W_Qtine2\6\5\4\B0000123456.RCP At this point, the messages can be released/rejected by an administrator or compliance reviewer from the Assentor Compliance user interface.

Resolving User Interface Problems

The Assentor Compliance user interface depends on the user table value in the Assentor DB2 database. For authentication to succeed, users must have valid Windows 2000 accounts with the appropriate permissions and they must be defined within Assentor Compliance. The following subsections address potential user interface issues: • Unable to contact web server • Unable to authenticate with Assentor Compliance • Users cannot view messages • Messages are received by users but are not visible in the user interface • Messages not visible in the user interface for a specific user • Browser hangs after user logs on to Assentor Compliance • Users cannot approve or reject messages

Unable to Contact Web Server

When this problem occurs, the web browser attempting to access the user interface reports that the host or site is inaccessible.

Check the following items: 1. Verify that the web service is running and operating normally by accessing the Assentor web server from the web browser on the server. If this is not successful then there may be a problem with the server or browser. If it is successful, go to step 2. 2. Determine the IP address that the web browser is trying to reach and confirm that it is the correct IP address for Assentor Compliance. 3. Verify that the network path from the remote browser to the server is correct. If the browser cannot reach the destination server, it is most likely a network or name resolution issue. If the client browser uses a proxy when accessing the Assentor Compliance server, consider bypassing this proxy during Assentor sessions. You can configure this option within the web browser. Proxies may have caching schemes that affect the operation of Assentor.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 30 Troubleshooting Assentor Compliance

Unable to Authenticate with Assentor

When this problem occurs an Assentor Compliance security check is failing for one of the following: • IIS basic authentication • User profile authentication

IIS Authentication If IIS does not permit authentication, users continually receive a login box, but each time Problem the login fails.

To troubleshoot this problem, do the following: 1. Turn on auditing for IIS through the User Manager for the users in question. Correct the problem specified by the audit log. 2. Consider trying other user accounts. If all accounts fail, there may be a web server setting that is incorrect.

Note: All Assentor Compliance users must have valid Windows 2000 accounts with rights to log on locally to the server.

Assentor Compliance When this problem occurs the user cannot logon to the user interface. Assentor Compliance Authentication Problem blocks access to the specified user and displays a message stating that “ is not permitted to access Assentor.” This problem is typically caused by the profile being incorrectly defined in Assentor Compliance. Also, if the user is accessing Assentor Compliance with domain account, the login name must be fully specified in Assentor Compliance. Check the following: • Verify that the user account that was added to the Assentor interface is the same as the account that the user is logging in with. • If the user types information and is returned back to the logon screen then verify that the user account and password are correct and verify that the user account has been granted access to the server.

Users Cannot View Messages

If users can not view messages when they select a type in the Show drop down list box on the Browse Messages page in the UI, do the following: 1. Verify that all servers have the AssentorController started. 2. Check %DataLocation%\program files\assentor\logs\webui_x.log files.

Browser Hangs after User Logs On to Assentor Compliance

If the user logs on to Assentor Compliance and the browser hangs, do the following: 1. Attempt to stop WWW service. 2. If the service hangs, then kill the service and have the user logon to the Assentor Compliance interface. If the user still can't log in, kill the service again and logon with an Administrator account. If this is successful, find out what security patches and modifications were made to the server. This is a permissions issue.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 31 Troubleshooting Assentor Compliance

3. Call Assentor Support Center for the proper permissions needed.

Users Cannot Approve or Reject Messages

If users cannot approve or reject messages, verify that user's login account exists on all servers or AssentorUser group exist on all servers.

Resource Management Errors

If the Content Manager log shows an Exception when you are running the mail processor and cites the security token (as shown in the log below), you should refresh the security token as per the IBM instructions. To do this: 1. Stop the Resource Management (RM) server for Content Manager. 2. Use the control center to logon to Content Manager. 3. Click the Refresh Security Token button. This should take about a minute. If it happens instantly, it is likely that the refresh failed. Be sure that RM is stopped and try again. 4. Log out and restart the RM server.

20050203.23:40:03.341|----|SYS2MP1|ContentManager|009248|008692|> >: : setMessage: add() or update() 20050203.23:40:03.341|----|SYS2MP1|ContentManager|009248|008692|> >: : XXXXXXXXXXXXXXXXXXXXXXXXXXXX 20050203.23:40:03.341|----|SYS2MP1|ContentManager|009248|008692|> >: : X !!! Exception !!! X 20050203.23:40:03.341|----|SYS2MP1|ContentManager|009248|008692|> >: : XXXXXXXXXXXXXXXXXXXXXXXXXXXX 20050203.23:40:03.341|----|SYS2MP1|ContentManager|009248|008692|> >: : Name: DKXDOError 20050203.23:40:03.341|----|SYS2MP1|ContentManager|009248|008692|> >: : Message ID: 0 20050203.23:40:03.341|----|SYS2MP1|ContentManager|009248|008692|> >: : Error State: 20050203.23:40:03.341|----|SYS2MP1|ContentManager|009248|008692|> >: : Error Code: 9804 20050203.23:40:03.341|----|SYS2MP1|ContentManager|009248|008692|> >: : Location 0: DKLobICM.cpp::[3911] 20050203.23:40:03.341|----|SYS2MP1|ContentManager|009248|008692|> >: : Message 0: ICM9804: The security token supplied with order store was invalid.::HTTP/1.1 214 ICMResourceManager response data in message (SERVER RC) : 9804 20050203.23:40:03.341|----|SYS2MP1|ContentManager|009248|008692|> >: : ------20050203.23:40:03.341|EROR|SYS2MP1|ContentManager|009248|008692|> >: setMessage: ERROR: unable to add document to CM. Make sure WAS is started. Consider recreating ItemType.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 32 Troubleshooting Assentor Compliance

Assentor Services Register/Unregister

The following table provides information on the Assentor Services and the Register/Unregister file names associated with the Service, Executable, and Internal Registry names. Table 10. Assentor Services Register/Unregister

Assentor/Discovery Service Names Executable Name Internal Registry name Register Unregister Assentor Address Processor addressprocessor.exe AddressProcessor -service -unregserver Assentor Bloomberg Processor bbprocessor.exe BloombergProcessor -install -uninstall Assentor Bloomberg2 Processor bb2processor.exe Bloomberg2_Processor -install -uninstall Assentor Bond Desk Processor bdprocessor.exe BondDeskProcessor -install -uninstall Assentor BulkApprove BulkApprove.exe AssentorBulkApprove /regserver /unregserver Assentor Collator collator.exe collatorsvc add remove Assentor Controller assentorcontroller.exe AssentorController -service -unregserver Assentor IM Link (4 entries) IMProcessor.exe InstantMessagingX (X=1-4) -install -uninstall Assentor InternetBloomberg IBprocessor.exe InternetBloombergProcessor -install -uninstall Assentor Mail Engine (20 entries) mengine.exe mesvcXX (XX=01-20) add remove Assentor Quarantine Manager quarantinemanager.exe qtine2svc add remove AssentorMIME AssentorMIME.exe AssentorMIME /regserver /unregserver CMAdd CMAdd.exe CMAdd /service /unregister CMComplianceCOM CMComplianceCOM.exe CMComplianceCOM /service /unregister CMDisco CMDisco.exe CMDisco /service /unregister CMDiscoveryCOM CMDiscoveryCOM.exe CMDiscoveryCOM /service /unregister CMDiscoveryOutputCOM CMDiscoveryOutputCOM.exe CMDiscoveryOutputCOM /service /unregister CMIndex CMIndex.exe CMIndex /service /unregister CMPurge CMPurge.exe CMPurge /service /unregister CMQueue CMQueue.exe CMQueue /service /unregister CMSearch CMSearch.exe CMSearch /service /unregister CMService CMService.exe CMService /service /unregister CMStore CMStore.exe CMStore /service /unregister DiscoDelete DiscoDelete.exe DiscoDelete /regserver /unregserver DiscoFile DiscoFile.exe DiscoFile /regserver /unregserver DiscoPDF DiscoPDF.exe DiscoPDF /regserver /unregserver DiscoPrint DiscoPrint.exe DiscoPrint /regserver /unregserver DiscoPST DiscoPST.exe DiscoPST /regserver /unregserver DiscoSave DiscoSave.exe DiscoSave /regserver /unregserver DiscoSearch DiscoSearch.exe DiscoSearch /service /unregserver Discovery Indexer DiscoveryIndexer.exe DiscoveryIndexer add remove iLuminSDS IluminSDS.exe IluminSDS /regserver /unregserver JournalReader JournalReader.exe JournalReader /regserver /unregserver MSWRewrite MSWRewrite.exe MSWRewrite /regserver /unregserver

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 33 Glossary

This glossary explains words and terms and acronyms according to their use in Assentor* Compliance. In other contexts they may have different meanings.

A

active message. The message that is currently being displayed in a Message Detail page.

Address Processor. Component of Assentor Compliance that handles the analysis and manipulation of addressing elements in email headers—if a firm is using the support for Microsoft Exchange mail option or the independent contractor option.

alias processing. The translation of real to apparent addresses and apparent to real addresses in email, which is handled by the Assentor Address Processor.

all messages. Category of email messages in Assentor Compliance comprised of all messages that have passed through Assentor Compliance.

approved messages. Category of email messages in Assentor Compliance comprised of messages that have been approved for delivery by a compliance reviewer.

Assentor controller. The service that starts, monitors, and stops the operation of all Assentor-related services.

AssentorController.ini file. A configurable file that specifies key configurations for the operation of Assentor Compliance services.

auditor. An employee or contractor at a firm who is tasked to audit messages and determine whether the message reviewers are taking appropriate action on violations to ensure compliance with SEC regulations.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 34 Glossary

B

BCC. Abbreviation for blind carbon copy.

Bloomberg Mail. An email system that is designed for the financial community. Assentor Compliance post-processes Bloomberg Mail.

browse. Go to the list of messages or return to the previous list of messages.

browser—Microsoft Internet Explorer 5.5 SP2 or later. bulk release. Assentor Compliance function for releasing post-review quarantined messages for archiving. If a compliance reviewer performs an Advanced Search on All Quarantined and then clicks Bulk Release, all post-review quarantined messages in the entire result set of messages are released.

C

clean messages. Category of email messages comprised of messages in which Assentor Compliance did not find any compliance violations or that a compliance reviewer approved.

Client Support Center (CSC). The iLumin group that provides technical support to Assentor customers.

compliance reviewer (CR). An employee or contractor at a firm who is tasked to review the firm’s email messages for compliance and policy violations and is authorized to take the appropriate action on the messages to ensure compliance with the regulatory agency and the firm’s policies.

Content Management for Message Monitoring and Retention (CM for MMR). An IBM product that interfaces with Assentor Compliance. It is used for message monitoring and retention and provides access to the archived messages that Assentor Compliance processes.

COTS. Abbreviation for commercial off-the-shelf.

CR. Abbreviation for compliance reviewer.

CSC. Abbreviation for Client Support Center.

D

direction. The assignment of inbound, internal, or outbound that is associated with a quarantined message.

DNS. Abbreviation for Domain Name Service.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 35 Glossary

Domain Name Service (DNS). The Internet utility that implements the domain name system. DNS servers, also called name servers, maintain databases containing the addresses and are accessed transparently to the user.

duplicate message. A message that has the message content and status (quarantined or not quarantined) as the active message. It is assigned to the same compliance reviewer and processed within a configurable amount of time before or after the active message in the Message Detail page. Duplicate message may have the same direction or different directions.

E

email. electronic mail.

exclusion list. A list of words and phrases that a firm wants Assentor Compliance to ignore and not quarantine in email.

expired document. A document that was registered as pre-approved in Assentor Compliance and whose pre-approved registration has expired. If not deleted, Assentor Compliance could review and quarantine it.

extraction. Capability in Assentor Compliance (via a Save icon) that allows a reviewer to get specific information from email messages and send the output to a file that can be viewed or printed.

F

file transfer protocol (FTP). An agreed set of standards covering transmission speed, format of data, and the signals required o synchronize the transfer for the transfer of files, or data, between different devices.

FTP. Abbreviation for file transfer protocol.

G

Gb. Abbreviation for gigabyte.

ghost reviewer. An employee or contractor at a firm who is tasked to review the work of compliance reviewers to determine whether they are performing compliance reviews of email messages in a proper and timely fashion.

gigabyte (Gb). One billion bytes, one million kilobytes, or one thousand megabytes.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 36 Glossary

H

I

IC. Abbreviation for independent contractor.

ID. Abbreviation for identifier.

identifier (ID). Any numeric or text string used as a label.

IIS. Abbreviation for Internet Information Server**.

iLumin SMTP Receiver Service. iLumin’s Receiver Service that is optimized to send and receive. The service checks messages to determine which need to be blocked (later released by Assentor Compliance). Messages that do not need blocking are delivered immediately.

iLumin SMTP Delivery Service. iLumin’s Delivery Service that attempts to deliver the messages once they are either approved by the compliance reviewer for release or are determined to be clean.

inclusion list. A list of words and phrases that a firm wants Assentor Compliance to detect and quarantine in email messages.

information technology (IT). Collective term for the various technologies involved in processing and transmitting information, or a department of employees responsible for the implementation of information technology in a firm.

Instant Message. Type of mail message that Assentor Compliance can process.

J

K

L

LAN. Acronym for local area network.

local area network (LAN). In computing, a network that is restricted to a single room or building and that enables multiple devices to be connected together.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 37 Glossary

M

mass approve. Assentor Compliance function for approving some or all quarantined messages. Mass Approve is available for the All Quarantined message type and for Advanced Search where All Quarantined messages have been selected as search criteria. The mass approve function approves only the messages that have been selected on the current screen (i.e., if the user sets the display for 20 messages per page, then 20 is the maximum number of messages that can be selected and approved at one time). Compliance reviewers must have Mass Message Privileges to use the mass approve feature.

mass reject. Assentor Compliance procedure for rejecting a group of duplicate quarantined messages. Only compliance reviewers can reject quarantined messages.

mass view. Assentor Compliance procedure for marking a group of duplicate messages as viewed. Only compliance reviewers can perform this function.

mass violation. Assentor Compliance procedure for marking a group of duplicate non-quarantined messages as a false negative. Only compliance reviewers can perform this function.

Mb. Abbreviation for megabytes.

Megabytes (Mb). Roughly one million bytes or one thousand kilobytes.

message summary. A file that can be generated in Assentor Compliance and includes summary information extracted from one or multiple messages.

N

natural language content analysis (NLCA). The study of strategies for computer programs to recognize and understand language in spoken and written form.

NLCA. Abbreviation for natural language content analysis.

non-quarantined. Email messages in which Assentor Compliance did not find compliance violations and, as a result, did not place into quarantine.

O

output rate. Number of Bloomberg messages that are output by the Bloomberg Processor each hour. For example, if the output rate is set to 60 messages an hour, a Bloomberg message is placed in the Assentor queue every minute.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 38 Glossary

P

pending. Category of email messages in Assentor Compliance comprised of messages that have been put in a pending folder by a compliance reviewer for further review.

post-review messages. Email messages that Assentor Compliance examined and delivered if clean, or put into quarantine for suspected compliance violations after they were delivered.

postprocessing. Analysis that Assentor Compliance performs on an archive file to detect compliance violations after the mail was sent (typically for Bloomberg or BondDesk mail).

pre-approved document. A document that a firm has registered in Assentor Compliance as acceptable so that it will not be quarantined when it is used in an email.

pre-review messages. Email messages that Assentor Compliance examined and put into quarantine for suspected compliance violations before they were delivered. Because these messages have not been delivered, compliance reviewers should either approve or reject pre-review messages before they review post-review messages.

Q

quarantined. Category of email messages comprised of messages in which Assentor Compliance found suspected compliance violations and which will not be delivered until reviewed and approved by a compliance reviewer.

queue depth threshold. The number of total (Bloomberg or otherwise) files that are permitted in the output queue.

R

RAID. Acronym for Redundant Array of Inexpensive/Independent Disks.

RAM. Acronym for random access memory.

random access memory (RAM). The working memory of the computer into which application programs can be loaded from outside and then executed.

real address. Address assigned by the ISP to an independent contractor. It is the address from which the contractor actually sends email.

registered documents. A firm’s documents that the firm has registered in Assentor Compliance as pre-approved so that the documents can be used in email without being quarantined by Assentor.

rejected message. A message that was quarantined by Assentor Compliance, has been reviewed by a compliance reviewer, was determined to contain compliance violations, and was deleted from the system by the compliance reviewer without being delivered to the intended recipient.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 39 Glossary

retrieving messages. Process of finding stored message information on an optical disk for read-only viewing.

S

SEC. Abbreviation for Securities and Exchange Commission.

SMTP. Abbreviation for Simple Mail Transfer Protocol.

SP1. Abbreviation for Service Pack 1.

standard warning messages. Preconfigured messages for a firm that compliance reviewers can email to employees who send non-compliant messages

suspected violation. A word or phrase in an email message scanned by Assentor that may not conform the firm’s compliance rules.

T

tag. Graphical insertions used to highlight suspected violations and people, entities, places, and numerical data that may appear in the body of an email message displayed in the Message Detail page.

TCP/IP. Abbreviation for Transmission Control Protocol/Internet Protocol.

thresholds. A set of pre-defined criteria that Assentor uses to analyze a firm’s email messages and bring to the firm’s attention the employees whose business practices are not compliant with SEC regulations and firm policies.

Transmission Control Protocol/Internet Protocol (TCP/IP). A networking protocol with broad support deriving from its ability to connect disparate hosts.

U

UI. Abbreviation for user interface.

undetected violation. A violation that Assentor did not detect in a message, but which a compliance reviewer found and marked.

universal resource locator (URL). An address for a resource on the Internet. URLs are used by web browsers to locate Internet resources.

unviewed messages. Category of email messages in Assentor Compliance comprised of clean and quarantined messages that have not yet been viewed by a compliance reviewer.

URL. Abbreviation for universal resource locator.

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 40 Glossary

user interface (UI). The procedures and methods through which a user operates a program.

V

viewed messages. Category of email messages in Assentor Compliance comprised of messages that have been viewed by a compliance reviewer in the Message Detail page in Assentor.

W

WINS. Acronym for Windows Internet Naming Service.

X

Y

Z

Assentor Operations Guide for Content Manager for Message Monitoring and Retention 41