HPE IDOL Server Software Version: 11.2 Administration Guide Document Release Date: October 2016 Software Release Date: October 2016 Administration Guide Legal Notices Warranty The only warranties for Hewlett Packard Enterprise Development LP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HPE shall not be liable for technical or editorial errors or omissions contained herein. The information contained herein is subject to change without notice. Restricted Rights Legend Confidential computer software. Valid license from HPE required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. Copyright Notice © Copyright 2016 Hewlett Packard Enterprise Development LP Trademark Notices Adobe™ is a trademark of Adobe Systems Incorporated. Microsoft® and Windows® are U.S. registered trademarks of Microsoft Corporation. UNIX® is a registered trademark of The Open Group. This product includes an interface of the 'zlib' general purpose compression library, which is Copyright © 1995-2002 Jean-loup Gailly and Mark Adler. Documentation Updates The title page of this document contains the following identifying information: l Software Version number, which indicates the software version. l Document Release Date, which changes each time the document is updated. l Software Release Date, which indicates the release date of this version of the software. To check for recent software updates, go to https://downloads.autonomy.com/productDownloads.jsp. To verify that you are using the most recent edition of a document, go to https://softwaresupport.hpe.com/group/softwaresupport/search-result?doctype=online help. This site requires that you register for an HPE Passport and sign in. To register for an HPE Passport ID, go to https://hpp12.passport.hpe.com/hppcf/login.do. You will also receive updated or new editions if you subscribe to the appropriate product support service. Contact your HPE sales representative for details. Support Visit the HPE Software Support Online web site at https://softwaresupport.hpe.com. This web site provides contact information and details about the products, services, and support that HPE Software offers. HPE Software online support provides customer self-solve capabilities. It provides a fast and efficient way to access interactive technical support tools needed to manage your business. As a valued support customer, you can benefit by using the support web site to: HPE IDOL Server (11.2) Page 2 of 555 Administration Guide l Search for knowledge documents of interest l Submit and track support cases and enhancement requests l Access product documentation l Manage support contracts l Look up HPE support contacts l Review information about available services l Enter into discussions with other software customers l Research and register for software training Most of the support areas require that you register as an HPE Passport user and sign in. Many also require a support contract. To register for an HPE Passport ID, go to https://hpp12.passport.hpe.com/hppcf/login.do. To find more information about access levels, go to https://softwaresupport.hpe.com/web/softwaresupport/access-levels. To check for recent software updates, go to https://downloads.autonomy.com/productDownloads.jsp. HPE IDOL Server (11.2) Page 3 of 555 Administration Guide Contents Part I: Introduction 21 Chapter 1: Introduction to HPE IDOL Server 23 IDOL Server Operations 23 Agents 23 Alerts 24 Automatic Query Guidance 24 Categorization 24 Category Matching 24 Channels 24 Cluster Information 25 Collaboration 25 Dynamic Clusters 25 Dynamic Thesaurus 25 Eduction 25 Expertise 25 Hyperlinks 26 Email Users 26 Profiles 26 Search and Retrieval 26 Conceptual Matches 26 Advanced Keyword Search 27 Boolean and Bracketed Boolean Search 27 Exact Phrase Search 27 Field Restrictions 27 FieldText Search 27 Fuzzy Search 27 Parametric Search 27 Proper Names Search 28 Proximity Search 28 Soundex Keyword Search 28 Synonym Search 28 Spell Check 28 Summarization 28 Taxonomy Generation 28 Automatic Taxonomy Based on Cluster Result 29 Automatic Taxonomy to Category Generation 29 View Documents 29 Getting Started 29 Send Actions to IDOL Server 29 Display Online Help 30 HPE IDOL Server (11.2) Page 4 of 555 Administration Guide Part II: Store Content in HPE IDOL Server 33 Chapter 2: Configure Content Storage 35 Edit the Configuration File 35 Stored Content 35 Disable Content Storage 35 Store HPE IDOL Server Data Files on Multiple Disks 36 Compress the Data Index 36 Allocate Files to HPE IDOL Server Databases 37 Set up the Field Index Process 38 Index XML Attributes 39 Configure IDOL Server for Language and to Encode 40 Optimize Index Process 41 Index Process 41 Delayed Synchronization 41 Chapter 3: Index Data 43 Index Overview 43 Process Data before you Index 44 DREADD: Index IDX and XML Files Directly 44 DREADD Parameters 45 DREADD Examples 48 Specify Field Names 48 DREADDDATA: Index Data over a Socket 50 DREADDDATA Parameters 50 Send Data with a POST Method 51 Use the cURL Command-Line Tool 51 Use a Script 52 DREADDDATA Examples 53 Index Stop Words 54 Index Nonalphanumeric Characters 55 Term Separators 55 Index Nonalphanumeric Characters for Retrieval 56 Hyphenated Terms 57 Character Tokenization 58 Prevent Duplicate Documents 59 Deduplication Options—KillDuplicates 59 Enable Deduplication for all Index Jobs 61 Limit ReferenceType Fields used for Deduplication 62 Use KillDuplicatesChecksumField to Prevent Unnecessary Indexing 62 Use KillDuplicatesPreserveFields to Preserve a Field 63 Enable Deduplication for Individual Index Jobs 63 Use KeepExisting to Minimize the Index Load 64 Enable Deduplication for Connector Index Jobs 64 Deduplication Constraints 64 Use the Combine Operation 64 Use Deduplication with DIH Reference-Based Indexing 64 HPE IDOL Server (11.2) Page 5 of 555 Administration Guide Use Deduplication with DIH Field-Based Indexing 65 Add Metadata to Documents 65 Check Index Status 65 IndexerGetStatus Status Codes 68 Tag Documents into Clusters 70 Chapter 4: Fields 73 About Fields 73 Configure a Field Process 76 Update Field Configuration 79 Update Fields in the Configuration File 81 Update Field Configuration with an Index Action 82 Update Field Configuration with IDOL Admin 83 Index Fields 83 Configure the Number Index Process 85 NumericDateType Fields 86 NumericType Fields 87 FieldCheckType Fields 88 ReferenceType Fields 90 Set up ReferenceType Fields 90 Use KillDuplicates and Combine on ReferenceType Fields 91 Highlight Fields 93 BitFieldType Fields 94 Edit Set Information after Indexing 95 Find Documents within a Set 96 Metadata Fields 96 Change Field Values 97 Chapter 5: Language Support 99 IDOL Language Support Concepts 99 Run IDOL Server in Multiple Languages 101 Determine the Languages that are Enabled 102 Define Language Types 103 Associate Language Types with Documents 105 Documents that Contain a Language Type Field 105 Documents that Contain Field Data that can Identify Language 106 Add LanguageType Fields to Documents 107 Define a Default Language Type 108 Define a General Language 108 Enable Automatic Language Detection 109 Specify the Language Type of a Query 110 Convert Results to a Specific Encoding 110 Text Queries 111 Text-Free Queries 111 Return Documents in Multiple Languages 112 International Stop List 112 Return Documents in a Specific Language 113 Create a Custom Stem File for a Language 114 HPE IDOL Server (11.2) Page 6 of 555 Administration Guide Decompose Compound Words 114 Enable Transliteration 115 Enable Generic Transliteration 115 Enable Transliteration for Individual Languages 116 Chapter 6: Set Up Document Tracking 119 Set up Document Tracking with an SQL Back End 119 Set up Document Tracking with PostgreSQL 119 Set up PostgreSQL to Store Tracking Information 119 Install the SQL Database 120 Set up the Database and Table 120 Database Access Permissions 123 Set up the IDOL Host Machines 124 Install the SQL Driver and Manager for PostgreSQL 124 Check the Installed Drivers 124 Configure IDOL Components 125 Set up Document Tracking with Microsoft SQL Server 126 Set up Microsoft SQL Server to Store Tracking Information 126 Configuration Example for Microsoft SQL Server 126 Troubleshoot Connection and Authentication Problems 127 Initialization Commands 127 Set up the IDOL Host Machines 129 Install the SQL Driver and Manager for Microsoft SQL Server 129 Check the Installed Drivers 130 Configure IDOL Components 130 Verify the Setup 132 Check IDOL Configuration 132 Index Content 132 Query Your Document 133 Clean Results 133 Set up Document Tracking with a Log Back End 133 Configure Event Storage 134 Document Tracking Event Definitions 135 Part III: HPE IDOL Server Operations 137 Chapter 7: Agents 139 About Agents 139 Manipulate Agents 139 Create an Agent 139 Edit an Agent 140 Retrain an Agent 140 Copy an Agent 140 View Agent Details 140 Delete an Agent 141 Query with Agents 141 Modify Document References for an Agent 141 HPE IDOL Server (11.2) Page 7 of 555 Administration Guide Collaboration and Expertise with Agents 142 Collaboration 142 Expertise 142 Chapter 8: Categorization 143 Introduction to Categorization 143 Create a Hierarchical Category Structure 144 Create Categories from Scratch 144 Create Categories