KeyView Software Version 12.2 Filter SDK C++ Programming Guide Document Release Date: February 2019 Software Release Date: February 2019 Filter SDK C++ Programming Guide Legal notices Copyright notice © Copyright 2016-2019 Micro Focus or one of its affiliates. The only warranties for products and services of Micro Focus and its affiliates and licensors (“Micro Focus”) are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Micro Focus shall not be liable for technical or editorial errors or omissions contained herein. The information contained herein is subject to change without notice. Documentation updates The title page of this document contains the following identifying information: l Software Version number, which indicates the software version. l Document Release Date, which changes each time the document is updated. l Software Release Date, which indicates the release date of this version of the software. You can check for more recent versions of a document through the MySupport portal. Many areas of the portal, including the one for documentation, require you to sign in with a Software Passport. If you need a Passport, you can create one when prompted to sign in. Additionally, if you subscribe to the appropriate product support service, you will receive new or updated editions of documentation. Contact your Micro Focus sales representative for details. Support Visit the MySupport portal to access contact information and details about the products, services, and support that Micro Focus offers. This portal also provides customer self-solve capabilities. It gives you a fast and efficient way to access interactive technical support tools needed to manage your business. As a valued support customer, you can benefit by using the MySupport portal to: l Search for knowledge documents of interest l Access product documentation l View software vulnerability alerts l Enter into discussions with other software customers l Download software patches l Manage software licenses, downloads, and support contracts l Submit and track service requests l Contact customer support l View information about all services that Support offers Many areas of the portal require you to sign in with a Software Passport. If you need a Passport, you can create one when prompted to sign in. To learn about the different access levels the portal uses, see the Access Levels descriptions. KeyView (12.2) Page 2 of 231 Filter SDK C++ Programming Guide Contents Part I: Overview of Filter SDK 11 Chapter 1: Introducing Filter SDK 12 Overview 12 Features 12 Platforms, Compilers, and Dependencies 13 Supported Platforms 13 Supported Compilers 13 C++ Filter SDK 14 Software Dependencies 14 Windows Installation 15 UNIX Installation 16 Package Contents 17 License Information 18 Enable Advanced Document Readers 18 Update License Information 18 Directory Structure 19 Chapter 2: Getting Started 21 Use the C++ Language Implementation of the API 21 Build the C++ API 21 Create a KeyView Session 22 Configure your session 22 Detect the Format of a File 23 Filter a File 23 Extract Subfiles 23 Extract Metadata 24 Exceptions 24 Generic IO Types 24 Part II: Use Filter SDK 27 Chapter 3: Use the File Extraction API 28 Introduction 28 Extract Subfiles 29 Extract Images 30 Extract Mail Metadata 30 Default Metadata Set 30 Extract the Default Metadata Set 31 Extract Subfiles from Outlook Express Files 31 Extract Subfiles from Mailbox Files 31 Extract Subfiles from Outlook Personal Folders Files 32 Choose the Reader to use for PST Files 32 KeyView (12.2) Page 3 of 231 Filter SDK C++ Programming Guide MAPI Attachment Methods 34 Open Secured PST Files 34 Detect PST Files While the Outlook Client is Running 34 Extract Subfiles from Lotus Domino XML Language Files 35 Extract .DXL Files to HTML 35 Extract Subfiles from Lotus Notes Database Files 36 System Requirements 36 Installation and Configuration 36 Windows 37 Solaris 37 AIX 5.x 37 Linux 38 Open Secured NSF Files 38 Format Note Subfiles 38 Extract Subfiles from PDF Files 39 Improve Performance for PDFs with Many Small Images 39 Extract Embedded OLE Objects 39 Extract Subfiles from ZIP Files 40 Extract Metadata 40 Chapter 4: Use the Filter API 41 Generate an Error Log 41 Enable or Disable Error Logging 42 Use the API 42 Use Environment Variables 42 Change the Path and File Name of the Log File 42 Report Memory Errors 42 Use the API 43 Use Environment Variables 43 Specify a Memory Guard 43 Specify the Maximum Size of the Log File 43 Extract Metadata 44 Convert Character Sets 44 Determine the Character Set of the Output Text 44 Guidelines for Character Set Conversion 45 Set the Character Set During Filtering 45 Set the Character Set During Subfile Extraction 46 Extract Deleted Text Marked by Tracked Changes 46 Filter a File 46 Filter PDF Files 47 Filter PDF Files to a Logical Reading Order 47 Enable Logical Reading Order 48 Use the C++ API 48 Use the formats.ini File 48 Rotated Text 49 Extract Custom Metadata from PDF Files 49 Extract All Custom Metadata 49 KeyView (12.2) Page 4 of 231 Filter SDK C++ Programming Guide Filter Tagged PDF Content 49 Skip Embedded Fonts 50 Use the formats.ini File 50 Use the C++ API 50 Control Hyphenation 51 Use the formats.ini File 51 Use the C++ API 51 Filter Spreadsheet Files 51 Filter Worksheet Names 51 Filter Hidden Text in Microsoft Excel Files 51 Specify Date and Time Format on UNIX Systems 52 Filter Very Large Numbers in Spreadsheet Cells to Precision Numbers 52 Extract Microsoft Excel Formulas 53 Configure Headers and Footers 54 Filter Hidden Data 55 Hidden Data in HTML Documents 55 Tab Delimited Output for Embedded Tables 55 Table Detection for PDF Files 56 Exclude Japanese Guide Text 56 Source Code Identification 56 Chapter 5: Sample Programs 59 Introduction 59 Build the Sample Programs 59 Run the Sample Programs 60 detect 60 extract 61 filter_document 61 metadata 62 subfiles 62 filter_container 62 Part III: C++ API Reference 63 Chapter 7: InputTypes and OutputTypes 65 Chapter 8: The keyview Namespace 67 The Session Class 67 Constructor 67 config 68 detect 68 filter 68 metadata_map 68 subfiles 69 The Configuration Class 69 Constructor 69 custom_pdf_metadata 69 date_time_field_codes 69 KeyView (12.2) Page 5 of 231 Filter SDK C++ Programming Guide extraction_timeout 70 filename_field_code 70 formatted_mail 70 header_and_footer 70 header_and_footer_tags 71 hidden_text 71 no_encoding_conversion 71 out_of_process_log 71 out_of_process_memory_log 72 password 72 pdf_logical_reading 72 revision_marks 72 skip_comments 73 skip_embedded_fonts 73 skip_thumbnail 73 soft_hyphens 73 source_encoding 74 tagged_pdf_content 74 target_encoding 74 string& temporary_directory 74 timeout 75 unicode_byte_order_marker 75 The DetectionInfo Class 75 appleDoubleEncoded 75 appleSingleEncoded 75 category 76 category_name 76 description 76 encrypted 76 extension 76 format 77 macBinaryEncoded 77 version 77 wangGDLencoded 77 windowRMSEncrypted 77 The Container Class 77 The Subfile Class 78 extract 78 children 78 index 78 is_folder 79 mail_metadata 79 parent 79 rawname 79 size 79 time 80 KeyView (12.2) Page 6 of 231 Filter SDK C++ Programming Guide Enumerations 80 LogicalPDFDirection 80 Exceptions 80 Chapter 9: The keyview::io Namespace 83 InputFile 83 Constructors 83 OutputFile 83 Constructors 83 OutputStdout 83 Constructors 84 InMemoryFile 84 Constructors 84 Appendixes 85 Appendix A: Supported Formats 86 Supported Formats 86 Archive Formats 87 Binary Format 90 Computer-Aided Design Formats 91 Database Formats 92 Desktop Publishing 93 Display Formats 93 Graphic Formats 94 Mail Formats 98 Multimedia Formats 101 Presentation Formats 104 Spreadsheet Formats 107 Text and Markup Formats 109 Word Processing Formats 110 Appendix B: Detected Formats 116 Key to Detected Formats Table 116 Detected Formats 118 Appendix C: Character Sets 165 Multibyte and Bidirectional Support 165 Coded Character Sets 173 Appendix D: Extract and Format Lotus Notes Subfiles 179 Overview 179 Customize XML Templates 179 Use Demo Templates 180 Use Old Templates 180 Disable XML Templates 180 Template Elements and Attributes 181 Conditional Elements 181 Control Elements 182 KeyView (12.2) Page 7 of 231 Filter SDK C++ Programming Guide Data Elements 183 Date and Time Formats 186 Lotus Notes Date and Time Formats 186 KeyView Date and Time Formats 187 Appendix E: File Format Detection 192 Introduction 192 Extract Format Information 192 Determine Format Support 192 Example formats.ini file entries 193 Refine Detection of Text Files 193 Allow Consecutive NULL Bytes in a Text File 194 Translate Format Information 195 Distinguish Between Formats 195 Determine a Document Reader 196 Category Values in formats.ini 196 Appendix F: List of Required Files for Redistribution 200 Core Files 200 Support Files 201 Document Readers 202 Appendix G: Develop a Custom Reader 209 Introduction 209 How to Write a Custom Reader 210 Naming Conventions 210 Basic Steps 211 Token Buffer 211 Macros 212 Reader Interface 213 Function Flow 213 Example Development of fffFillBuffer() 214 Implementation 1—fpFillBuffer() Function 214 Structure of Implementation 1 215 Problems with Implementation 1 215 Implementation 2—Processing a Large Token Stream 215 Structure of Implementation 2 216 Problems with Implementation 2 217 Boundary Conditions 217 Implementation 3—Interrupting Structured Access Layer Calls 218 Structure of Implementation 3 220 Development Tips 220 Functions 221 xxxsrAutoDet() 221 xxxAllocateContext() 222 xxxFreeContext() 223 xxxInitDoc() 223 xxxFillBuffer() 224 KeyView (12.2) Page 8 of 231 Filter SDK C++ Programming Guide xxxGetSummaryInfo() 225 xxxOpenStream() 226 xxxCloseStream() 227 xxxCharSet() 227 Appendix H: Password Protected Files 229 Supported Password Protected File Types 229 Send documentation feedback 231 KeyView (12.2) Page 9 of 231 Filter SDK C++ Programming Guide KeyView (12.2) Page 10 of 231 Part I: Overview of Filter SDK This section provides an overview of the Micro Focus KeyView Filter SDK and describes how to use the C++ implementation of the API.