Power NX Gzip Accelerator User’s Manual Version 0.9 Advance Dec 7, 2018 Public DRAFT Disclaimer © Copyright International Business Machines Corporation 2017 A.D. All Rights Reserved Printed in the United States of America Dec 7, 2018 The following are trademarks of International Business Machines Corporation in the United States, or other countries, or both. IBM IBM Logo Other company, product, and service names may be trademarks or service marks of others. All information contained in this document is subject to change without notice. The products described in this document are NOT intended for use in applications such as implantation, life support, or other hazardous uses where malfunction could result in death, bodily injury, or catastrophic property damage. The information contained in this document does not affect or change IBM product specifications or warranties. Nothing in this document shall operate as an express or implied license or indemnity under the intellectual property rights of IBM or third parties. All information contained in this docu- ment was obtained in specific environments, and is presented as an illustration. The results obtained in other operating environments may vary. Advance Products - Legal Disclaimer While the information contained herein is believed to be accurate, such information is preliminary, and should not be relied upon for accuracy or completeness, and no representations or warranties of accuracy or completeness are made. Note: This document contains information on products in the design, sampling and/or initial production phases of development. This information is subject to change without notice. Verify with your IBM field applications engineer that you have the latest version of this document before finalizing a design. THE INFORMATION CONTAINED IN THIS DOCUMENT IS PROVIDED ON AN “AS IS” BASIS. In no event will IBM be liable for damages arising directly or indirectly from any use of the information contained in this document. The IBM home page can be found at http://www.ibm.com nx_wb_title.fm.0.9 Dec 7, 2018 Public DRAFT Advance P9 NX Gzip Accelerator Table of Contents Table of Contents ........................................................................................................... 3 List of Figures ................................................................................................................ 6 List of Tables .................................................................................................................. 7 Preface ............................................................................................................................ 8 About this Book ...................................................................................................................................... 8 Who Should Use this Book .................................................................................................................... 8 Document Control .................................................................................................................................. 8 Document Owner ................................................................................................................................... 8 1. Overview ...................................................................................................................... 9 2. Understanding the NX-gzip accelerator for software integration ....................... 11 2.1 The accelerator user interface: basics ............................................................................................ 11 2.2 Embedded implementation concepts .............................................................................................. 12 2.3 Zlib integration concepts ................................................................................................................. 12 2.4 Understanding DEFLATE, GZIP and ZLIB compressed data formats ............................................ 13 2.4.1 GZIP (RFC1952) and ZLIB (RFC1950) header and trailer formats ....................................... 13 2.5 Deflate compressed data format ..................................................................................................... 14 2.5.1 Deflate stream ....................................................................................................................... 14 2.5.2 Deflate block header .............................................................................................................. 14 2.5.3 Bit and byte ordering (when to examining binary dumps) ..................................................... 14 2.5.4 Literal block (also called the Stored block) ............................................................................ 15 2.5.5 Creating Multiple block streams ............................................................................................ 15 2.5.6 Z_SYNC_FLUSH is an empty literal block for byte aligning multiple blocks in a stream ...... 16 2.5.7 LZ77 and Huffman encoding ................................................................................................. 16 2.5.8 Fixed Huffman block .............................................................................................................. 16 2.5.9 Dynamic Huffman block ......................................................................................................... 17 2.5.9.1 Counting LZ77 symbols for compressing with the dynamic Huffman functions ............. 17 2.5.9.2 How to make a DHT code table ..................................................................................... 17 2.5.9.3 Compression with dynamic Huffman functions .............................................................. 18 2.5.9.4 Throughput optimizations when using DH compress functions ...................................... 18 2.5.9.5 Canned and Cached DHTs: Don’t forget to initialize zero symbol counts to a non-zero value ........................................................................................................................................... 18 2.5.9.6 Making DHTs in the two pass mode: consider Initializing Length and Distance symbols counts to a non-zero value ......................................................................................................... 19 2.6 Suspend and Resume functions ..................................................................................................... 20 2.7 Suspending a compress job ............................................................................................................ 20 2.7.1 Input was final and the accelerator has written the final block .............................................. 20 2.7.2 Input was not final and more jobs are to follow ..................................................................... 20 2.8 Resuming a compress job ............................................................................................................... 21 2.8.1 Resume Compress function codes ........................................................................................ 21 2.8.2 Implementing Z_FULL_FLUSH with Compress Resume ...................................................... 21 nx_wbTOC.fm.0.5 Table of Contents May 23, 2017 Public Page 3 of 74 DRAFT Advance P9 NX Gzip Accelerator 2.9 Suspending a decompress job ........................................................................................................ 22 2.10 Resuming a decompress job ......................................................................................................... 22 2.11 Decompress single block and suspend (Z_BLOCK) ..................................................................... 23 2.12 Zlib dictionary operations .............................................................................................................. 23 2.12.0.1 Compressing with a preset dictionary (additional details) ............................................ 24 2.12.0.2 Decompressing with a preset dictionary (additional details) ........................................ 25 3. Exception handling ................................................................................................... 26 3.1 Checking forward progress with certain corrupt input ..................................................................... 26 4. Programming Notes .................................................................................................. 27 4.1 HLIT and HDIST field treatment ...................................................................................................... 27 4.2 Computing checksums when creating a literal block ...................................................................... 27 4.2.1 Computing checksums for non-compression applications .................................................... 28 4.3 User mode vs kernel mode ............................................................................................................. 28 4.4 Fairness and Queue management .................................................................................................. 28 4.4.1 What are good values for the Max Byte Count High and Low registers? .............................. 29 4.4.2 When to switch between software and hardware gzip? ........................................................ 29 4.5 Handling target buffer full condition ................................................................................................. 30 4.6 Inflating compressed