High Throughput JPEG 2000 (HTJ2K): Algorithm, Performance and Potential
Total Page:16
File Type:pdf, Size:1020Kb
High Throughput JPEG 2000 (HTJ2K): Algorithm, Performance and Potential Authors: David Taubman1,2, Aous Naman1,2, Reji Mathew1, Michael Smith3, Osamu Watanabe4 and Pierre-Anthony Lemieux5 Affiliations: 1. UNSW Sydney, Australia. 2. Kakadu R&D Pty. Ltd., Australia 3. Wavelet Consulting LLC, U.S.A. 4. Takushoku University, Japan 5. Sandflow Consulting LLC, U.S.A. Last updated: 29 May 2020 Forward This document is exPected to evolve over time to Provide as much information as possible to facilitate assessment and deployment of the flexible and efficient new High Throughput JPEG 2000 standard: ITU-T Rec T.814 | IS 15444-15. TABLE OF CONTENTS 1. OVERVIEW ....................................................................................................................................... 3 1.1. THROUGHPUT AND KEY FEATURES OF HTJ2K ........................................................................................... 4 1.2. QUALITY SCALABILITY IN HTJ2K ................................................................................................................. 4 1.3. RELEVANT STANDARDS .................................................................................................................................. 5 1.4. ROYALTY STATUS OF HTJ2K ......................................................................................................................... 5 1.5. SOFTWARE TOOLS AND RESOURCES FOR HTJ2K ....................................................................................... 6 1.6. ORGANIZATION OF THIS DOCUMENT ............................................................................................................ 7 2. HTJ2K USAGE SCENARIOS .......................................................................................................... 8 2.1. DISTRIBUTION, ARCHIVAL AND MANAGEMENT OF CINEMA AND VIDEO ASSETS .................................... 8 2.2. ENERGY EFFICIENT IMAGE CAPTURE AND PREVIEW ................................................................................... 8 2.3. LIVE PERFORMANCES, VIEW FINDING AND TELEPRESENCE SYSTEMS ...................................................... 9 2.4. HIGH RESOLUTION VIDEO CAPTURE ........................................................................................................... 10 2.5. CACHE MANAGEMENT FOR INTERACTIVE RENDERING ............................................................................ 10 2.6. OPTIMIZED IMAGE DELIVERY IN WEBSERVERS ........................................................................................ 11 3. OVERVIEW OF THE HTJ2K CODEC ........................................................................................ 12 3.1. HIGH LEVEL OVERVIEW ............................................................................................................................... 12 3.2. RELATIONSHIP BETWEEN HT AND J2K-1 BLOCK CODING PASSES ..................................................... 12 3.3. RATE CONTROL AND COMPLEXITY CONTROL FOR HTJ2K ENCODERS .................................................. 14 3.4. Q-FACTOR FOR JPEG 2000 ....................................................................................................................... 16 4. INTRODUCTION TO THE HT BLOCK CODING ALGORITHM .......................................... 20 4.1. HT CODEWORD SEGMENTS AND CONCURRENCY ...................................................................................... 20 4.2. HT REFINEMENT PASSES ............................................................................................................................. 20 4.3. CLEANUP MAGNITUDE EXPONENTS, EXPONENT BOUNDS AND MAGSGN VALUES ............................... 21 4.4. MAIN ELEMENTS OF THE HT CLEANUP PASS ........................................................................................... 22 4.5. FORMATION OF THE HT CLEANUP AND REFINEMENT BYTE-STREAMS ............................................... 25 5. LOW COMPLEXITY AND HIGH CONCURRENCY ATTRIBUTES OF HTJ2K .................. 27 5.1. PARALLELISM AND CONCURRENCY ............................................................................................................ 27 5.2. ENCODING RESOURCES AND VECTORIZABILITY ........................................................................................ 28 5.3. DECODING RESOURCES AND VECTORIZABILITY ........................................................................................ 29 6. HTJ2K CODE-STREAM STRUCTURE AND FEATURES ...................................................... 31 7. CODING EFFICIENCY AND THROUGHPUT IN SOFTWARE ............................................. 32 7.1. COMPRESSION OF PROFESSIONAL PHOTOGRAPHIC MATERIAL .............................................................. 32 7.2. 4K 4:4:4 12BIT VIDEO ENCODING AND DECODING ON AN I7 4-CORE CPU ....................................... 35 7.3. 4K 4:2:2 10BIT VIDEO ENCODING AND DECODING ON AN I7 4-CORE CPU ....................................... 41 7.4. COMPRESSION OF LARGE AERIAL IMAGERY .............................................................................................. 43 7.5. COMPRESSION OF ENTERTAINMENT IMAGERY ........................................................................................ 45 7.5.1. 3840X2160P 10BIT ITU-R BT.709 4:4:4 23.976P SDR .......................................................... 46 7.5.2. 3840X2160P 12BIT 24P DCINEMA DCDM SDR .......................................................................... 48 7.5.3. 3840X2160 24P 16BIT HALF-FLOAT OPENEXR (SCENE-LINEAR HDR) ................................. 50 8. BRIEF DISCUSSION OF GPU AND HARDWARE DEPLOYMENTS ................................... 52 9. LOW-LATENCY COMPRESSION WITH HTJ2K .................................................................... 53 10. JPH COMPARISON WITH JPG .................................................................................................. 65 11. CONCLUSIONS .............................................................................................................................. 68 12. BIBLIOGRAPHY ........................................................................................................................... 69 Page 2 of 69 1. Overview JPEG 2000 provides a rich set of features that find application in many diverse fields. Some of the most important features, as found in JPEG 2000 Part-1 [1] are as follows: • Compression efficiency • Quality scalability: ability to extract almost any reduced quality rePresentation from the code-stream while retaining full coding efficiency • Resolution scalability: ability to extract almost any power-of-2 related resolution from the code-stream while retaining the full coding efficiency • Region-of-interest accessibility: ability to reconstruct or communicate an arbitrary spatial region, while retaining high coding efficiency • Parallelism: ability to perform the encoding or decoding processes in parallel across a large number of CPU cores, GPU threads or in hardware • Non-iterative optimal rate control: ability to achieve a target comPressed size without iterative encoding. • The ability to target visually relevant optimization objectives • Error resilience • ComPressed domain (i.e. very low memory) transPosition, flipPing and cropping operations • Ability to re-sequence information at the code-block, precinct or J2K packet level, so as to target progressive build-up or low latency objectives. Most of these features derive from the use of the EBCOT algorithm (Embedded BlocK Coding with OPtimized Truncation), while use of the hierarchical Discrete Wavelet Transform (DWT) also Plays an imPortant role. In addition to these core features, the JPEG 2000 suite of standards provide good support for the following applications: • Efficient and responsive remote interactive browsing of imagery (including video and animations), via JPEG 2000 Part-9 [2], also Known as the JPIP standard. • Efficient on demand rendering of arbitrary regions from huge imagery sources. • High dynamic range compression, through the use of non-linear tone curves and/or custom floating-point mappings, as defined in JPEG 2000 Part-2 [3]. • Rich metadata annotation, as also defined in JPEG 2000 Part-2. • Efficient compression of hyper-spectral and volumetric content, as also defined in JPEG 2000 Part-2 [3]. The one significant drawbacK of the JPEG 2000 standards is computational complexity. JPEG 2000 provides many modes that can be used to target efficient hardware solutions. Also, software implementations are sufficiently efficient that images of just about any displayable size can be rendered nearly instantaneously on modern platforms, including mobile devices. However, for video aPPlications and for applications that are especially power/energy conscious, compression and rendering complexity can become a problem. This document describes a droP-in rePlacement for the original JPEG 2000 block coding algorithm, along with its imPlications for the whole JPEG 2000 family of standards and for a variety of image and video compression, archival and rendering tasKs. In this document, we refer to the original blocK coding algorithm, defined in JPEG 2000 Part-1 as the “J2K-1” algorithm, while the new blocK coder is Known as the “HT” (High Throughput) algorithm. Page 3 of 69 Terminology and Standards: - HT = “High Throughput” - HTJ2K code-stream = a coded image representation conforming to ITU-T Rec T.814 | IS 15444-15, read as “High Throughput JPEG 2000, which is based on the representation in JPEG 2000 Part-1