OPTIMIZATION METHODS FOR DATA COMPRESSION A Dissertation Presented to The Faculty of the Graduate School of Arts and Sciences Brandeis University Computer Science James A. Storer Advisor In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy by Giovanni Motta May 2002 This dissertation, directed and approved by Giovanni Motta's Committee, has been accepted and approved by the Graduate Faculty of Brandeis University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY _________________________ Dean of Arts and Sciences Dissertation Committee __________________________ James A. Storer ____________________________ Martin Cohn ____________________________ Jordan Pollack ____________________________ Bruno Carpentieri ii To My Parents. iii ACKNOWLEDGEMENTS I wish to thank: Bruno Carpentieri, Martin Cohn, Antonella Di Lillo, Jordan Pollack, Francesco Rizzo, James Storer for their support and collaboration. I also thank Jeanne DeBaie, Myrna Fox, Julio Santana for making my life at Brandeis easier and enjoyable. iv ABSTRACT Optimization Methods for Data Compression A dissertation presented to the Faculty of the Graduate School of Arts and Sciences of Brandeis University, Waltham, Massachusetts by Giovanni Motta Many data compression algorithms use ad–hoc techniques to compress data efficiently. Only in very few cases, can data compressors be proved to achieve optimality on a specific information source, and even in these cases, algorithms often use sub–optimal procedures in their execution. It is appropriate to ask whether the replacement of a sub–optimal strategy by an optimal one in the execution of a given algorithm results in a substantial improvement of its performance. Because of the differences between algorithms the answer to this question is domain dependent and our investigation is based on a case–by–case analysis of the effects of using an optimization procedure in a data compression algorithm. The question that we want to answer is how and how much the replacement of a sub– optimal strategy by an optimal one influences the performance of a data compression algorithm. We analyze three algorithms, each in a different domain of data compression: vector quantization, lossless image compression and video coding. Two algorithms are new, introduced by us and one is a widely accepted and well–known standard in video coding to which we apply a novel optimized rate control. Besides the contributions consisting of the introduction of two new data compression algorithms that improve the current state of the art, and the introduction of a novel rate control algorithm suitable for video compression, this work is relevant for a number of reasons: • A measure of the improvement achievable by an optimal strategy provides powerful insights about the best performance obtainable by a data compression algorithm; • As we show in the case of low bit rate video compression, optimal algorithms can frequently be simplified to provide effective heuristics; • Existing and new heuristics can be carefully evaluated by comparing their complexity and performance to the characteristics of an optimal solution; • Since the empirical entropy of a “natural” data source is always unknown, optimal data compression algorithms provide improved upper bounds on that measure. v CONTENTS CONTENTS.....................................................................................................................VI INTRODUCTION............................................................................................................. 1 KEY TECHNOLOGIES IN DATA COMPRESSION.................................................. 8 2.1 SIGNAL REPRESENTATION........................................................................................ 11 2.1.1 Sampling .......................................................................................................... 11 2.1.2 Quantization..................................................................................................... 13 2.2 DIGITAL DATA FORMATS......................................................................................... 15 2.2.1 Audio Formats ................................................................................................. 15 2.2.2 Still Image Formats.......................................................................................... 16 2.2.3 Digital Video Formats ..................................................................................... 17 2.3 BASIC METHODS ...................................................................................................... 20 2.3.1 Entropy Coding................................................................................................ 20 2.3.2 Run Length Coding .......................................................................................... 21 2.3.3 Huffman Coding............................................................................................... 21 2.3.4 Arithmetic Coding............................................................................................ 23 2.3.5 Golomb Coding................................................................................................ 25 2.3.6 Textual Substitution Methods........................................................................... 26 2.3.7 Statistical Methods........................................................................................... 28 2.3.8 Vector Quantization......................................................................................... 29 2.3.9 Prediction......................................................................................................... 30 2.3.10 Transform and Sub–Band Coding ................................................................. 33 2.3.11 Fractal Coding............................................................................................... 42 2.4 INTER BAND DECORRELATION................................................................................. 42 2.4.1 Color Decorrelation......................................................................................... 44 2.4.2 Motion Compensation...................................................................................... 47 2.4.3 Multi and Hyperspectral Images ..................................................................... 49 2.5 QUALITY ASSESSMENT ............................................................................................ 51 2.5.1 Digital Images.................................................................................................. 53 2.5.2 Video ................................................................................................................ 57 DATA COMPRESSION STANDARDS....................................................................... 60 3.1 AUDIO ...................................................................................................................... 60 3.1.1 Pulse Code Modulation.................................................................................... 60 3.1.2 MPEG Audio.................................................................................................... 60 vi 3.2 SPEECH .................................................................................................................... 65 3.2.1 A–law and µ–law ............................................................................................. 65 3.2.2 Differential PCM ............................................................................................. 67 3.2.3 LPC–10 (Linear Predictive Coding of 10th order).......................................... 69 3.2.4 Codebook Excited Linear Prediction (CELP) ................................................. 75 3.3 IMAGE ...................................................................................................................... 76 3.3.1 JPEG................................................................................................................ 76 3.3.2 JPEG–LS.......................................................................................................... 79 3.3.3 JBIG ................................................................................................................. 81 3.3.4 JPEG–2000...................................................................................................... 83 3.3.5 GIF................................................................................................................... 84 3.4 VIDEO ...................................................................................................................... 85 3.4.1 H.261................................................................................................................ 85 3.4.2 MPEG–1 .......................................................................................................... 87 3.4.3 H.263................................................................................................................ 91 3.4.4 MPEG–2 .......................................................................................................... 97 3.4.5 MPEG–4 ........................................................................................................ 103 TRELLIS CODED VECTOR RESIDUAL QUANTIZATION ............................... 105 4.1 BACKGROUND........................................................................................................ 105 4.2 INTRODUCTION TO THE PROBLEM .........................................................................