DOCSLIB.ORG

Optimization Methods for Data Compression

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Optimization Methods for Data Compression OPTIMIZATION METHODS FOR DATA COMPRESSION A Dissertation Presented to The Faculty of the Graduate School of Arts and Sciences Brandeis University Computer Science James A. Storer Advisor In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy by Giovanni Motta May 2002 This dissertation, directed and approved by Giovanni Motta's Committee, has been accepted and approved by the Graduate Faculty of Brandeis University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY _________________________ Dean of Arts and Sciences Dissertation Committee __________________________ James A. Storer ____________________________ Martin Cohn ____________________________ Jordan Pollack ____________________________ Bruno Carpentieri ii To My Parents. iii ACKNOWLEDGEMENTS I wish to thank: Bruno Carpentieri, Martin Cohn, Antonella Di Lillo, Jordan Pollack, Francesco Rizzo, James Storer for their support and collaboration. I also thank Jeanne DeBaie, Myrna Fox, Julio Santana for making my life at Brandeis easier and enjoyable. iv ABSTRACT Optimization Methods for Data Compression A dissertation presented to the Faculty of the Graduate School of Arts and Sciences of Brandeis University, Waltham, Massachusetts by Giovanni Motta Many data compression algorithms use ad–hoc techniques to compress data efficiently. Only in very few cases, can data compressors be proved to achieve optimality on a specific information source, and even in these cases, algorithms often use sub–optimal procedures in their execution. It is appropriate to ask whether the replacement of a sub–optimal strategy by an optimal one in the execution of a given algorithm results in a substantial improvement of its performance. Because of the differences between algorithms the answer to this question is domain dependent and our investigation is based on a case–by–case analysis of the effects of using an optimization procedure in a data compression algorithm. The question that we want to answer is how and how much the replacement of a sub– optimal strategy by an optimal one influences the performance of a data compression algorithm. We analyze three algorithms, each in a different domain of data compression: vector quantization, lossless image compression and video coding. Two algorithms are new, introduced by us and one is a widely accepted and well–known standard in video coding to which we apply a novel optimized rate control. Besides the contributions consisting of the introduction of two new data compression algorithms that improve the current state of the art, and the introduction of a novel rate control algorithm suitable for video compression, this work is relevant for a number of reasons: • A measure of the improvement achievable by an optimal strategy provides powerful insights about the best performance obtainable by a data compression algorithm; • As we show in the case of low bit rate video compression, optimal algorithms can frequently be simplified to provide effective heuristics; • Existing and new heuristics can be carefully evaluated by comparing their complexity and performance to the characteristics of an optimal solution; • Since the empirical entropy of a “natural” data source is always unknown, optimal data compression algorithms provide improved upper bounds on that measure. v CONTENTS CONTENTS.....................................................................................................................VI INTRODUCTION............................................................................................................. 1 KEY TECHNOLOGIES IN DATA COMPRESSION.................................................. 8 2.1 SIGNAL REPRESENTATION........................................................................................ 11 2.1.1 Sampling .......................................................................................................... 11 2.1.2 Quantization..................................................................................................... 13 2.2 DIGITAL DATA FORMATS......................................................................................... 15 2.2.1 Audio Formats ................................................................................................. 15 2.2.2 Still Image Formats.......................................................................................... 16 2.2.3 Digital Video Formats ..................................................................................... 17 2.3 BASIC METHODS ...................................................................................................... 20 2.3.1 Entropy Coding................................................................................................ 20 2.3.2 Run Length Coding .......................................................................................... 21 2.3.3 Huffman Coding............................................................................................... 21 2.3.4 Arithmetic Coding............................................................................................ 23 2.3.5 Golomb Coding................................................................................................ 25 2.3.6 Textual Substitution Methods........................................................................... 26 2.3.7 Statistical Methods........................................................................................... 28 2.3.8 Vector Quantization......................................................................................... 29 2.3.9 Prediction......................................................................................................... 30 2.3.10 Transform and Sub–Band Coding ................................................................. 33 2.3.11 Fractal Coding............................................................................................... 42 2.4 INTER BAND DECORRELATION................................................................................. 42 2.4.1 Color Decorrelation......................................................................................... 44 2.4.2 Motion Compensation...................................................................................... 47 2.4.3 Multi and Hyperspectral Images ..................................................................... 49 2.5 QUALITY ASSESSMENT ............................................................................................ 51 2.5.1 Digital Images.................................................................................................. 53 2.5.2 Video ................................................................................................................ 57 DATA COMPRESSION STANDARDS....................................................................... 60 3.1 AUDIO ...................................................................................................................... 60 3.1.1 Pulse Code Modulation.................................................................................... 60 3.1.2 MPEG Audio.................................................................................................... 60 vi 3.2 SPEECH .................................................................................................................... 65 3.2.1 A–law and µ–law ............................................................................................. 65 3.2.2 Differential PCM ............................................................................................. 67 3.2.3 LPC–10 (Linear Predictive Coding of 10th order).......................................... 69 3.2.4 Codebook Excited Linear Prediction (CELP) ................................................. 75 3.3 IMAGE ...................................................................................................................... 76 3.3.1 JPEG................................................................................................................ 76 3.3.2 JPEG–LS.......................................................................................................... 79 3.3.3 JBIG ................................................................................................................. 81 3.3.4 JPEG–2000...................................................................................................... 83 3.3.5 GIF................................................................................................................... 84 3.4 VIDEO ...................................................................................................................... 85 3.4.1 H.261................................................................................................................ 85 3.4.2 MPEG–1 .......................................................................................................... 87 3.4.3 H.263................................................................................................................ 91 3.4.4 MPEG–2 .......................................................................................................... 97 3.4.5 MPEG–4 ........................................................................................................ 103 TRELLIS CODED VECTOR RESIDUAL QUANTIZATION ............................... 105 4.1 BACKGROUND........................................................................................................ 105 4.2 INTRODUCTION TO THE PROBLEM .........................................................................
Load more

Recommended publications
  • Error Correction Capacity of Unary Coding
    Error Correction Capacity of Unary Coding Pushpa Sree Potluri1 Abstract Unary coding has found applications in data compression, neural network training, and in explaining the production mechanism of birdsong. Unary coding is redundant; therefore it should have inherent error correction capacity. An expression for the error correction capability of unary coding for the correction of single errors has been derived in this paper. 1. Introduction The unary number system is the base-1 system. It is the simplest number system to represent natural numbers. The unary code of a number n is represented by n ones followed by a zero or by n zero bits followed by 1 bit [1]. Unary codes have found applications in data compression [2],[3], neural network training [4]-[11], and biology in the study of avian birdsong production [12]-14]. One can also claim that the additivity of physics is somewhat like the tallying of unary coding [15],[16]. Unary coding has also been seen as the precursor to the development of number systems [17]. Some representations of unary number system use n-1 ones followed by a zero or with the corresponding number of zeroes followed by a one. Here we use the mapping of the left column of Table 1. Table 1. An example of the unary code N Unary code Alternative code 0 0 0 1 10 01 2 110 001 3 1110 0001 4 11110 00001 5 111110 000001 6 1111110 0000001 7 11111110 00000001 8 111111110 000000001 9 1111111110 0000000001 10 11111111110 00000000001 The unary number system may also be seen as a space coding of numerical information where the location determines the value of the number.
    [Show full text]
  • A Deblocking Filter Hardware Architecture for the High Efficiency
    A Deblocking Filter Hardware Architecture for the High Efficiency Video Coding Standard Cláudio Machado Diniz1, Muhammad Shafique2, Felipe Vogel Dalcin1, Sergio Bampi1, Jörg Henkel2 1Informatics Institute, PPGC, Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil 2Chair for Embedded Systems (CES), Karlsruhe Institute of Technology (KIT), Germany {cmdiniz, fvdalcin, bampi}@inf.ufrgs.br; {muhammad.shafique, henkel}@kit.edu Abstract—The new deblocking filter (DF) tool of the next encoder configuration: (i) Random Access (RA) configuration1 generation High Efficiency Video Coding (HEVC) standard is with Group of Pictures (GOP) equal to 8 (ii) Intra period2 for one of the most time consuming algorithms in video decoding. In each video sequence is defined as in [8] depending upon the order to achieve real-time performance at low-power specific frame rate of the video sequence, e.g. 24, 30, 50 or 60 consumption, we developed a hardware accelerator for this filter. frames per second (fps); (iii) each sequence is encoded with This paper proposes a high throughput hardware architecture four different Quantization Parameter (QP) values for HEVC deblocking filter employing hardware reuse to QP={22,27,32,37} as defined in the HEVC Common Test accelerate filtering decision units with a low area cost. Our Conditions [8]. Fig. 1 shows the accumulated execution time architecture achieves either higher or equivalent throughput (in % of total decoding time) of all functions included in C++ (4096x2048 @ 60 fps) with 5X-6X lower area compared to state- class TComLoopFilter that implement the DF in HEVC of-the-art deblocking filter architectures. decoder software. DF contributes to up to 5%-18% to the total Keywords—HEVC coding; Deblocking Filter; Hardware decoding time, depending on video sequence and QP.
    [Show full text]
  • An Efficient Pipeline Architecture for Deblocking Filter in H.264/AVC
    IEICE TRANS. INF. & SYST., VOL.E90–D, NO.1 JANUARY 2007 99 PAPER Special Section on Advanced Image Technology An Efficient Pipeline Architecture for Deblocking Filter in H.264/AV C Chung-Ming CHEN†a), Member and Chung-Ho CHEN†b), Nonmember SUMMARY In this paper, we study and analyze the computational complexity of deblocking filter in H.264/AVC baseline decoder based on SimpleScalar/ARM simulator. The simulation result shows that the mem- ory reference, content activity check operations, and filter operations are known to be very time consuming in the decoder of this new video cod- ing standard. In order to improve overall system performance, we propose a novel processing order with efficient VLSI architecture which simultane- ously processes the horizontal filtering of vertical edge and vertical filtering of horizontal edge. As a result, the memory performance of the proposed architecture is improved by four times when compared to the software im- plementation. Moreover, the system performance of our design signifi- cantly outperforms the previous proposals. key words: deblocking filter, H.264/AVC, video coding 1. Introduction Fig. 1 Block diagram of H.264/AV C . Video compression is a critical component in today’s multi- media systems. The limited transmission bandwidth or stor- As our experiment result indicates, the operation of age capacity for applications such as DVD, digital televi- the deblocking filter is the most time consuming part of sion, or internet video streaming emphasizes the demand for the H.264/AVC video decoder. The block-based structure higher video compression rates. To achieve this demand, of the H.264/AVC architecture produces artifacts known as the new video coding standard Recommendation H.264 of blocking artifacts.
    [Show full text]
  • THIAGO JOSÉ CÓSER Possibilidades Da Produção Artística Via
    THIAGO JOSÉ CÓSER Possibilidades da produção artística via prototipagem rápida: processos CAD/CAM na elaboração e confecção de obras de arte e o vislumbre de um percurso poético individualizado neste ensaio. Dissertação apresentada ao Instituto de Artes da Universidade Estadual de Campinas, para a obtenção do título de mestre em Artes. Área de concentração: Artes Visuais Orientador: Prof. Dr. Marco Antonio Alves do Valle Campinas 2010 3 FICHA CATALOGRÁFICA ELABORADA PELA BIBLIOTECA DO INSTITUTO DE ARTES DA UNICAMP Cóser, Thiago José. C89p Possibilidades da produção artística via Prototipagem Rápida: Processos CAD/CAM na elaboração e confecção de obras de arte e o vislumbre de um percurso poético individualizado neste ensaio. : Thiago José Cóser. – Campinas, SP: [s.n.], 2010. Orientador: Prof. Dr. Marco Antonio Alves do Valle. Dissertação(mestrado) - Universidade Estadual de Campinas, Instituto de Artes. 1. Prototipagem rápida. 2. Arte. 3. Sistema CAD/CAM. 4. Modelagem 3D. 5. escultura. I. Valle, Marco Antonio Alves do. II. Universidade Estadual de Campinas. Instituto de Artes. III. Título. (em/ia) Título em inglês: “Possibilities of Art via Rapid Prototyping: using CAD / CAM systems to create art works and a glimpse of a poetic route individualized essay.” Palavras-chave em inglês (Keywords): Rapid prototyping ; Art ; CAD/CAM systems. ; 3D modelling ; Sculpture. Titulação: Mestre em Artes. Banca examinadora: Prof. Dr. Marco Antonio Alves do Valle. Profª. Drª. Sylvia Helena Furegatti. Prof. Dr. Francisco Borges Filho. Prof. Dr. Carlos Roberto Fernandes. (suplente) Prof. Dr. José Mario De Martino. (suplente) Data da Defesa: 26-02-2010 Programa de Pós-Graduação: Artes. 4 5 Agradecimentos Ao meu orientador, profº Dr.
    [Show full text]
  • The Missing Link Between Information Visualization and Art
    Visualization Criticism – The Missing Link Between Information Visualization and Art Robert Kosara The University of North Carolina at Charlotte [email protected] Abstract of what constitutes visualization and a foundational theory are still missing. Even for the practical work that is be- Classifications of visualization are often based on tech- ing done, there is very little discussion of approaches, with nical criteria, and leave out artistic ways of visualizing in- many techniques being developed ad hoc or as incremental formation. Understanding the differences between informa- improvements of previous work. tion visualization and other forms of visual communication Since this is not a technical problem, a purely techni- provides important insights into the way the field works, cal approach cannot solve it. We therefore propose a third though, and also shows the path to new approaches. way of doing information visualization that not only takes We propose a classification of several types of informa- ideas from both artistic and pragmatic visualization, but uni- tion visualization based on aesthetic criteria. The notions fies them through the common concepts of critical thinking of artistic and pragmatic visualization are introduced, and and criticism. Visualization criticism can be applied to both their properties discussed. Finally, the idea of visualiza- artistic and pragmatic visualization, and will help to develop tion criticism is proposed, and its rules are laid out. Visu- the tools to build a bridge between them. alization criticism bridges the gap between design, art, and technical/pragmatic information visualization. It guides the view away from implementation details and single mouse 2 Related Work clicks to the meaning of a visualization.
    [Show full text]
  • Soft Compression for Lossless Image Coding
    1 Soft Compression for Lossless Image Coding Gangtao Xin, and Pingyi Fan, Senior Member, IEEE Abstract—Soft compression is a lossless image compression method, which is committed to eliminating coding redundancy and spatial redundancy at the same time by adopting locations and shapes of codebook to encode an image from the perspective of information theory and statistical distribution. In this paper, we propose a new concept, compressible indicator function with regard to image, which gives a threshold about the average number of bits required to represent a location and can be used for revealing the performance of soft compression. We investigate and analyze soft compression for binary image, gray image and multi-component image by using specific algorithms and compressible indicator value. It is expected that the bandwidth and storage space needed when transmitting and storing the same kind of images can be greatly reduced by applying soft compression. Index Terms—Lossless image compression, information theory, statistical distributions, compressible indicator function, image set compression. F 1 INTRODUCTION HE purpose of image compression is to reduce the where H(X1;X2; :::; Xn) is the joint entropy of the symbol T number of bits required to represent an image as much series fXi; i = 1; 2; :::; ng. as possible under the condition that the fidelity of the It’s impossible to reach the entropy rate and everything reconstructed image to the original image is higher than a we can do is to make great efforts to get close to it. Soft com- required value. Image compression often includes two pro- pression [1] was recently proposed, which uses locations cesses, encoding and decoding.
    [Show full text]
  • Variable Block-Based Deblocking Filter for H.264/Avc
    VARIABLE BLOCK-BASED DEBLOCKING FILTER FOR H.264/AVC Seung-Ho Shin, Young-Joon Chai, Kyu-Sik Jang, Tae-Yong Kim GSAIM, Chung-Ang University, Seoul, Korea ABSTRACT macroblocks up to 4x4 blocks, it is possible to decrease artifacts on the boundaries between 4x4 blocks. In the The deblocking filter in H.264/AVC is used on low-end process of decreasing the blocking artifacts, however, the terminals to a limited extent due to computational actual images’ edges may erroneously be blurred. And it complexity. In this paper, Variable block-based deblocking may not be used on low-end terminals due to complex filter that efficiently eliminates the blocking artifacts computation and large memory capacity. Despite such occurred during the compression of low-bit rates digital shortcomings, however, the deblocking filter can be said to motion pictures is suggested. Blocking artifacts are plaid be the most essential technology in enhancing the subjective images appear on the block boundaries due to DCT and image quality. The general opinion of the subjective image quantization. In the method proposed in this paper, the quality test proves that there is distinguishing difference in image's spatial correlational characteristics are extracted by the image qualities with and without the deblocking filter using the variable block information of motion used [6]. compensation; the filtering is divided into 4 modes according to the characteristics, and adaptive filtering is In this paper, we suggest a block-based deblocking filter by executed in the divided regions. The proposed deblocking using the variable block information of the motion method eliminates the blocking artifacts, prevents excessive compensation.
    [Show full text]
  • Colorization Algorithm Using Probabilistic Relaxation
    Colorization Algorithm Using Probabilistic Relaxation Takahiko HORIUCHI Department of Information and Image Sciences, Faculty of Engineering Chiba University 1-33, Yayoi-cho, Inage-ku, Chiba 263-8522, Japan Email: [email protected] TEL/FAX +81-43-290-3485 ABSTRACT: This paper presents a method of colorizing a black and white imagery based on the probabilistic relaxation algorithm. Since the colorization is an ill-posed problem, a user specifies a suitable color on each isolated pixel of an image as a prior information in this paper. Then other pixels in the image are colorized automatically. The colorizing process is done by assuming local Markov property on the images. By minimizing a total of RGB pixel-wise differences, the problem can be considered as a combinatorial optimization problem and it is solved by using the probabilistic relaxation. The proposed algorithm works very well when a few percent color pixels are known with confidence. Keywords – Colorization, probabilistic relaxation, Markov property, grayscale image Colorization Algorithm Using Probabilistic Relaxation 1. INTRODUCTION Colorization is a computerized process that adds color to a black and white print, movie and TV program, supposedly invented by Wilson Markle. It was initially used in 1970 to add color to footage of the moon from the Apollo mission. The demand of adding color to grayscale images such as BW movies and BW photos has been increasing. For example, in the amusement field, many movies and video clips have been colorized by human’s labor, and many grayscale images have been distributed as vivid images. In other fields such as archaeology dealing with historical grayscale data and security dealing with grayscale images by a crime prevention camera, we can imagine easily that colorization techniques are useful.
    [Show full text]
  • Deblocking Scheme for JPEG-Coded Images Using Sparse Representation and All Phase Biorthogonal Transform
    Journal of Communications Vol. 11, No. 12, December 2016 Deblocking Scheme for JPEG-Coded Images Using Sparse Representation and All Phase Biorthogonal Transform Liping Wang, Chengyou Wang, and Xiao Zhou School of Mechanical, Electrical and Information Engineering, Shandong University, Weihai 264209, China Email: [email protected]; {wangchengyou, zhouxiao}@sdu.edu.cn Abstract—For compressed images, a major drawback is that The image deblocking algorithm aims to alleviate those images will exhibit severe blocking artifacts at very low blocking artifacts and improve visual quality of bit rates due to adopting Block-Based Discrete Cosine compressed images. Deblocking methods can be mainly Transform (BDCT). In this paper, a novel deblocking scheme divided into two categories: in-loop processing methods using sparse representation is proposed. A new transform called All Phase Biorthogonal Transform (APBT) was proposed in and post-processing methods. The in-loop deblocking recent years. APBT has the similar energy compaction property processing methods not only avoid the propagation of with Discrete Cosine Transform (DCT). It has very good blocking artifacts between adjacent frames, but also can column properties, high frequency attenuation characteristics, enhance coding efficiency. For example, H.264/AVC [2] low frequency energy aggregation, and so on. In this paper, we adopts the in-loop deblocking processing. Some use it to generate the over-completed dictionary for sparse researchers have designed the in-loop deblocking filter coding. For Orthogonal Matching Pursuit (OMP), we select an [4]. Numerous experimental results manifest that the in- adaptive residual threshold by combining blind image blocking assessment. Experimental results show that this new scheme is loop deblocking filter can provide both objective and effective in image deblocking and can avoid over-blurring of subjective improvement compared with the compressed edges and textures.
    [Show full text]
  • Generalized Golomb Codes and Adaptive Coding of Wavelet-Transformed Image Subbands
    IPN Progress Report 42-154 August 15, 2003 Generalized Golomb Codes and Adaptive Coding of Wavelet-Transformed Image Subbands A. Kiely1 and M. Klimesh1 We describe a class of prefix-free codes for the nonnegative integers. We apply a family of codes in this class to the problem of runlength coding, specifically as part of an adaptive algorithm for compressing quantized subbands of wavelet- transformed images. On test images, our adaptive coding algorithm is shown to give compression effectiveness comparable to the best performance achievable by an alternate algorithm that has been previously implemented. I. Generalized Golomb Codes A. Introduction Suppose we wish to use a variable-length binary code to encode integers that can take on any non- negative value. This problem arises, for example, in runlength coding—encoding the lengths of runs of a dominant output symbol from a discrete source. Encoding cannot be accomplished by simply looking up codewords from a table because the table would be infinite, and Huffman’s algorithm could not be used to construct the code anyway. Thus, one would like to use a simply described code that can be easily encoded and decoded. Although in practice there is generally a limit to the size of the integers that need to be encoded, a simple code for the nonnegative integers is often still the best choice. We now describe a class of variable-length binary codes for the nonnegative integers that contains several useful families of codes. Each code in this class is completely specified by an index function f from the nonnegative integers onto the nonnegative integers.
    [Show full text]
  • The Pillars of Lossless Compression Algorithms a Road Map and Genealogy Tree
    International Journal of Applied Engineering Research ISSN 0973-4562 Volume 13, Number 6 (2018) pp. 3296-3414 © Research India Publications. http://www.ripublication.com The Pillars of Lossless Compression Algorithms a Road Map and Genealogy Tree Evon Abu-Taieh, PhD Information System Technology Faculty, The University of Jordan, Aqaba, Jordan. Abstract tree is presented in the last section of the paper after presenting the 12 main compression algorithms each with a practical This paper presents the pillars of lossless compression example. algorithms, methods and techniques. The paper counted more than 40 compression algorithms. Although each algorithm is The paper first introduces Shannon–Fano code showing its an independent in its own right, still; these algorithms relation to Shannon (1948), Huffman coding (1952), FANO interrelate genealogically and chronologically. The paper then (1949), Run Length Encoding (1967), Peter's Version (1963), presents the genealogy tree suggested by researcher. The tree Enumerative Coding (1973), LIFO (1976), FiFO Pasco (1976), shows the interrelationships between the 40 algorithms. Also, Stream (1979), P-Based FIFO (1981). Two examples are to be the tree showed the chronological order the algorithms came to presented one for Shannon-Fano Code and the other is for life. The time relation shows the cooperation among the Arithmetic Coding. Next, Huffman code is to be presented scientific society and how the amended each other's work. The with simulation example and algorithm. The third is Lempel- paper presents the 12 pillars researched in this paper, and a Ziv-Welch (LZW) Algorithm which hatched more than 24 comparison table is to be developed.
    [Show full text]
  • New Approaches to Fine-Grain Scalable Audio Coding
    New Approaches to Fine-Grain Scalable Audio Coding Mahmood Movassagh Department of Electrical & Computer Engineering McGill University Montreal, Canada December 2015 Research Thesis submitted to McGill University in partial fulfillment of the requirements for the degree of PhD. c 2015 Mahmood Movassagh In memory of my mother whom I lost in the last year of my PhD studies To my father who has been the greatest support for my studies in my life Abstract Bit-rate scalability has been a useful feature in the multimedia communications. Without the need to re-encode the original signal, it allows for improving/decreasing the quality of a signal as more/less of a total bit stream becomes available. Using scalable coding, there is no need to store multiple versions of a signal encoded at different bit-rates. Scalable coding can also be used to provide users with different quality streaming when they have different constraints or when there is a varying channel; i.e., the receivers with lower channel capacities will be able to receive signals at lower bit-rates. It is especially useful in the client- server applications where the network nodes are able to drop some enhancement layer bits to satisfy link capacity constraints. In this dissertation, we provide three contributions to practical scalable audio coding systems. Our first contribution is the scalable audio coding using watermarking. The proposed scheme uses watermarking to embed some of the information of each layer into the previous layer. This approach leads to a saving in bitrate, so that it outperforms (in terms of rate-distortion) the common scalable audio coding based on the reconstruction error quantization (REQ) as used in MPEG-4 audio.
    [Show full text]