Greedy Algorithm Implementation in Huffman Coding Theory
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Large Alphabet Source Coding Using Independent Component Analysis Amichai Painsky, Member, IEEE, Saharon Rosset and Meir Feder, Fellow, IEEE
IEEE TRANSACTIONS ON INFORMATION THEORY 1 Large Alphabet Source Coding using Independent Component Analysis Amichai Painsky, Member, IEEE, Saharon Rosset and Meir Feder, Fellow, IEEE Abstract Large alphabet source coding is a basic and well–studied problem in data compression. It has many applications such as compression of natural language text, speech and images. The classic perception of most commonly used methods is that a source is best described over an alphabet which is at least as large as the observed alphabet. In this work we challenge this approach and introduce a conceptual framework in which a large alphabet source is decomposed into “as statistically independent as possible” components. This decomposition allows us to apply entropy encoding to each component separately, while benefiting from their reduced alphabet size. We show that in many cases, such decomposition results in a sum of marginal entropies which is only slightly greater than the entropy of the source. Our suggested algorithm, based on a generalization of the Binary Independent Component Analysis, is applicable for a variety of large alphabet source coding setups. This includes the classical lossless compression, universal compression and high-dimensional vector quantization. In each of these setups, our suggested approach outperforms most commonly used methods. Moreover, our proposed framework is significantly easier to implement in most of these cases. I. INTRODUCTION SSUME a source over an alphabet size m, from which a sequence of n independent samples are drawn. The classical A source coding problem is concerned with finding a sample-to-codeword mapping, such that the average codeword length is minimal, and the samples may be uniquely decodable. -
Design and Implementation of a Decompression Engine for a Huffman-Based Compressed Data Cache Master’S Thesis in Embedded Electronic System Design
Chapter 1 Introduction Design and implementation of a decompression engine for a Huffman-based compressed data cache Master’s Thesis in Embedded Electronic System Design LI KANG Department of Computer Science and Engineering CHALMERS UNIVERSITY OF TECHNOLOGY Gothenburg, Sweden, 2014 The Author grants to Chalmers University of Technology and University of Gothenburg the non-exclusive right to publish the Work electronically and in a non-commercial purpose make it accessible on the Internet. The Author warrants that he/she is the author to the Work, and warrants that the Work does not contain text, pictures or other material that violates copyright law. The Author shall, when transferring the rights of the Work to a third party (for example a publisher or a company), acknowledge the third party about this agreement. If the Author has signed a copyright agreement with a third party regarding the Work, the Author warrants hereby that he/she has obtained any necessary permission from this third party to let Chalmers University of Technology and University of Gothenburg store the Work electronically and make it accessible on the Internet. Design and implementation of a decompression engine for a Huffman-based compressed data cache Li Kang © Li Kang January 2014. Supervisor & Examiner: Angelos Arelakis, Per Stenström Chalmers University of Technology Department of Computer Science and Engineering SE-412 96 Göteborg Sweden Telephone + 46 (0)31-772 1000 [Cover: Pipelined Huffman-based decompression engine, page 8. Source: A. Arelakis and P. Stenström, “A Case for a Value-Aware Cache”, IEEE Computer Architecture Letters, September 2012.] Department of Computer Science and Engineering Göteborg, Sweden January 2014 2 Abstract This master thesis studies the implementation of a decompression engine for Huffman based compressed data cache. -
CALIFORNIA STATE UNIVERSITY, NORTHRIDGE LOSSLESS COMPRESSION of SATELLITE TELEMETRY DATA for a NARROW-BAND DOWNLINK a Graduate P
CALIFORNIA STATE UNIVERSITY, NORTHRIDGE LOSSLESS COMPRESSION OF SATELLITE TELEMETRY DATA FOR A NARROW-BAND DOWNLINK A graduate project submitted in partial fulfillment of the requirements For the degree of Master of Science in Electrical Engineering By Gor Beglaryan May 2014 Copyright Copyright (c) 2014, Gor Beglaryan Permission to use, copy, modify, and/or distribute the software developed for this project for any purpose with or without fee is hereby granted. THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. Copyright by Gor Beglaryan ii Signature Page The graduate project of Gor Beglaryan is approved: __________________________________________ __________________ Prof. James A Flynn Date __________________________________________ __________________ Dr. Deborah K Van Alphen Date __________________________________________ __________________ Dr. Sharlene Katz, Chair Date California State University, Northridge iii Contents Copyright .......................................................................................................................................... ii Signature Page ............................................................................................................................... -
Modification of Adaptive Huffman Coding for Use in Encoding Large Alphabets
ITM Web of Conferences 15, 01004 (2017) DOI: 10.1051/itmconf/20171501004 CMES’17 Modification of Adaptive Huffman Coding for use in encoding large alphabets Mikhail Tokovarov1,* 1Lublin University of Technology, Electrical Engineering and Computer Science Faculty, Institute of Computer Science, Nadbystrzycka 36B, 20-618 Lublin, Poland Abstract. The paper presents the modification of Adaptive Huffman Coding method – lossless data compression technique used in data transmission. The modification was related to the process of adding a new character to the coding tree, namely, the author proposes to introduce two special nodes instead of single NYT (not yet transmitted) node as in the classic method. One of the nodes is responsible for indicating the place in the tree a new node is attached to. The other node is used for sending the signal indicating the appearance of a character which is not presented in the tree. The modified method was compared with existing methods of coding in terms of overall data compression ratio and performance. The proposed method may be used for large alphabets i.e. for encoding the whole words instead of separate characters, when new elements are added to the tree comparatively frequently. Huffman coding is frequently chosen for implementing open source projects [3]. The present paper contains the 1 Introduction description of the modification that may help to improve Efficiency and speed – the two issues that the current the algorithm of adaptive Huffman coding in terms of world of technology is centred at. Information data savings. technology (IT) is no exception in this matter. Such an area of IT as social media has become extremely popular 2 Study of related works and widely used, so that high transmission speed has gained a great importance. -
A Survey on Different Compression Techniques Algorithm for Data Compression Ihardik Jani, Iijeegar Trivedi IC
International Journal of Advanced Research in ISSN : 2347 - 8446 (Online) Computer Science & Technology (IJARCST 2014) Vol. 2, Issue 3 (July - Sept. 2014) ISSN : 2347 - 9817 (Print) A Survey on Different Compression Techniques Algorithm for Data Compression IHardik Jani, IIJeegar Trivedi IC. U. Shah University, India IIS. P. University, India Abstract Compression is useful because it helps us to reduce the resources usage, such as data storage space or transmission capacity. Data Compression is the technique of representing information in a compacted form. The actual aim of data compression is to be reduced redundancy in stored or communicated data, as well as increasing effectively data density. The data compression has important tool for the areas of file storage and distributed systems. To desirable Storage space on disks is expensively so a file which occupies less disk space is “cheapest” than an uncompressed files. The main purpose of data compression is asymptotically optimum data storage for all resources. The field data compression algorithm can be divided into different ways: lossless data compression and optimum lossy data compression as well as storage areas. Basically there are so many Compression methods available, which have a long list. In this paper, reviews of different basic lossless data and lossy compression algorithms are considered. On the basis of these techniques researcher have tried to purpose a bit reduction algorithm used for compression of data which is based on number theory system and file differential technique. The statistical coding techniques the algorithms such as Shannon-Fano Coding, Huffman coding, Adaptive Huffman coding, Run Length Encoding and Arithmetic coding are considered. -
16.1 Digital “Modes”
Contents 16.1 Digital “Modes” 16.5 Networking Modes 16.1.1 Symbols, Baud, Bits and Bandwidth 16.5.1 OSI Networking Model 16.1.2 Error Detection and Correction 16.5.2 Connected and Connectionless 16.1.3 Data Representations Protocols 16.1.4 Compression Techniques 16.5.3 The Terminal Node Controller (TNC) 16.1.5 Compression vs. Encryption 16.5.4 PACTOR-I 16.2 Unstructured Digital Modes 16.5.5 PACTOR-II 16.2.1 Radioteletype (RTTY) 16.5.6 PACTOR-III 16.2.2 PSK31 16.5.7 G-TOR 16.2.3 MFSK16 16.5.8 CLOVER-II 16.2.4 DominoEX 16.5.9 CLOVER-2000 16.2.5 THROB 16.5.10 WINMOR 16.2.6 MT63 16.5.11 Packet Radio 16.2.7 Olivia 16.5.12 APRS 16.3 Fuzzy Modes 16.5.13 Winlink 2000 16.3.1 Facsimile (fax) 16.5.14 D-STAR 16.3.2 Slow-Scan TV (SSTV) 16.5.15 P25 16.3.3 Hellschreiber, Feld-Hell or Hell 16.6 Digital Mode Table 16.4 Structured Digital Modes 16.7 Glossary 16.4.1 FSK441 16.8 References and Bibliography 16.4.2 JT6M 16.4.3 JT65 16.4.4 WSPR 16.4.5 HF Digital Voice 16.4.6 ALE Chapter 16 — CD-ROM Content Supplemental Files • Table of digital mode characteristics (section 16.6) • ASCII and ITA2 code tables • Varicode tables for PSK31, MFSK16 and DominoEX • Tips for using FreeDV HF digital voice software by Mel Whitten, KØPFX Chapter 16 Digital Modes There is a broad array of digital modes to service various needs with more coming. -
On the Coding Gain of Dynamic Huffman Coding Applied to a Wavelet-Based Perceptual Audio Coder
ON THE CODING GAIN OF DYNAMIC HUFFMAN CODING APPLIED TO A WAVELET-BASED PERCEPTUAL AUDIO CODER N. Ruiz Reyes1, M. Rosa Zurera2, F. López Ferreras2, P. Jarabo Amores2, and P. Vera Candeas1 1Departamento de Electrónica, Universidad de Jaén, Escuela Universitaria Politénica, 23700 Linares, Jaén, SPAIN, e-mail: [email protected] 2Dpto. de Teoría de la Señal y Comunicaciones, Universidad de Alcalá, Escuela Politécnica 28871 Alcalá de Henares, Madrid, SPAIN, e-mail: [email protected] ABSTRACT The last one, the ISO/MPEG-4 standard, is composed of several speech and audio coders supporting bit rates This paper evaluates the coding gain of using a dynamic from 2 to 64 kbps per channel. ISO/MPEG-4 includes Huffman entropy coder in an audio coder that uses a the already proposed AAC standard, which provides high wavelet-packet decomposition that is close to the sub- quality audio coding at bit rates of 64 kbps per channel. band decomposition made by the human ear. The sub- Parallel to the definition of the ISO/MPEG standards, band audio signals are modeled as samples of a station- several audio coding algorithms have been proposed that ary random process with laplacian probability density use the wavelet transform as the tool to decompose the function because experimental results indicate that the audio signal. The most promising results correspond to highest coding efficiency is obtained in that case. We adapted wavelet-based audio coders. Probably, the most have also studied how the entropy coding gain varies cited is the one designed by Sinha and Tewfik [2], a high with the band index. -
Revisiting Huffman Coding: Toward Extreme Performance on Modern GPU Architectures
Revisiting Huffman Coding: Toward Extreme Performance on Modern GPU Architectures Jiannan Tian?, Cody Riveray, Sheng Diz, Jieyang Chenx, Xin Liangx, Dingwen Tao?, and Franck Cappelloz{ ?School of Electrical Engineering and Computer Science, Washington State University, WA, USA yDepartment of Computer Science, The University of Alabama, AL, USA zMathematics and Computer Science Division, Argonne National Laboratory, IL, USA xOak Ridge National Laboratory, TN, USA {University of Illinois at Urbana-Champaign, IL, USA Abstract—Today’s high-performance computing (HPC) appli- much more slowly than computing power, causing intra-/inter- cations are producing vast volumes of data, which are challenging node communication cost and I/O bottlenecks to become a to store and transfer efficiently during the execution, such that more serious issue in fast stream processing [6]. Compressing data compression is becoming a critical technique to mitigate the storage burden and data movement cost. Huffman coding is the raw simulation data at runtime and decompressing them arguably the most efficient Entropy coding algorithm in informa- before post-analysis can significantly reduce communication tion theory, such that it could be found as a fundamental step and I/O overheads and hence improving working efficiency. in many modern compression algorithms such as DEFLATE. On Huffman coding is a widely-used variable-length encoding the other hand, today’s HPC applications are more and more method that has been around for over 60 years [17]. It is relying on the accelerators such as GPU on supercomputers, while Huffman encoding suffers from low throughput on GPUs, arguably the most cost-effective Entropy encoding algorithm resulting in a significant bottleneck in the entire data processing. -
Lecture 2: Variable-Length Codes Continued
Data Compression Techniques Part 1: Entropy Coding Lecture 2: Variable-Length Codes Continued Juha K¨arkk¨ainen 01.11.2017 1 / 16 Kraft's Inequality When constructing a variable-length code, we are not really interested in what the individual codewords are as long as they satisfy two conditions: I The code is a prefix code (or at least a uniquely decodable code). I The codeword lengths are chosen to minimize the average codeword length. Kraft's inequality gives an exact condition for the existence of a prefix code in terms of the codeword lengths. Theorem (Kraft's Inequality) There exists a binary prefix code with codeword lengths `1; `2; : : : ; `σ if and only if σ X 2−`i ≤ 1 : i=1 2 / 16 Proof of Kraft's Inequality Consider a binary search on the real interval [0; 1). In each step, the current interval is split into two halves and one of the halves is chosen as the new interval. We can associate a search of ` steps with a binary string of length `: I Zero corresponds to choosing the left half. I One corresponds to choosing the right half. For any binary string w, let I(w) be the final interval of the associated search. Example 1011 corresponds to the search sequence [0; 1); [1=2; 2=2); [2=4; 3=4); [5=8; 6=8); [11=16; 12=16) and I(1011) = [11=16; 12=16). 3 / 16 Consider the set fI(w) j w 2 f0; 1g`g of all intervals corresponding to binary strings of lengths `. -
Applying Compression to a Game's Network Protocol
Applying Compression to a Game's Network Protocol Mikael Hirki Aalto University, Finland [email protected] Abstract. This report presents the results of applying different com- pression algorithms to the network protocol of an online game. The al- gorithm implementations compared are zlib, liblzma and my own imple- mentation based on LZ77 and a variation of adaptive Huffman coding. The comparison data was collected from the game TomeNET. The re- sults show that adaptive coding is especially useful for compressing large amounts of very small packets. 1 Introduction The purpose of this project report is to present and discuss the results of applying compression to the network protocol of a multi-player online game. This poses new challenges for the compression algorithms and I have also developed my own algorithm that tries to tackle some of these. Networked games typically follow a client-server model where multiple clients connect to a single server. The server is constantly streaming data that contains game updates to the clients as long as they are connected. The client will also send data to the about the player's actions. This report focuses on compressing the data stream from server to client. The amount of data sent from client to server is likely going to be very small and therefore it would not be of any interest to compress it. Many potential benefits may be achieved by compressing the game data stream. A server with limited bandwidth could theoretically serve more clients. Compression will lower the bandwidth requirements for each individual client. Compression will also likely reduce the number of packets required to transmit arXiv:1206.2362v1 [cs.IT] 28 May 2012 larger data bursts. -
The Deep Learning Solutions on Lossless Compression Methods for Alleviating Data Load on Iot Nodes in Smart Cities
sensors Article The Deep Learning Solutions on Lossless Compression Methods for Alleviating Data Load on IoT Nodes in Smart Cities Ammar Nasif *, Zulaiha Ali Othman and Nor Samsiah Sani Center for Artificial Intelligence Technology (CAIT), Faculty of Information Science & Technology, University Kebangsaan Malaysia, Bangi 43600, Malaysia; [email protected] (Z.A.O.); [email protected] (N.S.S.) * Correspondence: [email protected] Abstract: Networking is crucial for smart city projects nowadays, as it offers an environment where people and things are connected. This paper presents a chronology of factors on the development of smart cities, including IoT technologies as network infrastructure. Increasing IoT nodes leads to increasing data flow, which is a potential source of failure for IoT networks. The biggest challenge of IoT networks is that the IoT may have insufficient memory to handle all transaction data within the IoT network. We aim in this paper to propose a potential compression method for reducing IoT network data traffic. Therefore, we investigate various lossless compression algorithms, such as entropy or dictionary-based algorithms, and general compression methods to determine which algorithm or method adheres to the IoT specifications. Furthermore, this study conducts compression experiments using entropy (Huffman, Adaptive Huffman) and Dictionary (LZ77, LZ78) as well as five different types of datasets of the IoT data traffic. Though the above algorithms can alleviate the IoT data traffic, adaptive Huffman gave the best compression algorithm. Therefore, in this paper, Citation: Nasif, A.; Othman, Z.A.; we aim to propose a conceptual compression method for IoT data traffic by improving an adaptive Sani, N.S. -
Answers to Exercises
Answers to Exercises A bird does not sing because he has an answer, he sings because he has a song. —Chinese Proverb Intro.1: abstemious, abstentious, adventitious, annelidous, arsenious, arterious, face- tious, sacrilegious. Intro.2: When a software house has a popular product they tend to come up with new versions. A user can update an old version to a new one, and the update usually comes as a compressed file on a floppy disk. Over time the updates get bigger and, at a certain point, an update may not fit on a single floppy. This is why good compression is important in the case of software updates. The time it takes to compress and decompress the update is unimportant since these operations are typically done just once. Recently, software makers have taken to providing updates over the Internet, but even in such cases it is important to have small files because of the download times involved. 1.1: (1) ask a question, (2) absolutely necessary, (3) advance warning, (4) boiling hot, (5) climb up, (6) close scrutiny, (7) exactly the same, (8) free gift, (9) hot water heater, (10) my personal opinion, (11) newborn baby, (12) postponed until later, (13) unexpected surprise, (14) unsolved mysteries. 1.2: A reasonable way to use them is to code the five most-common strings in the text. Because irreversible text compression is a special-purpose method, the user may know what strings are common in any particular text to be compressed. The user may specify five such strings to the encoder, and they should also be written at the start of the output stream, for the decoder’s use.