DOCSLIB.ORG

Information Theory and Coding” (1963) As a Note on Pages 61-62

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Information Theory and Coding” (1963) As a Note on Pages 61-62 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Lecture 5 Lossless Coding (II) May 20, 2009 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Outline Review Arithmetic Coding Dictionary Coding Run-Length Coding Lossless Image Coding Data Compression: Hall of Fame References 1 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Review Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Image and video encoding: A big picture Differential Coding Motion Estimation and Compensation A/D Conversion Context-Based Coding Color Space Conversion … Pre-Filtering Predictive Partitioning Coding … Input Image/Video Post- Pre- Lossy Lossless Processing Processing Coding Coding (Post-filtering) Quantization Entropy Coding Transform Coding Dictionary-Based Coding Model-Based Coding Run-Length Coding Encoded … … Image/Video 3 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding The ingredients of entropy coding A random source (X, P) A statistical model (X, P’) as an estimation of the random source An algorithm to optimize the coding performance (i.e., to minimize the average codeword length) At least one designer … 4 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding FLC, VLC and V2FLC FLC = Fixed-length coding/code(s)/codeword(s) • Each symbol xi emitted from a random source (X, P) is encoded as an n-bit codeword, where |X|≤2n. VLC = Variable-length coding/code(s)/codeword(s) • Each symbol xi emitted from a random source (X, P) is encoded as an ni-bit codeword. • FLC can be considered as a special case of VLC, where n1=…=n|X|. V2FLC = Variable-to-fixed length coding/code(s)/codeword(s) • A symbol or a string of symbols is encoded as an n-bit codeword. • V2FLC can also be considered as a special case of VLC. 5 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Static coding vs. Dynamic/Adaptive coding Static coding = The statistical model P’ is static, i.e., it does not change over time. Dynamic/Adaptive coding = The statistical model P’ is dynamically updated, i.e., it adapts itself to the context (i.e., changes over time). • Dynamic/Adaptive coding ⊂ Context-based coding Hybrid coding = Static + Dynamic coding • A codebook is maintained at the encoder side, and the encoder dynamically chooses a code for a number of symbols and inform the decoder about the choice. 6 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding A coding Shannon coding Shannon-Fano coding Huffman coding Arithmetic coding (Range coding) • Shannon-Fano-Elisa coding Universal coding • Exp-Golomb coding (H.264/MPEG-4 AVC, Dirac) • Elias coding family , Levenshtein coding , … Non-universal coding • Truncated binary coding, unary coding, … • Golomb coding ⊃ Rice coding Tunstall coding ⊂ V2FLC David Salomon, Variable-length Codes … for Data Compression, Springer, 2007 7 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Shannon-Fano Code: An example X={A,B,C,D,E}, P={0.35,0.2,0.19,0.13,0.13}, Y={0,1} A Possible Code 0.35+0.2 0.19+0.13+0.13 A, B C, D, E A 00 B 01 0.13+0.13 0.35 0.2 0.19 D, E C 10 A B C D 110 0.13 0.13 E 111 D E 8 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Universal coding (code) A code is called universal if L≤C1(H+C2) for all possible values of H, where C1, C2≥1. • You may see a different definition somewhere, but the basic idea remains the same – a universal code works like an optimal code, except there is a bound defined by a constant C1. A universal code is called asymptotically optimal if C1→1 when H→∞. 9 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Coding positive/non-negative integers Naive binary coding • |X|=2k: k-bit binary representation of an integer. Truncated binary coding • |X|=2k+b: X={0,1,…,2k+b-1} 0 0 0 k k k 2 b 1 b0 bk 1 2 b b0 bk 2 + b 1 1 1 ⇒ ··· − − ⇒ //////////////////////··· − − ⇒ ··· − ⇒ ··· k k+1 | {zUnary} code (Stone-age binary coding) | {z } • |X|=∞: X=Z+={1,2,…} f(x)=0 0 1or1 1 0 ··· ··· x 1 x 1 − − | {z } | {z } 10 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Golomb coding and rice coding Golomb coding = Unary coding + Truncated binary coding • An integer x is divided into two parts (quotient and remainder) according to a parameter M: q = x/M , b c r=mod(x, M)=x-q*M. • Golumb code = unary code of q + truncated binary code of r. • When M=1, Golomb coding = unary coding. • When M=2k, Golomb coding = Rice coding. • Golomb code is the optimal code for the geometric distribution: Prob(x=i)=(1-p)i-1p, where 0<p<1. 11 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Exp-Golomb coding (Universal) Exp-Golomb coding ≠ Golomb coding Exp-Golomb coding of order k=0 is used in some video coding standards such as H.264. The encoding process • |X|=∞: X={0,1,2,…} k • Calculate q = x/ 2 +1 , and n q = log q . b c b 2 c • Exp-Golomb code = unary code of nq + nq LSBs of q + k-bit representation of r=mod(x,2k)=x-q*2k. 12 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Huffman code: An example X={1,2,3,4,5}, P=[0.4,0.2,0.2,0.1,0.1]. p1+2+3+4+5=1 A Possible Code p =0.6 1 00 1+2 p3+4+5=0.4 2 01 3 10 p1=0.4 p2=0.2 p4+5=0.2 p3=0.2 4 110 5 111 p4=0.1 p5=0.1 13 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Huffman code: An optimal code Relation between Huffman code and Shannon code: H≤LHuffman≤LHuffman-Fano≤LShannon<H+1 A stronger result (Gallager, 1978) • When pmax≥ 0.5, LHuffman≤H+pmax<H+1 • When pmax<0.5, LHuffman≤H+pmax+log2(2(log2e)/e)≈H+pmax+0.086<H+0.586 Huffman’s rules of optimal codes imply that Huffman code is optimal. • When each pi is a negative power of 2, Huffman code reaches the entropy. 14 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Huffman code: Small X problem Problem • When |X| is small, the coding performance is less obvious. • As a special case, when |X|=2, Huffman coding cannot compress the data at all – no matter what the probability is, each symbol has to be encoded as a single bit. Solutions • Solution 1: Work on Xn rather than X. • Solution 2: Dual tree coding = Huffman coding + Tunstall coding 15 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Huffman code: Variance problem Problem • There are multiple choices of two smallest probabilities, if more than two nodes have the same probability during any step of the coding process. • Huffman codes with a larger variance may cause trouble for data transmissions via a CBR (constant bit rate) channel – a larger buffer is needed. Solution • Shorter subtrees first. (A single node’s height is 0.) 16 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Modified Huffman code Problem • If |X| is too large, the construction of the Huffman tree will be too long and the memory used for the tree will be too demanding. Solution -v -v • Divide X into two set X1={si|p(si)>2 }, X2={si|p(si)≤2 }. • Perform Huffman coding for the new set X3=X1∪{X2}. • Append f(X2) as the prefix of naive binary representation of all symbols in X2. 17 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Huffman’s rules of making optimal codes Source statistics: P=P0=[p1,…,pm], where p1≥…≥pm-1≥pm. Rule 1: L1≤…≤Lm-1=Lm. Rule 2: If L1≤…≤Lm-2<Lm-1=Lm, Lm-1 and Lm differs from each other only for the last bit, i.e., f(xm-1)=b0 and f(xm)=b1, where b is a sequence of Lm-1 bits. Rule 3: Each possible bit sequence of length Lm-1 must be either a codeword or the prefix of some codewords. Answers: Read Section 5.2.1 (pp. 122-123) of the following book – Yun Q. Shi and Huifang Sun, Image and Video Compression for Multimedia Engineering, 2nd Edition, CRC Press, 2008 18 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Justify Huffman’s rules Rule 1 • If Li>Li+1, we can swap them to get a smaller average codeword length (when Pi=Pi+1 it does not make sense, though). • If Lm-1<Lm, then L1,…,Lm-1<Lm and the last bit of Lm is redundant. Rule 2 • If they do not have the same parent node, then the last bits of both codewords are redundant. Rule 3 • If there is an unused bit sequence of length Lm-1, we can use it for Lm. 19 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Arithmetic Coding Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Why do we need a new coding algorithm? Problems with Huffman coding • Each symbol in X is represented by at least one bit. • Coding for Xn: A Huffman tree with |X|n nodes needs to be constructed. The value of n cannot be too large. • Encoding with a Huffman tree is quite easy, but decoding can be difficult especially when X is large. • Dynamic Huffman coding is slow due to the update of the Huffman tree from time to time. Solution: Arithmetic coding • Encode a number of symbols (can be a message of a very large size n) incrementally (progressively) as a binary fraction in a real range [low, high) ⊆ [0,1).
Load more

Recommended publications
  • Source Coding: Part I of Fundamentals of Source and Video Coding
    Foundations and Trends R in sample Vol. 1, No 1 (2011) 1–217 c 2011 Thomas Wiegand and Heiko Schwarz DOI: xxxxxx Source Coding: Part I of Fundamentals of Source and Video Coding Thomas Wiegand1 and Heiko Schwarz2 1 Berlin Institute of Technology and Fraunhofer Institute for Telecommunica- tions — Heinrich Hertz Institute, Germany, [email protected] 2 Fraunhofer Institute for Telecommunications — Heinrich Hertz Institute, Germany, [email protected] Abstract Digital media technologies have become an integral part of the way we create, communicate, and consume information. At the core of these technologies are source coding methods that are described in this text. Based on the fundamentals of information and rate distortion theory, the most relevant techniques used in source coding algorithms are de- scribed: entropy coding, quantization as well as predictive and trans- form coding. The emphasis is put onto algorithms that are also used in video coding, which will be described in the other text of this two-part monograph. To our families Contents 1 Introduction 1 1.1 The Communication Problem 3 1.2 Scope and Overview of the Text 4 1.3 The Source Coding Principle 5 2 Random Processes 7 2.1 Probability 8 2.2 Random Variables 9 2.2.1 Continuous Random Variables 10 2.2.2 Discrete Random Variables 11 2.2.3 Expectation 13 2.3 Random Processes 14 2.3.1 Markov Processes 16 2.3.2 Gaussian Processes 18 2.3.3 Gauss-Markov Processes 18 2.4 Summary of Random Processes 19 i ii Contents 3 Lossless Source Coding 20 3.1 Classification
    [Show full text]
  • Data Compression for Remote Laboratories
    Paper—Data Compression for Remote Laboratories Data Compression for Remote Laboratories https://doi.org/10.3991/ijim.v11i4.6743 Olawale B. Akinwale Obafemi Awolowo University, Ile-Ife, Nigeria [email protected] Lawrence O. Kehinde Obafemi Awolowo University, Ile-Ife, Nigeria [email protected] Abstract—Remote laboratories on mobile phones have been around for a few years now. This has greatly improved accessibility of these remote labs to students who cannot afford computers but have mobile phones. When money is a factor however (as is often the case with those who can’t afford a computer), the cost of use of these remote laboratories should be minimized. This work ad- dressed this issue of minimizing the cost of use of the remote lab by making use of data compression for data sent between the user and remote lab. Keywords—Data compression; Lempel-Ziv-Welch; remote laboratory; Web- Sockets 1 Introduction The mobile phone is now the de facto computational device of choice. They are quite powerful, connected devices which are almost always with us. This is so much so that about every important online service has a mobile version or at least a version compatible with mobile phones and tablets (for this paper we would adopt the phrase mobile device to represent mobile phones and tablets). This has caused online labora- tory developers to develop online laboratory solutions which are mobile device- friendly [1, 2, 3, 4, 5, 6, 7, 8]. One of the main benefits of online laboratories is the possibility of using fewer re- sources to serve more people i.e.
    [Show full text]
  • Error Correction Capacity of Unary Coding
    Error Correction Capacity of Unary Coding Pushpa Sree Potluri1 Abstract Unary coding has found applications in data compression, neural network training, and in explaining the production mechanism of birdsong. Unary coding is redundant; therefore it should have inherent error correction capacity. An expression for the error correction capability of unary coding for the correction of single errors has been derived in this paper. 1. Introduction The unary number system is the base-1 system. It is the simplest number system to represent natural numbers. The unary code of a number n is represented by n ones followed by a zero or by n zero bits followed by 1 bit [1]. Unary codes have found applications in data compression [2],[3], neural network training [4]-[11], and biology in the study of avian birdsong production [12]-14]. One can also claim that the additivity of physics is somewhat like the tallying of unary coding [15],[16]. Unary coding has also been seen as the precursor to the development of number systems [17]. Some representations of unary number system use n-1 ones followed by a zero or with the corresponding number of zeroes followed by a one. Here we use the mapping of the left column of Table 1. Table 1. An example of the unary code N Unary code Alternative code 0 0 0 1 10 01 2 110 001 3 1110 0001 4 11110 00001 5 111110 000001 6 1111110 0000001 7 11111110 00000001 8 111111110 000000001 9 1111111110 0000000001 10 11111111110 00000000001 The unary number system may also be seen as a space coding of numerical information where the location determines the value of the number.
    [Show full text]
  • Nxadmin CLI Reference Guide Unity Iv Contents
    HYPER-UNIFIED STORAGE nxadmin Command Line Interface Reference Guide NEXSAN | 325 E. Hillcrest Drive, Suite #150 | Thousand Oaks, CA 91360 USA Printed Thursday, July 26, 2018 | www.nexsan.com Copyright © 2010—2018 Nexsan Technologies, Inc. All rights reserved. Trademarks Nexsan® is a trademark or registered trademark of Nexsan Technologies, Inc. The Nexsan logo is a registered trademark of Nexsan Technologies, Inc. All other trademarks and registered trademarks are the property of their respective owners. Patents This product is protected by one or more of the following patents, and other pending patent applications worldwide: United States patents US8,191,841, US8,120,922; United Kingdom patents GB2466535B, GB2467622B, GB2467404B, GB2296798B, GB2297636B About this document Unauthorized use, duplication, or modification of this document in whole or in part without the written consent of Nexsan Corporation is strictly prohibited. Nexsan Technologies, Inc. reserves the right to make changes to this manual, as well as the equipment and software described in this manual, at any time without notice. This manual may contain links to web sites that were current at the time of publication, but have since been moved or become inactive. It may also contain links to sites owned and operated by third parties. Nexsan is not responsible for the content of any such third-party site. Contents Contents Contents iii Chapter 1: Accessing the nxadmin and nxcmd CLIs 15 Connecting to the Unity Storage System using SSH 15 Prerequisite 15 Connecting to the Unity
    [Show full text]
  • Image Data Compression Introduction to Coding
    Image Data Compression Introduction to Coding © 2018-19 Alexey Pak, Lehrstuhl für Interaktive Echtzeitsysteme, Fakultät für Informatik, KIT 1 Review: data reduction steps (discretization / digitization) Continuous 2D siGnal Fully diGital siGnal (liGht intensity on sensor) gq (xa, yb,ti ) Discrete time siGnal (pixel voltaGe readinGs) g(xa, yb,ti ) g(x, y,t) Spatial discretization Temporal discretization and diGitization g(xa, yb,t) g(xa, yb,t) gq (xa, yb,t) Discrete value siGnal AnaloG siGnal Spatially discrete siGnal (e.G., # of electrons at each (liGht intensity at a pixel) (pixel-averaGed intensity) pixel of the CCD matrix) © 2018-19 Alexey Pak, Lehrstuhl für Interaktive Echtzeitsysteme, Fakultät für Informatik, KIT 2 Review: data reduction steps (discretization / digitization) Discretization of 1D continuous-time signals (sampling) • Important signal transformations: up- and down-sampling • Information-preserving down-sampling: rate determined based on signal bandwidth • Fourier space allows simple interpretation of the effects due to decimation and interpolation (techniques of up-/down-sampling) Scalar (one-dimensional) signal quantization of continuous-value signals • Quantizer types: uniform, simple non-uniform (with a dead zone, with a limited amplitude) • Advanced quantizers: PDF-optimized (Max-Lloyd algorithm), perception-optimized, SNR- optimized • Implementation: pre-processing with a compander function + simple quantization Vector (multi-dimensional) signal quantization • Terminology: quantization, reconstruction, codebook, distance metric, Voronoi regions, space partitioning • Relation to the general classification problem (from Machine Learning) • Linde-Buzo-Gray algorithm of constructing (sub-optimal) codebooks (aka k-means) © 2018-19 Alexey Pak, Lehrstuhl für Interaktive Echtzeitsysteme, Fakultät für Informatik, KIT 3 LGB vector quantization – 2D example [Linde, Buzo, Gray ‘80]: 1.
    [Show full text]
  • Soft Compression for Lossless Image Coding
    1 Soft Compression for Lossless Image Coding Gangtao Xin, and Pingyi Fan, Senior Member, IEEE Abstract—Soft compression is a lossless image compression method, which is committed to eliminating coding redundancy and spatial redundancy at the same time by adopting locations and shapes of codebook to encode an image from the perspective of information theory and statistical distribution. In this paper, we propose a new concept, compressible indicator function with regard to image, which gives a threshold about the average number of bits required to represent a location and can be used for revealing the performance of soft compression. We investigate and analyze soft compression for binary image, gray image and multi-component image by using specific algorithms and compressible indicator value. It is expected that the bandwidth and storage space needed when transmitting and storing the same kind of images can be greatly reduced by applying soft compression. Index Terms—Lossless image compression, information theory, statistical distributions, compressible indicator function, image set compression. F 1 INTRODUCTION HE purpose of image compression is to reduce the where H(X1;X2; :::; Xn) is the joint entropy of the symbol T number of bits required to represent an image as much series fXi; i = 1; 2; :::; ng. as possible under the condition that the fidelity of the It’s impossible to reach the entropy rate and everything reconstructed image to the original image is higher than a we can do is to make great efforts to get close to it. Soft com- required value. Image compression often includes two pro- pression [1] was recently proposed, which uses locations cesses, encoding and decoding.
    [Show full text]
  • 16.1 Digital “Modes”
    Contents 16.1 Digital “Modes” 16.5 Networking Modes 16.1.1 Symbols, Baud, Bits and Bandwidth 16.5.1 OSI Networking Model 16.1.2 Error Detection and Correction 16.5.2 Connected and Connectionless 16.1.3 Data Representations Protocols 16.1.4 Compression Techniques 16.5.3 The Terminal Node Controller (TNC) 16.1.5 Compression vs. Encryption 16.5.4 PACTOR-I 16.2 Unstructured Digital Modes 16.5.5 PACTOR-II 16.2.1 Radioteletype (RTTY) 16.5.6 PACTOR-III 16.2.2 PSK31 16.5.7 G-TOR 16.2.3 MFSK16 16.5.8 CLOVER-II 16.2.4 DominoEX 16.5.9 CLOVER-2000 16.2.5 THROB 16.5.10 WINMOR 16.2.6 MT63 16.5.11 Packet Radio 16.2.7 Olivia 16.5.12 APRS 16.3 Fuzzy Modes 16.5.13 Winlink 2000 16.3.1 Facsimile (fax) 16.5.14 D-STAR 16.3.2 Slow-Scan TV (SSTV) 16.5.15 P25 16.3.3 Hellschreiber, Feld-Hell or Hell 16.6 Digital Mode Table 16.4 Structured Digital Modes 16.7 Glossary 16.4.1 FSK441 16.8 References and Bibliography 16.4.2 JT6M 16.4.3 JT65 16.4.4 WSPR 16.4.5 HF Digital Voice 16.4.6 ALE Chapter 16 — CD-ROM Content Supplemental Files • Table of digital mode characteristics (section 16.6) • ASCII and ITA2 code tables • Varicode tables for PSK31, MFSK16 and DominoEX • Tips for using FreeDV HF digital voice software by Mel Whitten, KØPFX Chapter 16 Digital Modes There is a broad array of digital modes to service various needs with more coming.
    [Show full text]
  • Shannon's Coding Theorems
    Saint Mary's College of California Department of Mathematics Senior Thesis Shannon's Coding Theorems Faculty Advisors: Author: Professor Michael Nathanson Camille Santos Professor Andrew Conner Professor Kathy Porter May 16, 2016 1 Introduction A common problem in communications is how to send information reliably over a noisy communication channel. With his groundbreaking paper titled A Mathematical Theory of Communication, published in 1948, Claude Elwood Shannon asked this question and provided all the answers as well. Shannon realized that at the heart of all forms of communication, e.g. radio, television, etc., the one thing they all have in common is information. Rather than amplifying the information, as was being done in telephone lines at that time, information could be converted into sequences of 1s and 0s, and then sent through a communication channel with minimal error. Furthermore, Shannon established fundamental limits on what is possible or could be acheived by a communication system. Thus, this paper led to the creation of a new school of thought called Information Theory. 2 Communication Systems In general, a communication system is a collection of processes that sends information from one place to another. Similarly, a storage system is a system that is used for storage and later retrieval of information. Thus, in a sense, a storage system may also be thought of as a communication system that sends information from one place (now, or the present) to another (then, or the future) [3]. In a communication system, information always begins at a source (e.g. a book, music, or video) and is then sent and processed by an encoder to a format that is suitable for transmission through a physical communications medium, called a channel.
    [Show full text]
  • Efficient Variable-To-Fixed Length Coding Algorithms for Text
    Efficient Variable-to-Fixed Length Coding Algorithms for Text Compression （テキスト圧縮に対する効率よい可変長-固定長符号化アルゴリズム） Satoshi Yoshida February, 2014 Abstract Data compression is a technique for reducing the storage space and the cost of trans- ferring a large amount of data, using redundancy hidden in the data. We focus on lossless compression for text data, that is, text compression, in this thesis. To reuse a huge amount of data stored in secondary storage, I/O speeds are bottlenecks. Such a communication-speed problem can be relieved if we transfer only compressed data through the communication channel and furthermore can perform every necessary pro- cesses, such as string search, on the compressed data itself without decompression. Therefore, a new criterion \ease of processing the compressed data" is required in the field of data compression. Development of compression algorithms is currently in the mainstream of data compression field but many of them are not adequate for that criterion. The algorithms employing variable length codewords succeeded to achieve an extremely good compression ratio, but the boundaries between codewords are not obvious without a special processing. Such an \unclear boundary problem" prevents us from direct accessing to the compressed data. On the contrary, Variable-to-Fixed-length coding, which is referred to as VF coding, is promising for our demand. VF coding is a coding scheme that segments an input text into a consecutive sequence of substrings (called phrases) and then assigns a fixed length codeword to each substring. Boundaries between codewords of VF coding are obvious because all of them have the same length. Therefore, we can realize \accessible data compression" by VF coding.
    [Show full text]
  • Generalized Golomb Codes and Adaptive Coding of Wavelet-Transformed Image Subbands
    IPN Progress Report 42-154 August 15, 2003 Generalized Golomb Codes and Adaptive Coding of Wavelet-Transformed Image Subbands A. Kiely1 and M. Klimesh1 We describe a class of prefix-free codes for the nonnegative integers. We apply a family of codes in this class to the problem of runlength coding, specifically as part of an adaptive algorithm for compressing quantized subbands of wavelet- transformed images. On test images, our adaptive coding algorithm is shown to give compression effectiveness comparable to the best performance achievable by an alternate algorithm that has been previously implemented. I. Generalized Golomb Codes A. Introduction Suppose we wish to use a variable-length binary code to encode integers that can take on any non- negative value. This problem arises, for example, in runlength coding—encoding the lengths of runs of a dominant output symbol from a discrete source. Encoding cannot be accomplished by simply looking up codewords from a table because the table would be infinite, and Huffman’s algorithm could not be used to construct the code anyway. Thus, one would like to use a simply described code that can be easily encoded and decoded. Although in practice there is generally a limit to the size of the integers that need to be encoded, a simple code for the nonnegative integers is often still the best choice. We now describe a class of variable-length binary codes for the nonnegative integers that contains several useful families of codes. Each code in this class is completely specified by an index function f from the nonnegative integers onto the nonnegative integers.
    [Show full text]
  • Smaller Flight Data Recorders{
    Available online at http://docs.lib.purdue.edu/jate Journal of Aviation Technology and Engineering 2:2 (2013) 45–55 Smaller Flight Data Recorders{ Yair Wiseman and Alon Barkai Bar-Ilan University Abstract Data captured by flight data recorders are generally stored on the system’s embedded hard disk. A common problem is the lack of storage space on the disk. This lack of space for data storage leads to either a constant effort to reduce the space used by data, or to increased costs due to acquisition of additional space, which is not always possible. File compression can solve the problem, but carries with it the potential drawback of the increased overhead required when writing the data to the disk, putting an excessive load on the system and degrading system performance. The author suggests the use of an efficient compressed file system that both compresses data in real time and ensures that there will be minimal impact on the performance of other tasks. Keywords: flight data recorder, data compression, file system Introduction A flight data recorder is a small line-replaceable computer unit employed in aircraft. Its function is recording pilots’ inputs, electronic inputs, sensor positions and instructions sent to any electronic systems on the aircraft. It is unofficially referred to as a "black box". Flight data recorders are designed to be small and thoroughly fabricated to withstand the influence of high speed impacts and extreme temperatures. A flight data recorder from a commercial aircraft can be seen in Figure 1. State-of-the-art high density flash memory devices have permitted the solid state flight data recorder (SSFDR) to be implemented with much larger memory capacity.
    [Show full text]
  • Information Theory and Coding
    Information Theory and Coding Introduction – What is it all about? References: C.E.Shannon, “A Mathematical Theory of Communication”, The Bell System Technical Journal, Vol. 27, pp. 379–423, 623–656, July, October, 1948. C.E.Shannon “Communication in the presence of Noise”, Proc. Inst of Radio Engineers, Vol 37 No 1, pp 10 – 21, January 1949. James L Massey, “Information Theory: The Copernican System of Communications”, IEEE Communications Magazine, Vol. 22 No 12, pp 26 – 28, December 1984 Mischa Schwartz, “Information Transmission, Modulation and Noise”, McGraw Hill 1990, Chapter 1 The purpose of telecommunications is the efficient and reliable transmission of real data – text, speech, image, taste, smell etc over a real transmission channel – cable, fibre, wireless, satellite. In achieving this there are two main issues. The first is the source – the information. How do we code the real life usually analogue information, into digital symbols in a way to convey and reproduce precisely without loss the transmitted text, picture etc.? This entails proper analysis of the source to work out the best reliable but minimum amount of information symbols to reduce the source data rate. The second is the channel, and how to pass through it using appropriate information symbols the source. In doing this there is the problem of noise in all the multifarious ways that it appears – random, shot, burst- and the problem of one symbol interfering with the previous or next symbol especially as the symbol rate increases. At the receiver the symbols have to be guessed, based on classical statistical communications theory. The guessing, unfortunately, is heavily influenced by the channel noise, especially at high rates.
    [Show full text]