DOCSLIB.ORG

Double Compression of Test Data Using Huffman Code

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Double Compression of Test Data Using Huffman Code Journal of Theoretical and Applied Information Technology 15 May 2012. Vol. 39 No.2 © 2005 - 2012 JATIT & LLS. All rights reserved . ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195 DOUBLE COMPRESSION OF TEST DATA USING HUFFMAN CODE 1R.S.AARTHI, 2D. MURALIDHARAN, 3P. SWAMINATHAN 1 School of Computing, SASTRA University, Thanjavur, Tamil Nadu, India 2Assistant professor, School of Computing, SASTRA University, Thanjavur 3Dean, School of Computing, SASTRA University, Thanjavur E-Mail: [email protected] , [email protected] , [email protected] ABSTRACT Increase in design complexity and fabrication technology results in high test data volume. As test size increases memory capacity also increases, which becomes the major difficulty in testing System-on-Chip (SoC). To reduce the test data volume, several compression techniques have been proposed. Code based schemes is one among those compression techniques. Run length coding is one of the most popular coding methodology in code based compression. Run length codes like Golomb code, Frequency directed run Length Code (FDR code), Extended FDR, Modified FDR, Shifted Alternate FDR and OLEL coding compress the test data and the compression ratio increases drastically. For further reduction of test data, double compression technique is proposed using Huffman code. Compression ratio using Double compression technique is presented and compared with the compression ratio obtained by other Run length codes. Keywords:– Test Data Compression, Run Length Codes, Golomb Code, OLEL Code, Huffman Code 1. INTRODUCTION be decreased. One way to achieve it is to compress the test data. Compression is possible for data that are redundant or repeated in a given test set. Test time ≥ Compression method is broadly divided into lossless and lossy methods, lossless method which can reconstruct the original data exactly from the Test data compression tells about, compressing compressed data and lossy method which can only the test data to reduce the test volume and reconstruct an approximation of the original data. increasing the compression ratio. Test data Lossless methods are commonly used for text and compression is divided into three categories: 1. lossy methods used for images and sound where a Code-based schemes, 2. Linear-decompression- little bit of loss in resolution is often undetectable based schemes, 3. Broadcast-scan-based schemes. or at least acceptable. Code based schemes use data compression codes to encode the test data. This performs partitioning the Test data volume is a major problem encountered original data into groups, and each group is in the testing for System-on-Chip (SoC) design. replaced with a codeword to produce the The volume of test data for a SoC is increasing compressed data. Linear - Decompression - based rapidly. The clock rate of the channel, number of schemes decompress the data using only linear channels, amount of test data of the channel are operations with wires, LFSR and XOR networks. some of the important parameters for the Automatic Broadcast - scan - based schemes are based on the Test Equipment (ATE) to function. Since all these idea of broadcasting the same value for multiple parameters are large in number, the test time is also scan chains, i.e., a single channel drives multiple high and low on the other case. Therefore, to scan chains [1]. mitigate the test time, the amount of test data must 104 Journal of Theoretical and Applied Information Technology 15 May 2012. Vol. 39 No.2 © 2005 - 2012 JATIT & LLS. All rights reserved . ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195 Among the code-based techniques, run length 4. SHANON-FANO CODING coding is an important methodology. Run length coding is one of the lossless type data compressions The procedure is done by a more frequently like Huffman and Arithmetic coding technique that occurring string which is encoded by a shorter is based on a statistical model (occurrence encoding vector and a less frequently occurring probabilities). Universal coding consists of string is encoded by a longer encoding vector. Fibonacci coding, Elias coding, Levenstein coding. Shannon-Fano coding relied on the occurrence of Elias coding has Elias Delta, Elias Gamma, and each character or symbol with their frequencies in a Elias Omega coding. Dictionary based compression list and is also called as a variable length coding. also involves LZW coding. More number of According to their frequency, the list of symbols is efficient compression techniques have been sorted, by the most frequently occurring symbols at proposed, such as Golomb code [16] [17] [18] the left side and the least most is takes place at the Variable-length Input code codes (VIHC) 15 , right. Divide the symbols into two sets, with the Frequency Directed run length code (FDR) [19] total frequency of the left set being as close to the [20], Extended FDR [21], Modified FDR [22], total of the right set as possible and assign 0 in the Shifted Alternate FDR [23] [24], which will reduce left set of the list whereas assign 1 in right set of the the total number of test data and also increases the list . Repeatedly divide the sets and assigning 0 and compression ratio. 1 to the left and right set of the lists until each symbol has a unique coding [5]. 2. RUN LENGTH BASED CODES 5. DICTIONARY BASED COMPRESSION Run length code is a very simple form of data compression technique, in which sequences or runs Lempel-Ziv-Welch (LZW) is a category of a of consecutive repeated data values are replaced by dictionary–based compression method [6] [7] [8]. It a single data value. There are several run length maps a variable number of symbols to a fixed based codes like Golomb code, Frequency Directed length code. LZW places longer and longer run length code (FDR), Extended FDR, Modified repeated entries into a dictionary. It emits the code FDR, and Shifted Alternate FDR. Other than this, for an element, rather than the string itself, if the we have an OLEL coding methodology. From these element has already been placed in the dictionary. techniques, better compression ratio is achieved. To find out the compression ratio the equation is In a dictionary-based data compression formulated as, technique, a symbol or a string of symbols generated from a source alphabet is represented by an index to a dictionary constructed from the source %compression= alphabet. In a dictionary based coding to build a dictionary that has frequently occurring symbols and string of symbols. When a new symbol or 3. HUFFMAN CODING string is found and that is contained in the dictionary, it is encoded with an index to the Huffman code is mapped to the fixed length dictionary. Or else, the new symbol is not in the symbols to variable length codes. Huffman dictionary, the symbol or strings of symbols are algorithm begins, based on the list of all the encoded in a less efficient manner. symbols or data which are arranged in descending order of probabilities. Next it generates a binary 6. ARITHMETIC CODING tree, by the bottom-up approach with a symbol at every leaf. This has some steps, in each step two One of the powerful techniques is called arithmetic symbols with the smallest frequencies are chosen, coding [9]. This converts the entire input data into a added to the top of the partial tree. The selected single floating point number. Between source smallest frequencies symbols are deleted from the symbols and code word, there is no one-to-one list, which are replaced by a secondary symbol correspondence. Here a single arithmetic code word denoting the two original symbols. Therefore, the is assigned for an entire sequence of symbols. The list is reduced to one secondary symbol, which interval of real numbers between 0 and 1 is defined denotes the tree is complete. Finally, assign a by the codeword. The interval denoting it become codeword for each leaf or symbol based on the path smaller and the number of bits which are required from the root node to symbols in the list [3] [4]. to represent the interval become larger. According 105 Journal of Theoretical and Applied Information Technology 15 May 2012. Vol. 39 No.2 © 2005 - 2012 JATIT & LLS. All rights reserved . ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195 to the probability of occurrence, each bit symbol repeat the second step, and then subtract one from reduces the interval size. the number of previously calculated digits, after the new encoded number is found. 7. UNIVERSAL CODING Fibonacci code is a one type of universal code Elias Gamma code, Elias Delta code, Elias that converts positive integer numbers into binary Omega code [11] [12] the positive integers are code words. Each binary code word ends with 11 encoded by the universal code, developed by Peter and it doesn’t have other instances of “11” before Elias. Levenstein code is also a universal code that the end of the binary codeword [13] [14]. encodes the non negative integers by Vladimir Levenshtein. An advantage of universal code is 8. GOLOMB CODE [16] behalf of Huffman codes. It is not mandatory to find the probabilities of the source string exactly. Golomb code was invented by Solomon W. Golomb in 1960. Golomb code is variable-variable 7.1. Elias Gamma coding: length code. The main advantage of Golomb code is the very high compression ratio. Encoding In Elias Gamma coding, binary values are always procedure has two steps, in first step a test set TD is expressed as x, and it will to produce its difference vector test set Tdiff . So the begin with 1, in this, few bits as possible. From pre-computed test set is TD = {t 1, t 2, t 3.
Load more

Recommended publications
  • Pixel-Based Video Coding
    UPTEC IT 14 003 Examensarbete 30 hp Mars 2014 Pixel-based video coding Johannes Olsson Sandgren Abstract Pixel-based video coding Johannes Olsson Sandgren Teknisk- naturvetenskaplig fakultet UTH-enheten This paper studies the possibilities of extending the pixel-based compression algorithm LOCO-I, used by the lossless and near lossless image compression standard Besöksadress: JPEG-LS, introduced by the Joint Photographic Experts Group (JPEG) in 1999, to Ångströmlaboratoriet Lägerhyddsvägen 1 video sequences and very low bit-rates. Bitrates below 1 bit per pixel are achieved Hus 4, Plan 0 through skipping signaling when the prediction of a pixels sufficiently good. The pixels to be skipped are implicitly detected by the decoder, minimizing the overhead. Postadress: Different methods of quantization are tested, and the possibility of using vector Box 536 751 21 Uppsala quantization is investigated, by matching pixel sequences against a dynamically generated vector tree. Several different prediction schemes are evaluated, both linear Telefon: and non-linear, with both static and adaptive weights. Maintaining the low 018 – 471 30 03 computational complexity of LOCO-I has been a priority. The results are compared Telefax: to different HEVC implementations with regards to compression speed and ratio. 018 – 471 30 00 Hemsida: http://www.teknat.uu.se/student Handledare: Jonatan Samuelsson Ämnesgranskare: Cris Luengo Examinator: Lars-Åke Nordén ISSN: 1401-5749, UPTEC IT 14 003 Tryckt av: Reprocentralen ITC Sammanfattning på svenska De dominerande videokomprimeringsstandarderna idag, som H.26x, är blockbaserade, och har förhållandevis hög beräkningsmässig komplexitet, framförallt i kodningsfasen. I följande text utforskas möjligheten att utöka en välkänd algoritm, LOCO-I, för pixel- baserad komprimering så att komprimering lägre än 1 bit per pixel blir möjlig.
    [Show full text]
  • Vosoughi, Mbw2, Cavallar}@Rice.Edu
    Baseband Signal Compression in Wireless Base Stations Aida Vosoughi, Michael Wu, and Joseph R. Cavallaro {vosoughi, mbw2, cavallar}@rice.edu Unused Significant USBR code Motivation 010, 5(=101) Experimental Setup and Results Bit Removal binary code 01011001 11001 Using WARP hardware and WARPLab framework for over-the- With new wireless standards, base stations require 01010000 10000 Used before in air experiments: (a) Downlink, (b) Uplink greater data rates over the serial data link between base 01000111 00111 Network on chip serial station processor and (remote) RF unit. 01001100 01100 link compression 10011100 1001, 4(=100) This link is especially important in: 10010011 1100 10011010 0011 • Distributed Antenna Systems 10010001 1010 • Cooperating base stations …. 0001 • Cloud-based Radio Access Networks … We propose to use a differencing step before compressing with Baseband signals: each of the above lossless schemes. • Usually oversampled and correlated Compression Ratio Results: • Represented with extra bit resolution Proposed Lossy Compression Lossless Schemes: Lossy Scheme: LSB Removal/Quantization: Algorithm TX RX DQPSK 16-QAM Compression can decrease hardware complexity and cost. The number of LSBs that can [1] 1.51 1.07 TX 4 2.3 be removed without causing USBR 1.41 1.17 RX 3.5 2 distortion depends on the Elias-gamma + 1.40 0.84 Introduction and Related Work Adaptive AC modulation type. Adaptive AC 1.26 0.95 Base station architecture with compression/decompression: Elias-gamma 1.22 0.72 With higher size constellations, more bit
    [Show full text]
  • Compressing Integers for Fast File Access
    Compressing Integers for Fast File Access Hugh E. Williams and Justin Zobel Department of Computer Science, RMIT University GPO Box 2476V, Melbourne 3001, Australia Email: hugh,jz @cs.rmit.edu.au { } Fast access to files of integers is crucial for the efficient resolution of queries to databases. Integers are the basis of indexes used to resolve queries, for exam- ple, in large internet search systems and numeric data forms a large part of most databases. Disk access costs can be reduced by compression, if the cost of retriev- ing a compressed representation from disk and the CPU cost of decoding such a representation is less than that of retrieving uncompressed data. In this paper we show experimentally that, for large or small collections, storing integers in a com- pressed format reduces the time required for either sequential stream access or random access. We compare different approaches to compressing integers, includ- ing the Elias gamma and delta codes, Golomb coding, and a variable-byte integer scheme. As a conclusion, we recommend that, for fast access to integers, files be stored compressed. Keywords: Integer compression, variable-bit coding, variable-byte coding, fast file access, scientific and numeric databases. 1. INTRODUCTION codes [2], and parameterised Golomb codes [3]; second, we evaluate byte-wise storage using standard four-byte Many data processing applications depend on efficient integers and a variable-byte scheme. access to files of integers. For example, integer data is We have found in our experiments that coding us- prevalent in scientific and financial databases, and the ing a tailored integer compression scheme can allow indexes to databases can consist of large sequences of retrieval to be up to twice as fast than with integers integer record identifiers.
    [Show full text]
  • Lampiran a Listing Program
    LAMPIRAN A LISTING PROGRAM Private Sub Command3_Click() If UBound(OriginalArray) = 0 Then Exit Sub End MsgBox "There is nothing to End If compress/Decompress" End Sub Else Exit Sub LastCoder = WichCompressor End If Private Sub compress_Click() End If If AutoDecodeIsOn = False Then m1.Visible = True Call Copy_Orig2Work decompress = True m2.Visible = True LastDeCoded = decompress If CompDecomp = 0 Then Exit Sub End Sub If JustLoaded = True Then If CompDecomp = 1 Then decompress = False JustLoaded = False Else Private Sub d1_Click() ReDim UsedCodecs(0) decompress = True CompDecomp = 2 End If End If Dim wktmulai As Single compression.MousePointer = If decompress = True Then MousePointerConstants.vbHourglass Dim decompress As Boolean LastUsed = wktmulai = Timer Dim Dummy As Boolean UsedCodecs(UBound(UsedCodecs)) Call DeCompress_arithmetic_Dynamic(hasil) Dim StTime As Double If JustLoaded = True Then LastUsed = 0 Label12.Caption = Timer - wktmulai & " detik" Dim Text As String If WichCompressor <> LastUsed Then compression.MousePointer = Dim LastUsed As Integer Text = "This is not compressed with ." & MousePointerConstants.vbDefault Chr(13) Call Show_Statistics(False, hasil) MsgBox Text A-1 Call afterContent(hasil) If AutoDecodeIsOn = False Then End If compression.save.Visible = True decompress = True Call Copy_Orig2Work d1.Visible = False If CompDecomp = 0 Then Exit Sub LastDeCoded = decompress d2.Visible = False If CompDecomp = 1 Then decompress = False If JustLoaded = True Then End Sub Else JustLoaded = False decompress = True ReDim UsedCodecs(0)
    [Show full text]
  • Entropy Coding and Different Coding Techniques
    Journal of Network Communications and Emerging Technologies (JNCET) www.jncet.org Volume 6, Issue 5, May (2016) Entropy Coding and Different Coding Techniques Sandeep Kaur Asst. Prof. in CSE Dept, CGC Landran, Mohali Sukhjeet Singh Asst. Prof. in CS Dept, TSGGSK College, Amritsar. Abstract – In today’s digital world information exchange is been 2. CODING TECHNIQUES held electronically. So there arises a need for secure transmission of the data. Besides security there are several other factors such 1. Huffman Coding- as transfer speed, cost, errors transmission etc. that plays a vital In computer science and information theory, a Huffman code role in the transmission process. The need for an efficient technique for compression of Images ever increasing because the is a particular type of optimal prefix code that is commonly raw images need large amounts of disk space seems to be a big used for lossless data compression. disadvantage during transmission & storage. This paper provide 1.1 Basic principles of Huffman Coding- a basic introduction about entropy encoding and different coding techniques. It also give comparison between various coding Huffman coding is a popular lossless Variable Length Coding techniques. (VLC) scheme, based on the following principles: (a) Shorter Index Terms – Huffman coding, DEFLATE, VLC, UTF-8, code words are assigned to more probable symbols and longer Golomb coding. code words are assigned to less probable symbols. 1. INTRODUCTION (b) No code word of a symbol is a prefix of another code word. This makes Huffman coding uniquely decodable. 1.1 Entropy Encoding (c) Every source symbol must have a unique code word In information theory an entropy encoding is a lossless data assigned to it.
    [Show full text]
  • STUDIES on GOPALA-HEMACHANDRA CODES and THEIR APPLICATIONS. by Logan Childers December, 2020
    STUDIES ON GOPALA-HEMACHANDRA CODES AND THEIR APPLICATIONS. by Logan Childers December, 2020 Director of Thesis: Krishnan Gopalakrishnan, PhD Major Department: Computer Science Gopala-Hemachandra codes are a variation of the Fibonacci universal code and have ap- plications in data compression and cryptography. We study a specific parameterization of Gopala-Hemachandra codes and present several results pertaining to these codes. We show that GHa(n) always exists for n ≥ 1 when −2 ≥ a ≥ −4, meaning that these are universal codes. We develop two new algorithms to determine whether a GH code exists for a given a and n, and to construct them if they exist. We also prove that when a = −(4 + k) where k ≥ 1, that there are at most k consecutive integers for which GH codes do not exist. In 2014, Nalli and Ozyilmaz proposed a stream cipher based on GH codes. We show that this cipher is insecure and provide experimental results on the performance of our program that cracks this cipher. STUDIES ON GOPALA-HEMACHANDRA CODES AND THEIR APPLICATIONS. A Thesis Presented to The Faculty of the Department of Computer Science East Carolina University In Partial Fulfillment of the Requirements for the Degree Master of Science in Computer Science by Logan Childers December, 2020 Copyright Logan Childers, 2020 STUDIES ON GOPALA-HEMACHANDRA CODES AND THEIR APPLICATIONS. by Logan Childers APPROVED BY: DIRECTOR OF THESIS: Krishnan Gopalakrishnan, PhD COMMITTEE MEMBER: Venkat Gudivada, PhD COMMITTEE MEMBER: Karl Abrahamson, PhD CHAIR OF THE DEPARTMENT OF COMPUTER SCIENCE: Venkat Gudivada, PhD DEAN OF THE GRADUATE SCHOOL: Paul J. Gemperline, PhD Table of Contents LIST OF TABLES ..................................
    [Show full text]
  • Mp3
    <d.w.o> mp3 book: Table of Contents <david.weekly.org> January 4 2002 mp3 book Table of Contents Table of Contents auf deutsch en español {en français} Chapter 0: Introduction <d.w.o> ● What's In This Book about ● Who This Book Is For ● How To Read This Book books Chapter 1: The Hype code codecs ● What Is Internet Audio and Why Do People Use It? mp3 book ● Some Thoughts on the New Economy ● A Brief History of Internet Audio news ❍ Bell Labs, 1957 - Computer Music Is Born pictures ❍ Compression in Movies & Radio - MP3 is Invented! poems ❍ The Net Circa 1996: RealAudio, MIDI, and .AU projects ● The MP3 Explosion updates ❍ 1996 - The Release ❍ 1997 - The Early Adopters writings ❍ 1998 - The Explosion video ❍ sidebar - The MP3 Summit get my updates ❍ 1999 - Commercial Acceptance ● Why Did It Happen? ❍ Hardware ❍ Open Source -> Free, Convenient Software ❍ Standards ❍ Memes: Idea Viruses ● Conclusion page source http://david.weekly.org/mp3book/toc.php3 (1 of 6) [1/4/2002 10:53:06 AM] <d.w.o> mp3 book: Table of Contents Chapter 2: The Guts of Music Technology ● Digital Audio Basics ● Understanding Fourier ● The Biology of Hearing ● Psychoacoustic Masking ❍ Normal Masking ❍ Tone Masking ❍ Noise Masking ● Critical Bands and Prioritization ● Fixed-Point Quantization ● Conclusion Chapter 3: Modern Audio Codecs ● MPEG Evolves ❍ MP2 ❍ MP3 ❍ AAC / MPEG-4 ● Other Internet Audio Codecs ❍ AC-3 / Dolbynet ❍ RealAudio G2 ❍ VQF ❍ QDesign Music Codec 2 ❍ EPAC ● Summary Chapter 4: The New Pipeline: The New Way To Produce, Distribute, and Listen to Music ● Digital
    [Show full text]
  • CRAM Format Specification (Version 3.0)
    CRAM format specification (version 3.0) [email protected] 4 Feb 2021 The master version of this document can be found at https://github.com/samtools/hts-specs. This printing is version c467e20 from that repository, last modified on the date shown above. license: Apache 2.0 1 Overview This specification describes the CRAM 3.0 format. CRAM has the following major objectives: 1. Significantly better lossless compression than BAM 2. Full compatibility with BAM 3. Effortless transition to CRAM from using BAM files 4. Support for controlled loss of BAM data The first three objectives allow users to take immediate advantage of the CRAM format while offering a smooth transition path from using BAM files. The fourth objective supports the exploration of different lossy compres- sion strategies and provides a framework in which to effect these choices. Please note that the CRAM format does not impose any rules about what data should or should not be preserved. Instead, CRAM supports a wide range of lossless and lossy data preservation strategies enabling users to choose which data should be preserved. Data in CRAM is stored either as CRAM records or using one of the general purpose compressors (gzip, bzip2). CRAM records are compressed using a number of different encoding strategies. For example, bases are reference compressed by encoding base differences rather than storing the bases themselves.1 2 Data types CRAM specification uses logical data types and storage data types; logical data types are written as words (e.g. int) while physical data types are written using single letters (e.g.
    [Show full text]
  • Reducing Memory Access Latencies Using Data Compression in Sparse, Iterative Linear Solvers
    College of Saint Benedict and Saint John's University DigitalCommons@CSB/SJU All College Thesis Program, 2016-2019 Honors Program Spring 2019 Reducing Memory Access Latencies using Data Compression in Sparse, Iterative Linear Solvers Neil Lindquist [email protected] Follow this and additional works at: https://digitalcommons.csbsju.edu/honors_thesis Part of the Numerical Analysis and Computation Commons, and the Numerical Analysis and Scientific Computing Commons Recommended Citation Lindquist, Neil, "Reducing Memory Access Latencies using Data Compression in Sparse, Iterative Linear Solvers" (2019). All College Thesis Program, 2016-2019. 63. https://digitalcommons.csbsju.edu/honors_thesis/63 Copyright 2019 Neil Lindquist Reducing Memory Access Latencies using Data Compression in Sparse, Iterative Linear Solvers An All-College Thesis College of Saint Benedict/Saint John's University by Neil Lindquist April 2019 Project Title: Reducing Memory Access Latencies using Data Compression in Sparse, Iterative Linear Solvers Approved by: Mike Heroux Scientist in Residence Robert Hesse Associate Professor of Mathematics Jeremy Iverson Assistant Professor of Computer Science Bret Benesh Chair, Department of Mathematics Imad Rahal Chair, Department of Computer Science Director, All College Thesis Program i Abstract Solving large, sparse systems of linear equations plays a significant role in cer- tain scientific computations, such as approximating the solutions of partial dif- ferential equations. However, solvers for these types of problems usually spend most of their time fetching data from main memory. In an effort to improve the performance of these solvers, this work explores using data compression to reduce the amount of data that needs to be fetched from main memory. Some compression methods were found that improve the performance of the solver and problem found in the HPCG benchmark, with an increase in floating point operations per second of up to 84%.
    [Show full text]
  • The State of the Art in High Throughput Sequencing Data Compression
    The State of the Art in High Throughput Sequencing Data Compression Supplementary Materials Ibrahim Numanagic,´ James K. Bonfield, Faraz Hach, Jan Voges, Jorn¨ Ostermann, Claudio Alberti, Marco Mattavelli, S. Cenk Sahinalp 1 Nature Methods: doi:10.1038/nmeth.4037 Supplementary Tables and Figures Supplementary Figure 1: Visualised single-threaded and 4-threaded performance of FASTQ and SAM tools' compression rate and compression speed. Center of the coordinate system represents the Gzip's (pigz) or Samtools' performance. On the x-axis, left side represents the compression gain over pigz (Samtools), while right side denotes the compression loss, measured in multiples of pigz's (Samtools') size. For example, 1.5 on the left side in the SAM plot indicates that the compression ratio is 1.5x higher than Samtools. On the y-axis, upper part indicates the slow down, while lower part indicates speed-up. Each point represents the performance of one tool on one sample; shaded polygon represents the performance of one tool across the all samples. For each such polygon, its centroid (representing average performance of the tool) is shown as a large point in the middle. 2 Nature Methods: doi:10.1038/nmeth.4037 Sample SRR554369 SRR327342 MH0001.081026 SRR1284073 SRR870667 ERR174310 ERP001775 Original 165 1.00 947 1.00 512 1.00 649 1.00 7,463 1.00 20,966 1.00 368,183 1.00 pigz 48 0.29 277 0.29 149 0.29 188 0.29 2,108 0.28 5,982 0.29 104,927 0.28 pbzip2 44 0.27 251 0.27 139 0.27 176 0.27 1,879 0.25 5,473 0.26 95,969 0.26 DSRC2 41 0.25 257 0.27 128 0.25
    [Show full text]
  • A New Look at Counters: Don't Run Like Marathon in a Hundred Meter Race
    This is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication. The final version of record is available at http://dx.doi.org/10.1109/TC.2017.2710125 1 A New Look at Counters: Don’t Run Like Marathon in a Hundred Meter Race Avijit Dutta, Ashwin Jha and Mridul Nandi Abstract—In cryptography, counters (classically encoded as bit strings of fixed size for all inputs) are employed to prevent collisions on the inputs of the underlying primitive which helps us to prove the security. In this paper we present a unified notion for counters, called counter function family. Based on this generalization we identify some necessary and sufficient conditions on counters which give (possibly) simple proof of security for various counter-based cryptographic schemes. We observe that these conditions are trivially true for the classical counters. We also identify and study two variants of the classical counter which satisfy the security conditions. The first variant has message length dependent counter size, whereas the second variant uses universal coding to generate message length independent counter size. Furthermore, these variants provide better performance for shorter messages. For instance, when the message size is 219 bits, AES-LightMAC with 64-bit (classical) counter takes 1:51 cycles per byte (cpb), whereas it takes 0:81 cpb and 0:89 cpb for the first and second variant, respectively. We benchmark the software performance of these variants against the classical counter by implementing them in MACs and HAIFA hash function.
    [Show full text]
  • Asymmetric Numeral Systems
    Old and new coding techniques for data compression, error correction and nonstandard situations Lecture I: data compression … data encoding Efficient information encoding to: - reduce storage requirements, - reduce transmission requirements, - often reduce access time for storage and transmission (decompression is usually faster than HDD, e.g. 500 vs 100MB/s) Jarosław Duda Nokia, Kraków 25. IV. 2016 1 Symbol frequencies from a version of the British National Corpus containing 67 million words ( http://www.yorku.ca/mack/uist2011.html#f1 ): Brute: 풍품(ퟐퟕ) ≈ ퟒ. ퟕퟓ bits/symbol (푥 → 27푥 + 푠) ′ Huffman uses: 퐻 = ∑푖 푝푖 풓풊 ≈ ퟒ. ퟏퟐ bits/symbol Shannon: 퐻 = ∑푖 푝푖퐥퐠(ퟏ/풑풊) ≈ ퟒ. ퟎퟖ bits/symbol We can theoretically improve by ≈ ퟏ% here 횫푯 ≈ ퟎ. ퟎퟒ bits/symbol Order 1 Markov: ~ 3.3 bits/symbol (~gzip) Order 2: ~3.1 bps, word: ~2.1 bps (~ZSTD) Currently best compression: cmix-v9 (PAQ) (http://mattmahoney.net/dc/text.html ) 109 bytes of text from Wikipedia (enwik9) into 123874398 bytes: ≈ ퟏ bit/symbol 108 → ퟏퟓퟔ27636 106 → ퟏퟖퟏ476 Hilberg conjecture: 퐻(푛) ~ 푛훽, 훽 < 0.9 … lossy video compression: > 100× reduction 2 Lossless compression – fully reversible, e.g. gzip, gif, png, JPEG-LS, FLAC, lossless mode in h.264, h.265 (~50%) Lossy – better compression at cost of distortion Stronger distortion – better compression (rate distortion theory, psychoacoustics) General purpose (lossless, e.g. gzip, rar, lzma, bzip, zpaq, Brotli, ZSTD) vs specialized compressors (e.g. audiovisual, text, DNA, numerical, mesh,…) For example audio (http://www.bobulous.org.uk/misc/audioFormats.html
    [Show full text]