Massively-Parallel Lossless Data Decompression
Evangelia Sitaridi∗, Rene Mueller†, Tim Kaldewey‡, Guy Lohman† and Kenneth A. Ross∗ ∗Columbia University: {eva, kar}@cs.columbia.edu †IBM Almaden Research: {muellerr, lohmang}@us.ibm.com ‡IBM Watson: [email protected]
Abstract—Today’s exponentially increasing data volumes and returns caused by per-block overheads. In order to exploit the high cost of storage make compression essential for the Big the high degree of parallelism of GPUs, with potentially Data industry. Although research has concentrated on efficient thousands of concurrent threads, our implementation needs compression, fast decompression is critical for analytics queries that repeatedly read compressed data. While decompression can to take advantage of both intra-block parallelism and inter- be parallelized somewhat by assigning each data block to a block parallelism. For intra-block parallelism, a group of different process, break-through speed-ups require exploiting the GPU threads decompresses the same data block concurrently. massive parallelism of modern multi-core processors and GPUs Achieving this parallelism is challenging due to the inherent for data decompression within a block. We propose two new data dependencies among the threads that collaborate on techniques to increase the degree of parallelism during decom- pression. The first technique exploits the massive parallelism decompressing that block. of GPU and SIMD architectures. The second sacrifices some In this paper, we propose and evaluate two approaches to compression efficiency to eliminate data dependencies that limit address this intra-block decompression challenge. The first parallelism during decompression. We evaluate these techniques technique exploits the SIMD-like execution model of GPUs on the decompressor of the DEFLATE scheme, called Inflate, which is based on LZ77 compression and Huffman encoding. to coordinate the threads that are concurrently decompressing We achieve a 2× speed-up in a head-to-head comparison with a data block. The second approach avoids data dependencies several multi-core CPU-based libraries, while achieving a 17 % encountered during decompression by proactively eliminat- energy saving with comparable compression ratios. ing performance-limiting back-references during the compres- sion phase. The resulting speed gain comes at the price I.INTRODUCTION of a marginal loss of compression efficiency. We present With exponentially-increasing data volumes and the high Gompresso/Bit, a parallel implementation of an Inflate-like cost of enterprise data storage, data compression has become scheme [1] that aims at high decompression speed and is essential for reducing storage costs in the Big Data era. There suitable for massively-parallel processors such as GPUs. We exists a plethora of compression techniques, each having a also implement Gompresso/Byte, based on LZ77 with byte- different trade-off between its compression ratio (compression level encoding. It trades off slightly lower compression ratios efficiency) and its speed of execution (bandwidth). Most for an average 3× higher decompression speed. research so far has focused on the speed of compressing data In summary, the contributions of this paper are: as it is loaded into an information system, but the speed of decompressing that data can be even more important for Big • A technique to achieve massive intra-block parallelism Data workloads – usually data is compressed only once at during decompression by exploiting the SIMD-like ar- load time but repeatedly decompressed as it is read when chitecture of GPUs. executing analytics or machine learning jobs. Decompression • Improved intra-block parallelism by eliminating data speed is therefore crucial to minimizing response time of these dependencies during compression at a slight cost of compression efficiency. arXiv:1606.00519v1 [cs.DC] 2 Jun 2016 applications, which are typically I/O-bound. In an era of flattening processor speeds, parallelism provides • An evaluation of the impact of both techniques on com- our best hope of speeding up any process. In this work, pression ratio and speed. we leverage the massive parallelism provided by Graphics • Comparisons of Gompresso’s decompression speed and Processing Units (GPUs) to accelerate decompression. GPUs energy efficiency on the Tesla K40 GPU against several have already been successfully used to accelerate several other state-of-the-art multi-core CPU libraries, in which we data processing problems, while concomitantly providing a show that Gompresso/Bit is 2× faster while achieving better Performance/Watt ratio than conventional CPUs, as well. a 17 % energy saving. However, accelerating decompression on massively parallel Section II gives background on the essentials of the GPU processors like GPUs presents new challenges. Straightfor- architecture, and in Section II-C we discuss related work. Sec- ward parallelization methods, in which the input block is tion III analyzes how Gompresso parallelizes decompression simply split into many, much smaller data blocks that are to harvest the massive parallelization of GPUs. Section IV then processed independently by each processor, result in focuses on the alternative dependency resolution strategies we poorer compression efficiency, due to the reduced redundancy designed for LZ77. Section V presents the experimental results in the smaller blocks, as well as diminishing performance of tuning and comparing Gompresso against state-of-the-art Uncompressed Input Compressed Output words with varying bit-lengths. We also consider entropy en- a a c a a c b a c a d d 'a','a','c' (1) coding that operates at the level of bytes rather than bits. This sacrifices some compression efficiency for speed. Existing Find longest match Literal Back-reference dictionary-based compression schemes that use byte-level cod- a a c a a c b a c a d d 'a','a','c', (0,3) (2) ing are LZRW1 [8], Snappy [9], and LZ4 [10]. We refer to the implementation using bit-level encoding as Gompresso/Bit. Match position Match length Similarly, the entropy encoder in Gompresso/Byte operates Recently Encoded Input Unencoded Lookahead Characters Minimum Match Length:3 at the byte level. Fig. 1: Illustration of LZ77 compression: (1) Literal is emitted because there was no match for ‘c’. (2) Back-reference is B. GPU Background emitted for a match on ‘aac’. We use the NVIDIA CUDA terminology and program- ming model; however, our work can be extended to other standards, such as OpenCL. GPU-accelerated applications are parallel CPU libraries. Finally, in Section VI we summarize implemented as a number of kernels. A kernel is a function our conclusions and suggest some interesting directions for that is invoked by the host CPU and is executed on the future work. A shorter version of the paper to appear as is[2]. GPU device. A kernel function is executed by a number of thread-groups. 1 A thread-group is further divided into II.BACKGROUNDAND RELATED WORK smaller units of 32 threads, called a warp. The threads in A. LZ77 Compression a given warp execute all the same instructions in lock step. Understanding this seemingly minor architectural detail is A common type of data compression replaces frequently- essential for our first data dependency resolution technique, occurring sequences with references to earlier occurrences. described in Section III. The execution model of a warp is This can be achieved by maintaining a dictionary of essentially equivalent to a 32-way single-instruction, multiple- frequently-occurring patterns, such as in the LZ78 [3] and data (SIMD) architecture that allows branch instructions. In the LZW [4] algorithms, or by maintaining a sliding window over presence of branches, the threads in a warp may have to follow the most recent data, as in the LZ77 algorithm [5]. A further different execution paths (diverge), based upon the outcome space reduction can be achieved by encoding individual char- of the branch condition on each thread. Due to the lock-step acters or dictionary symbols as variable-length code words. execution of the threads within a warp, however, these different This so-called entropy encoding assigns code words with fewer paths will be serialized. Thus, for better performance, ideally bits to more frequently occurring symbols. Popular entropy threads in a warp should not diverge and instead follow the encoding schemes are Huffman [6] or Arithmetic [7] coding. same execution path. Compression schemes typically combine a dictionary-based Because of the significance of warps as the unit of exe- technique and entropy coding. In this paper, we study a variant cution, GPUs provide several instructions that allow threads of the popular DEFLATE [1] compression scheme, which is within a warp to exchange data and reach consensus. We used in the gzip, ZIP, and PNG file formats. More precisely, we will use the following two instructions in this paper: The focus on the decompression process, called Inflate. DEFLATE ballot(b) instruction combines a binary voting bit bi from uses the LZ77 dictionary scheme followed by Huffman coding. each thread i and returns them to threads as a 32-bit value The dictionary in LZ77 is a sliding window over the recently 31 b312 + ··· + b12 + b0 that represents the individual votes. processed input. The LZ77 compressor produces a stream The “shuffle” shfl(v,i) instruction broadcasts the value v of token symbols, in which each token is either a back- of thread i to all other threads of the warp. reference to a position in the sliding window dictionary, or a literal containing a sequence of characters, if that sequence C. Related Work does not appear in the sliding window. A back-reference is represented as a tuple with the offset and the length of a match Although there are numerous compression schemes, we in the sliding window. Figure 1 illustrates both token types focus in this section on just the parallelization attempts of in a simple example. In the first step, the sequence ‘aa’ has the best-known compression schemes. already been processed and is in the sliding window dictionary. a) Parallel CPU Implementations: A parallel implemen- ‘caac. . . ’ is the input to be processed next. Since the window tation for CPUs of gzip compression in the pigz library [11] does not contain any sequence starting with ‘c’, LZ77 emits a achieves a linear speed-up of compression with the number literal token ‘c’ and appends ‘c’ to the window. The sequence of CPU cores. Decompression in pigz, however, has to be to be processed in the subsequent step is ‘aacb. . . ‘. ‘aac’ is single-threaded because of its variable-length blocks. Another found in the window at position 0. The match is 3 characters CPU compression library, pbzip [12], parallelizes the set of long, hence, LZ77 emits back-reference token (0,3). algorithms implemented by the bzip2 scheme. The input is The resulting stream of literal and back-reference tokens 1In CUDA, thread-groups are called thread blocks and in OpenCL work are then converted into sequences of codewords by an entropy groups. We use the term “group” instead of “block” to avoid confusion with coder. DEFLATE uses Huffman coding, which yields code the term data blocks. split into data blocks that can be compressed and decom- and in parallel. The block size is a configurable run-time pressed in parallel. As already described in the Introduction, parameter that is chosen depending on the total data size this inter-block parallelism alone is insufficient and results in and the number of available processing elements on the poor performance on GPUs. GPU. Each block is LZ77-compressed by a group of threads b) Hardware-Accelerated Implementations: Parallelizing using an exhaustive parallel matching technique we described compression schemes within a block is a bigger challenge earlier [23]. For Gompresso/Byte, the pipeline ends here, and for massively-parallel processors. For example, the GPU im- the resulting token streams are written into the output file plementation of bzip2 did not improve performance against using a direct byte-level encoding. Gompresso/Bit requires the single-core CPU bzip2 [13]. The major bottleneck was an additional step in which the tokens are encoded using a the string sort required for the Burrow-Wheeler-Transform Huffman coder. Similar to DEFLATE, Gompresso/Bit uses (BWT) compression layer. Future compressor implementations two separate Huffman trees to facilitate the encoding, one for could be accelerated by replacing string sort with suffix array the match offset values and the second for the length of the construction [14], [15], [16]. matches and the literals themselves. Both trees are created Most research has focused on accelerating compression, from the token frequencies for each block. To facilitate parallel rather than decompression [17]. Here, we address the thread decoding later on, the tokens of the data blocks are further split dependencies that limit the parallelism of the LZ77 decom- into smaller sub-blocks during encoding. A run-time parameter pression. In our implementation each thread writes multiple allows the user to set the number of sub-blocks per data back-reference characters at a time, avoiding the high per block; more sub-blocks per block increases parallelism and character cost. A parallel algorithm for LZ decompression, hence performance, but diminishes sub-block size and hence depending on the type of data dependencies, does not guar- compression ratio. Each encoded sub-block is written to the antee efficient GPU memory access[18]. Huffman encoding is output file, along with its compressed size in bits. The parallel typically added to improve the compression ratio[19]. How- decoder can determine the location of the encoded sub-blocks ever, decoding is hard to parallelize because it has to identify in the compressed bitstream with this size information. Finally, codeword boundaries for variable-length coding schemes. Our the Huffman trees are written in a canonical representation [6]. parallel decoding method splits data blocks into smaller sub- Figure 3 shows the structure of the compressed file format blocks to increase the available parallelism. We trade-off in detail. a little of compression efficiency but only make only one pass over the encoded data. Alternative parallel decoding B. Parallel Decompression algorithms do not affect the compression ratio but they require Gompresso/Byte can combine decoding and decompression multiple passes to decode the data for BWT decompression: A in a single pass because of its fixed-length byte-level coding first pass to determine the codeword boundaries and a second scheme. The token streams can be read directly from the for the actual decoding [15]. compressed output. Gompresso/Bit uses a variable-length Simpler compression schemes have been implemented on coding scheme for a higher compression ratio, and therefore GPUs in the context of a database system [20], but while needs to first decode the bitstream into a stream of tokens these algorithms achieve good compression ratios for database before proceeding with the LZ77 decompression. Gompresso columns, they are not efficient for Big Data workloads that assigns a group of GPU threads to collaborate on the Huff- might be unstructured. FPGAs and custom hardware have also man decoding and LZ77 decompression on the independently been used to accelerate compression, resulting in high speed- compressed data blocks. This permits an additional degree of ups [21], [22]. However, these hardware devices have very parallelism within data blocks. different characteristics and constraints than GPUs, so their 1) Huffman Decoding: Each thread of a group decodes a parallelization techniques generally aren’t applicable to GPUs. different sub-block of the compressed data block. The starting III. GOMPRESSO OVERVIEW offset of each sub-block in the bitstream is computed from the sub-block sizes in the file header. All sub-blocks of a given In this section, we provide an overview of Gompresso, data block decode their bitstreams using look-up tables created which exploits parallelism between and also within data from the same two Huffman trees for that block and stored in blocks. The most important design goal for Gompresso is a the software-controlled, on-chip memories of the GPU. We can high decompression speed, while maintaining a “reasonable” retrieve the original token symbol with a single lookup in each compression ratio. Gompresso implements both compression table, which is much faster than searching through the (more and decompression, and defines its own file format. compact) Huffman trees, which would introduce branches and Figure 2 gives an overview of the Gompresso compression hence divergence of the threads’ execution paths. The output of and decompression algorithms. We first briefly outline the the decoder is the stream of literal and back-reference tokens, parallel compression phase before describing parallel decom- and is written back to the device memory. pression, which is the focus of this paper. 2) LZ77 Decompression: Each data block is assigned to A. Parallel Compression a single GPU warp (32 threads operating in lock-step) for In the first step, Gompresso splits the input into equally- decompression. We chose to limit the group size to one warp sized data blocks, which are then compressed independently in order to be able to take advantage of the efficient voting Compression Decompression Thread group 1 Thread group 1 Sub-block 1 Sub-block 1 Block 1 Sub-block 2 Sub-block 2 Sub-block 3 Sub-block 3 Thread group 2 Thread group 2 Block 2 Sub-block 1 Sub-block 1 Sub-block 2 Sub-block 2 Sub-block 3 Sub-block 3 ...... Thread group n ...... Thread group n ... Sub-block 1 Sub-block 1 Block n Sub-block 2 Sub-block 2 Sub-block 3 Sub-block 3 LZ77 Compression Huffman Encoding Huffman Decoding LZ77 Decompression Uncompressed input LZ77 token stream Huffman coded bitstream Huffman coded bitstream LZ77 token stream Decompressed output
Fig. 2: Gompresso splits the input into equally-sized blocks, which are then LZ77-compressed independently and in parallel. In Gompresso/Bit, the resulting token streams are further split into equal-sized sub-blocks and Huffman-encoded. The inverse process follows for decompression. and shuffling instructions within a warp. Larger thread groups start position of the block. This prefix sum is computed from would require explicit synchronization and data exchange via the total number of bytes that each thread will write for its on-chip memory. We found that the potential performance gain sequence, i.e., the length of its literal string plus the match by the increased degree of parallelism is canceled out by this length of the back-reference. Once the source and destination additional coordination overhead. positions are determined from the two prefix sums, the threads We first group consecutive literals into a single literal string. can copy the literal strings from the token stream into the We further require that a literal string is followed by a back- output buffer. reference and vice versa, similar to the LZ4 [10] compression c) Copying back-references: This is the most challenging scheme. A literal string may have zero length if there is no step for parallel decompression, because of the data depen- literal token between two consecutive back-references. A pair dencies between threads in a warp. These dependencies arise consisting of a literal string and a back-reference is called a sequence. We assign each sequence to a different thread (see Figure 4). In our experiments, we found that this grouping File Header 1GB results in better decompression speed, since it not only assigns Uncompressed file size Block 1 Block size 32KB each thread a larger unit of work but its uniformity suits the Dictionary size 4096 lock-step execution model of the GPU. All threads in the Maximum match length 256 Block 2 #Tokens/Sub-block 16 warp concurrently alternate between executing instructions for Sub-block size list ... string literals and for back references. For each sequence, its [... thread performs: (a) read its sequence from device memory ] Block n and compute the start position of its string literal, (b) determine Compressed file header the output position of its literal, and copy its string literal to Compressed file the output buffer, and (c) resolve and write its back-reference. Compressed Block Contents We now describe each step in more detail: 01010101000110001111100110101110001100111 a) Reading sequences: Each warp uses the block offset Literal tree Match distance tree Compressed bitstream to determine the location of the first decoded token in the device memory. Each thread in the warp will read a different Fig. 3: The Gompresso file format, consisting of: (1) a file sequence (see Figure 4). The threads then need to determine header, (2) a sequence of compressed data blocks, each with its the start location of their literal strings in the token stream. two Huffman trees (Gompresso/Byte does not use Huffman This is accomplished by computing an intra-warp exclusive trees.) and encoded bitstream. prefix sum from the literal lengths of their sequences, in order to locate the start positions from which they can copy T1 T2 T3 Sequence 1 Sequence 2 Sequence 3 their literal strings. We use NVIDIA’s shuffle instructions to efficiently compute this prefix sum without memory accesses, Read input'aac', (0,3), 'b',(3,3),'d',(3,4) a common GPU technique. b) Copying literal strings: Next, the threads compute write positions in the decompressed output buffer. Since all aa ac a cd c ac a aaac a c cac bb caaa a caa c ddaca aa c c b blocks, except potentially the last, have the same uncom- 0 1 2 34 5 6 7 108911 12 13 14 pressed size, the threads can also easily determine the start position of their block in the uncompressed output stream. Fig. 4: Nested back-references: back-references in Sequence The start position of each thread’s literal string is determined 2 and 3 depend on Sequence 1, and cannot be resolved before by a second exclusive prefix sum, which is then added to the the output of Sequence 1 is available. 1: function MRR(HWM, read pos, write pos, length) track of the high-water mark (HWM) position of the output 2: pending ← true . thread has not written any output that has been written so far without gaps. A back-reference 3: do whose referenced interval is below the HWM can therefore be 4: if pending and read pos+length≤HWM then resolved. In each iteration, threads that have not yet written 5: copy length bytes from read pos to write pos their output use the high-water mark (HWM) to determine 6: pending ← false whether their back reference can be resolved (line 4). If so, 7: end if they copy the data from the referenced sequence to the output, 8: votes ← ballot(pending) and indicate that they completed their work (lines 5 and 6). 9: last writer ← count leading zero bits(votes) 10: HWM ← shfl(write pos+length, last writer) The HWM is updated at the end of each iteration. The 11: while votes> 0 . Repeat until all threads done algorithm determines the last sequence that was completed by 12: return HWM the warp, and sets the HWM past the highest write position of 13: end function that sequence’s back-reference. The threads can determine the last sequence without accessing shared memory by exploiting Fig. 5: Multi-Round Resolution (MRR) Algorithm the warp-voting instruction ballot on the pending flag (line 8). This produces a 32-bit bitmap that contains the pending states of all threads in this warp. Each thread when a back-reference points to another back-reference, and receives this bitmap and then counts the number of leading thus cannot be resolved before the former has been resolved. zeros in the bitmap in order to determine the ID of the We address these nested back-references in Section IV. After last_writer thread that completed the last sequence. A all the back-references have been resolved, the warp continues subsequent warp-shuffle instruction broadcasts the new HWM with the next 32 sequences. computed by the last_writer thread to all other threads in the warp (line 10). The iteration completes when all threads IV. DATA DEPENDENCIESIN NESTEDBACK-REFERENCES have processed their back-references. Before processing a back-reference, the data pointed to by this reference needs to be available in the output. This Τ1 Τ2 Τ3 introduces a data dependency and stalls threads with dependent Sequence 1 Sequence 2 Sequence 3 references until the referenced data becomes available. The Read input 'aac', (0,3), 'b',(3,3), 'd', (3,4) problem is illustrated in Figure 4. Threads T2 and T3 will have to wait for T1 to finish processing its sequence, because Write literal strings a a c db they both have back-references that point into the range that is HWM=2 0 1 2 34 5 6 7 9 108 11 12 13 14 written by T1. Resolving back-references sequentially would T1 writes B1 a a c a ac b d produce the correct output, but would also under-utilize the HWM=6 01 2 34 5 6 79 11108 12 13 14 available thread resources. To maximize thread utilization, T2 writes B2, a a c a a c b a a c da da c b we propose two strategies to handle these data dependencies. T3 writes B3 HWM=14 The first strategy uses warp shuffling and voting instructions 0 1 2 34 5 6 7 9 108 11 12 13 14 to process dependencies as soon as possible, i.e., as soon Fig. 6: Multi-Round Resolution (MRR) execution as all of the referenced data becomes available. The second strategy avoids data dependencies altogether by prohibiting construction of nested back-references during compression. Figure 6 illustrates the execution of MRR, for the set of 3 This second approach unfortunately reduces compression ef- sequences from Figure 4. Initially, all threads write in parallel ficiency somewhat, which we will quantify experimentally in their string of literals. In the next step, T1 copies the back- Section V. reference of Sequence 1. In the last step, after Sequence 1 has been processed, the dependencies of T2 and T3 are satisfied, A. Multi-Round Resolution (MRR) of Nested back-references so both threads can proceed to copy their back-references. Figure 5 shows the Multi-Round Resolution (MRR) algo- At least one back-reference is resolved during each iteration rithm for iterative resolution of nested back-references, which which guarantees termination of the algorithm. The HWM is executed by every thread in the warp. We follow the GPU increases strictly monotonically. The degree of achievable programming convention in which each of the variables is parallelism depends on nesting of back-references. As soon thread-private unless it is explicitly marked as locally or as the referenced ranges falls below the HWM they can be globally shared. The Boolean variable pending is initially set resolved simultaneously. Back-references that do not depend on 2 and is cleared once the thread has copied its back- on data produced by other back-references from the same warp reference to the output (line 6). can be resolved in one round leading to maximum parallelism Before calling MRR, all threads have written their literal of the warp. In the worst-case scenario all but one back- string from their sequence to the output, but no thread in the reference depends on another back-reference in the same warp. warp has written a back-reference yet. In order to determine MRR then leads to sequential execution. The next section when the referenced data becomes available, the threads keep describes a strategy that avoids this scenario. 1: pos ← 0 due to the additional checking and bookkeeping. As we will 2: while pos
#Bytes 1 10 100 0.1 5 50 0.01 Decompression time (ms) 0 Decompression speed (GB/s) 0 1 2 3 4 5 6 7 8 9 10 11 12 0 5 10 15 20 25 30 35 Wikipedia Matrix Round Nesting depth (a) (b) (c) Fig. 9: (a) Decompression speed of Gompresso/Byte (data transfer cost not included), using different dependency resolution strategies for the two datasets. (b) Number of bytes processed on each round of MRR. (c) Decompression speed of MRR as a function of the number of resolution rounds, for an artificially generated dataset. a 0.77 GB CSV file Matrix Market file [25]. Both sets are threads in the second round divided by the number of MRR highly compressible. For comparison, the gzip tool achieves a iterations executed for a dataset. The lower performance of compression ratio of 3.09:1 for the former and 4.99:1 for the MRR was surprising, given that we observed relatively few latter, using the default compression level setting (–6). The bytes processed after the first round. However, what limits performance measurements are conducted on a dual-socket performance is the number of rounds. For the Wikipedia system with two Intel E5-2620 v2 CPUs, 2 × 6 cores running dataset, the average number of resolution rounds is around 24 hardware threads. We add an NVIDIA Tesla K40 with 3, and for the Matrix dataset, 4. 2,880 CUDA cores to the system for the GPU measurements. To better understand the performance impact of multiple The device is connected via a PCI Express (PCIe) 3.0 x16 passes, we created a collection of artificial 1 GB datasets link with a nominal bandwidth of 16 GB/sec in each direction. that induce a specified depth of back-reference nesting. We We report bandwidth numbers that include PCIe transfers. In generate each dataset such that it leads to the desired depth. cases in which the PCIe bandwidth becomes the bottleneck, we The general idea is as follows: we repeat a 16-byte string report the bandwidth with input and output data residing in the with a one-byte change occurring in an alternating fashion GPU’s device memory. ECC is turned on in our measurements. at the first and last byte position. We chose the length of We determine the decompression bandwidth as the ratio of the 16 to be close to the average back-reference match length size of the uncompressed data over the total processing time. in the two real datasets used in our evaluation. A separator Unless otherwise noted, we are using a data block size of byte, chosen from a disjoint set of bytes, is used to prevent 256 KB and a sliding window of 8 KB. For compression, we accidental and undesired matches that cross different instances look at the next 64 bytes in the input for each match search in of the repeating string. the 8 KB window. To facilitate parallel Huffman decoding in Gompresso/Bit, we split the sequence stream into sub-blocks 32 rounds that are 16 sequences long. UABCDVABCEWFBCEYFBCHZ... A. Performance Impact of Nested back-references 16 rounds We first focus on just the LZ decompression throughput of Gompresso/Byte, i.e., with no entropy coding, for different UABCDVEFGHWABCEYEFGAZ... resolution strategies in Figure 9a. Sequential Copying (SC) is our baseline, in which threads copy their back-references in Fig. 10: Series of sequences inducing 32 and 16 rounds of a sequential order without intra-block parallelism. The figure resolution. shows that Dependency Elimination (DE) is the fastest strategy for decompression. It is at least 5× faster than SC. We place Figure 10 illustrates how sequences of nested back- the compressed input and the decompressed output in device references are created. We show two small examples for strings memory in this setup, and ignore PCIe transfers. The figure of length four, rather than 16 bytes, for space reasons. The shows that the decompression throughput is higher than the separator bytes are printed in black, while the repeating string theoretical maximal bandwidth of the PCIe link. As expected, is shown in green and orange colors. The arrows show the Multi-Round Resolution (MRR) performs better than SC due dependencies in the MRR algorithm. LZ decompression of the to the higher degree of parallelism, while DE out-performs dataset shown on top in Figure 10 will incur data dependencies MRR because it achieves an even higher degree of parallelism. of all 32 threads in the warp except the first, whose dependency Figure 9b shows the average number of bytes that are does not cause a stall because it points to the data that was resolved from back-references in each round. For example, processed by this warp previously. The nesting depth in a warp for round 2, we sum the number of bytes copied by the active is 32, so completing the resolution requires 32 rounds. In order 4 6 3.5 w/o DE 5 w/ DE 3 4 2.5 2 3 1.5 2 1 1
Compression ratio 0.5 0 0 Decompression speed (GB/s) 32 64 128 256 Wikipedia Matrix Block size (KB)
300 4 w/o DE 3.5 250 w/ DE 3 200 2.5 150 2 1.5 100 1 Compression ratio 50 0.5 0 0 32 64 128 256 Compression speed (MB/s) Wikipedia Matrix Block size (KB) Fig. 11: Degradation in compression ratio and compression Fig. 12: Decompression speed (data transfer cost included) and speed ratio of Gompresso/Bit for different block sizes to generate datasets with a smaller nesting depth, we alternate C. Dependency on Data Block Size multiple distinct repeated strings. For example, two repeated Figure 12 shows the decompression speed and compression strings result in depth 16, four repeated strings in depth 8, and ratio for different data block sizes. Larger blocks increase the so on. For a depth of 16, in each round, two back-references available parallelism for Huffman decoding because there are are copied, one for each repeated string. These two strings are more parallel sub-blocks in flight. Threads operating on sub- marked in green and orange in the lower example in Figure 10. blocks that belong to the same data block share the Huffman Figure 9c shows the decompression time for different nesting decoding tables, which are stored in the software-controlled, depths. The decompression time increases sharply until about on-chip memory of the GPU. This intra-block parallelism 16 rounds. The primary reason for the slower performance leads to a better utilization of the GPU’s compute resources of MRR is that all threads in a warp have to wait until the by scheduling more data blocks on the GPU’s processors entire warp’s back-references have been resolved. Threads that for concurrent execution (inter-block parallelism). The space resolve on the first round will be underutilized while other required by the Huffman decoding tables in the processors’ threads do work in subsequent passes. on-chip memory limit the number of data blocks that can be We also implemented an alternative variant of MRR that decoded concurrently on a single GPU processor. wrote nested back-references to device memory during each 2CWL round. Each round is performed in a separate kernel. Later Each Huffman decoding table has entries, where CWL passes read unresolved back-references and all threads in a is the maximum codeword length. To fit the look-up warp can be doing useful work. Because of the overhead tables in the on-chip memory, we are using limited-length CWL = 10 of writing to and reading from memory, together with the Huffman encoding with a maximum length of bits. increased complexity of tracking when a dependency can be Figure 12 shows that the compression ratio only marginally resolved, the alternative variant did not improve the perfor- degrades for smaller blocks, so the space overhead of storing mance of MRR. the block header for each compressed data block is not significant. B. Impact of DE on Compression Ratio and Speed Figure 11 shows the degradation in compression ratio and D. GPU vs. Multi-core CPU Performance compression speed when eliminating dependencies using the Lastly, we compare the performance of Gompresso to Dependency Elimination (DE) algorithm we implemented by state-of-the-art parallel CPU libraries regarding decompression modifying the LZ4 library. The maximum degradation is 13 % speed and overall energy consumption. We used a power in compression speed and 19 % in compression ratio, which meter to measure energy consumption at the wall socket. For is acceptable when we are aiming at fast decompression. CPU-only environments, we physically removed the GPUs In the remaining experiments, we use the DE method for from our server to avoid including the GPU’s idle power. We decompression. parallelized the single-threaded implementations of the CPU- Speed/Ratio English Wikipedia Energy/Ratio English Wikipedia 18 Gomp/Byte (No PCIe) 90 zlib 16 80 Zstd 14 70 Gomp/Bit Gomp/Byte (In) 12 LZ4 (CPU) 60 Gomp/Byte (In/Out) 10 50 Snappy (CPU) 8 40 Snappy Gomp/Byte (In/Out) 6 Gomp/Bit LZ4 Zstd (CPU) 30
4 Energy (Joules) zlib (CPU) Gomp/Byte (No PCIe) 2 0 20
Decompression speed (GB/s) 1.8 2 2.2 2.4 2.6 2.8 3 3.2 3.4 1.8 2 2.2 2.4 2.6 2.8 3 3.2 3.4 Compression ratio Compression ratio Speed/Ratio Sparse Matrix Fig. 14: GPU vs. multicore CPU energy consumption 18 16 Gomp/Byte (In) 14 bottleneck. In separate bandwidth tests, we were able to 12 Gomp/Byte (In/Out) achieve a PCIe peak bandwidth of 13 GB/sec. Gompresso/Bit, 10 though not PCIe-bound, is still 2× faster than zlib and Snappy (CPU) 8 Gompresso/Byte is 1.35× faster than LZ4. For the matrix 6 LZ4 (CPU) Gomp/Bit dataset, the decompression speed of Gompresso/Bit is around 4 Zstd(CPU) zlib (CPU) 2× faster than zlib. There is around 9 % degradation in com- 2 pression ratio because we use limited-length Huffman coding. 0
Decompression speed (GB/s) 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 Although it lowers the compression efficiency, it enables us to Compression ratio fit more Huffman decoding tables into the on-chip memory. Fig. 13: GPU vs multicore CPU performance. Cost for trans- Finally, we compare the energy consumed to decompress ferring data to and from the GPU is included for Gom- the Wikipedia dataset. In general, faster decompression on the presso/Bit. For Gompresso/Byte, we show the performance same hardware platform results in improved energy efficiency. both including and not including data transfers. This is because the power drawn at the system level, i.e., at the wall plug, does not differ significantly for different algorithms. More interesting is the energy efficiency when comparing different implementations on different hardware based state-of-the-art compression libraries by splitting the platforms, e.g., a parallel CPU vs. a GPU solution. Figure 14 input data into equally-sized blocks that are then processed shows the overall energy consumption versus the compression by the different cores in parallel. We chose a block size of ratio for Gompresso and a number of parallelized CPU-based 2 MB, as this size resulted in the highest decompression speeds libraries. Gompresso/Bit consumes 17 % less energy than the for the parallelized libraries. Once a thread has completed parallel zlib library. It also has similar energy consumption to decompressing a data block, it immediately processes the next Zstd, which implements a faster coding algorithm. block from a common queue. This balances the load across CPU threads despite input-dependent processing times for the VI.CONCLUSIONSAND FUTURE WORK different data blocks. Here, we developed techniques within our compression Figure 13 shows the trade-offs between decompression framework, Gompresso, for massively parallel decompression speed and compression ratio. In addition to the measurements using GPUs. We presented one solution for parallelizing of our Gompresso system, we include the performance of two Huffman decoding by using parallel sub-blocks, and two byte-level compression libraries (LZ4, Snappy) and for two techniques to resolve back-references in parallel. The first libraries using bit-level encoding (gzip, zlib) for comparison. technique iteratively resolves back-references and the second Zstd implements a different coding algorithm on top of LZ- eliminates data dependencies during compression that will stall compression that is typically faster than Huffman decoding, parallelism among collaborating threads concurrently decom- and we include it in our measurements for completeness [26]. pressing that set of sub-blocks. Gompresso decompresses two zlib implements the DEFLATE scheme for the CPU. For the real-world datasets 2× faster than the state-of-the-art block- GPU measurements, we show the end-to-end performance, parallel variant of zlib running on a modern multi-core CPU, including times for: (a) both compressed input and uncom- while suffering no more than a 10 % penalty in compression pressed output over PCIe, marked (In/Out) in Figure 13; (b) ratio. Gompresso also uses 17 % less energy by using GPUs. only the input transfers, marked as (In); and (c) ignoring data Future work includes determining the extent to which our transfers altogether, marked as No PCIe. techniques can be applied to alternative coding and context- For Gompresso/Byte, PCIe transfers turned out to be the based compression schemes, and evaluating their performance. ACKNOWLEDGMENT [12] J. Gilchrist and Y. Nikolov, “Parallel BZIP2,” http://compression.ca/ pbzip2/, accessed: 2015-04-02. Authors Evangelia Sitaridi and Kenneth Ross were partially [13] R. A. Patel, Y. Zhang, J. Mak, A. Davidson, and J. D. Owens, “Parallel supported by National Science Foundation grant IIS-1218222 lossless data compression on the GPU,” in InPar, May 2012, p. 9. and by an equipment gift from NVIDIA Corporation. [14] M. Deo and S. Keely, “Parallel suffix array and least common prefix for the GPU,” in PPoPP, 2013, pp. 197–206. REFERENCES [15] J. A. Edwards and U. Vishkin, “Parallel algorithms for Burrows–Wheeler compression and decompression,” TCS, vol. 525, pp. 10 – 22, 2014. [1] P. Deutsch, “DEFLATE compressed data format specification version [16] L. Wang, S. Baxter, and J. D. Owens, “Fast parallel suffix array on the 1.3,” RFC 1951 (Informational), IETF, may 1996. GPU,” in Euro-Par, 2015, pp. 573–587. [2] E. Sitaridi, R. Mueller, T. Kaldewey, G. Lohman, and K. Ross, [17] A. Ozsoy and M. Swany, “CULZSS: LZSS lossless data compression “Massively-parallel lossless data decompression,” in ICPP, 2016. on CUDA,” in CLUSTER, 2011, pp. 403–411. [3] J. Ziv and A. Lempel, “Compression of individual sequences via [18] S. D. Agostino, “Speeding up parallel decoding of LZ compressed text variable-rate coding,” IEEE Trans. on Information Theory, vol. 24, no. 5, on the PRAM EREW,” in SPIRE, 2000, pp. 2–7. pp. 530–536, Sep 1978. [19] A. Ozsoy, D. M. Swany, and A. Chauhan, “Optimizing LZSS compres- [4] T. A. Welch, “A technique for high-performance data compression,” sion on GPGPUs,” FGCS, vol. 30, pp. 170–178, 2014. Computer, vol. 17, no. 6, pp. 8–19, June 1984. [20] W. Fang, B. He, and Q. Luo, “Database compression on graphics [5] J. Ziv and A. Lempel, “A universal algorithm for sequential data processors,” Proc. VLDB Endow., vol. 3, no. 1-2, pp. 670–680, 2010. compression,” IEEE Trans. on Information Theory, vol. 23, no. 3, pp. [21] Xilinx, “GUNZIP/ZLIB/Inflate data decompression core,” http:// 337–343, May 1977. www.xilinx.com/products/intellectual-property/1-79drsh.html, accessed: [6] D. A. Huffman, “A method for the construction of minimum-redundancy 2015-11-14. codes,” Proc. of the I.R.E., pp. 1098–1102, Sep 1952. [22] M. S. Abdelfattah, A. Hagiescu, and D. Singh, “Gzip on a chip: High [7] J. J. Rissanen, “Generalized Kraft inequality and arithmetic coding,” performance lossless data compression on FPGAs using OpenCL,” in IBM Journal of Research and Development, vol. 20, no. 3, pp. 198– IWOCL, 2014, pp. 4:1–4:9. 203, May 1976. [23] E. A. Sitaridi, R. Mueller, and T. Kaldewey, “Parallel lossless compres- [8] R. Williams, “An extremely fast Ziv-Lempel data compression algo- sion using GPUs,” in GPU Technology Conference GTC’13, 2013. rithm,” in Data Compression Conference, 1991. DCC ’91., Apr 1991, [24] M. Mahoney. Large text compression benchmark. [Online]. Available: pp. 362–371. http://mattmahoney.net/dc/enwik9.zip [9] S. H. Gunderson, “Snappy a fast compressor/decompressor,” https: [25] UF sparse matrix collection. [Online]. Available: http://cise.ufl.edu/ //github.com/google/snappy, accessed: 2015-04-01. research/sparse/MM/LAW/hollywood-2009.tar.gz [10] Y. Collet, “LZ4–extremely fast compression,” https://github.com/ [26] Y. Collet, “Zstandard–fast and efficient compression algorithm,” https: Cyan4973/lz4, accessed: 2015-04-02. //github.com/Cyan4973/zstd, accessed: 2015-04-02. [11] M. Adler, “Parallel gzip,” http://zlib.net/pigz/, accessed: 2015-04-02.