How to Best Compress 3-D Measurement Data Under Given Guaranteed Accuracy
Total Page:16
File Type:pdf, Size:1020Kb
10 th IMEKO TC7 International Symposium June 30–July 2, 2004, Saint-Petersburg, Russia HOW TO BEST COMPRESS 3-D MEASUREMENT DATA UNDER GIVEN GUARANTEED ACCURACY Olga Kosheleva1; Sergio Cabrera1; Brian Usevitch1; Edward Vidal, Jr.2 1Electrical & Computer Engr., University of Texas, El Paso, USA 2Army Research Laboratory, While Sands Missile Range, New Mexico, USA Abstract. The existing image and data compression tech- niques try to minimize the mean square deviation between 1.3. 2-D data compression the original data f(x; y; z) and the compressed-decompressed In principle, it is possible to use these compression data fe(x; y; z). In many practical situations, reconstruction techniques to compress 2D measurement data as well. that only guaranteed mean square error over the data set is 1.4. Compressing 3-D data: layer-by-layer ap- unacceptable: for example, if we use the meteorological data proach to plan a best trajectory for a plane, what we really want to It is also possible to compress 3D measurement data know are the meteorological parameters such as wind, tem- perature, and pressure along the trajectory. If along this f(x; y; z) – e.g., meteorological measurements taken in line, the values are not reconstructed accurately enough, the different places (x; y) at different heights z. plane may crash – and the fact that on average, we get a One possibility is simply to apply the 2D JPEG2000 good reconstruction, does not help. What we need is a com- compression to each horizontal layer f(x; y; z0). (x; y) pression that guarantees that for each , the difference 1.5. Compressing 3-D data: an approach that uses e jf(x; y; z) ¡ f(x; y; z)j is bounded by a given value ¢ – KLT transform i.e., that the actual value f(x; y; z) belongs to the interval Another possibility, in accordance with Part 2 of [fe(x; y; z) ¡ ¢; fe(x; y; z) + ¢]. In this paper, we describe JPEG2000 standard, is to first apply the KLT transform new efficient techniques for data compression under such in- terval uncertainty. to each vertical line. Specifically, we: Keywords: data compression, guaranteed error bounds, ² compute the average value meteorological data 1 X f¹(z) = ¢ f(x; y; z) N 1. FORMULATION OF THE PROBLEM x;y of the analyzed quantity at a given height z, 1.1. Compression is important where N is the overall number of horizontal points At present, so much data is coming from measuring (x; y); instruments that it is necessary to compress this data before storing and processing. We can gain some stor- ² compute the covariances V (z1; z2) between differ- age space by using lossless compression, but often, this ent heights as gain is not sufficient, so we must use lossy compression 1 X as well. ¢ (f(x; y; z )¡f¹(z ))¢(f(x; y; z )¡f¹(z )); N 1 1 2 2 x;y 1.2. Successes of 2-D image compression In the last decades, there has been a great progress ² find the eigenvector ¸ and the eigenvectors e (z) in image and data compression. In particular, the k k of the covariance matrix V (z ; z ); we sort these JPEG2000 standard (see, e.g., [6]) uses the wavelet 1 2 eigenvalues into a sequence e (z); e (z);::: so transform methods together with other efficient com- 1 2 that j¸ j ¸ j¸ j ¸ :::; pression techniques to provide a very efficient compres- 1 2 sion of 2D images. ² finally, we represent the original 3D data values Within this standard, we can select different bitrates f(x; y; z) as a linear combination (i.e., number of bits per pixel that is required, on aver- X ¹ age, for the compressed image), and depending on the f(x; y; z) = f(z) + ak(x; y) ¢ ek(z) bitrate, get different degrees of compression. k When we select the highest possible bitrate, we get of the eigenvectors e (z). the lossless compressions that enables us to reconstruct k the original image precisely. When we decrease the As a result, we represent the original 3-D data as a se- bitrate, we get a lossy compression; the smaller the quence of the horizontal slices ak(x; y): bitrate, the more the compressed/decompressed image will differ from the original image. ² the first slice a1(x; y) corresponds to the main (1-st) eigenvalue; ² the second slice a2(x; y) corresponds to the next 1.10. What we need is data compression under in- (2-nd) eigenvalue; terval uncertainty In other words, what we need is a compression ² etc., that guarantees that for each (x; y), the difference e with the overall intensity decreasing from one slice to jf(x; y; z) ¡ f(x; y; z)j is bounded by a given value ¢ the next one. – i.e., that the actual value f(x; y; z) belongs to the in- e e Next, to each of these slices ak(x; y), we apply a 2D terval [f(x; y; z) ¡ ¢; f(x; y; z) + ¢]. JPEG2000 compression with the appropriate bit rate bk There exist several compressions that provide such depending on the slice k. a guarantee. For example, if for each slice, we use the largest possible bitrate – corresponding to lossless com- 1.6. Decompressing 3-D data: KLT-based approach pression – then ea (x; y) = a (x; y) hence fe(x; y; z) = To reconstruct the data, we so the following: k k f(x; y; z) – i.e., there is no distortion at all. ² First, we apply JPEG2000 decompression to each What we really want is, among all possible com- slice; as a result, we get the values ea[bk](x; y). pression schemes that guarantee the given upper bound k ¢ on the compression/decompression error, to find the ² Second, based on these reconstructed slices, we scheme for which the average bitrate now reconstruct the original 3-D data data as 1 X X b def= ¢ b [b ] k e ¹ k Nz f(x; y; z) = f(z) + eak (x; y) ¢ ek(z): k k is the smallest possible. In some cases, the bandwidth is limited, i.e., we 1.7. This approach is tailored towards image pro- know the largest possible average bitrate b0. In such cessing – and towards Mean Square Error cases, among all compression schemes with b · The problem with this approach is that for com- 1 b0, we must find a one for which the L compres- pressing measurement data, we use image compression sion/decompression error is the smallest possible. techniques. The main objective of image compression is to retain the quality of the image. From the view- 1.11. What we have done point of visual image quality, the image distortion can In this paper, we describe new efficient (suboptimal) be reasonably well described by the mean square dif- techniques for data compression under such interval un- ference MSE (a.k.a. L2-norm) between the original im- certainty. age I(x; y) and the compressed-decompressed image 2. NEW TECHNIQUE: MAIN IDEAS, Ie(x; y). As a result, sometimes, under the L2-optimal DESCRIPTION, RESULTS compression, an image may be vastly distorted at some points (x; y) – and this is OK as long as the overall 2.1. What exactly we do mean square error is small. Specifically, we have developed a new algorithm 1.8. For data compression, MSE may be a bad cri- that uses JPEG2000 to compress 3D measurement data terion with guaranteed accuracy. We are following the general When we compress measurement results, however, idea of Part 2 of JPEG2000 standard; our main con- our objective is to be able to reproduce each individual tribution is designing an algorithm that selects bitrates measurement result with a certain guaranteed accuracy. leading to a minimization of L1 norm as opposed to In such a case, reconstruction that only guaranteed the usual L2-norm. mean square error over the data set is unacceptable: for 2.2. Let us start our analysis with a 2-D case example, if we use the meteorological data to plan a Before we describe how to compress 3-D data, let best trajectory for a plane, what we really want to know us consider a simpler case of compressing 2-D data are the meteorological parameters such as wind, tem- f(x; y). In this case, for each bitrate b, we can apply perature, and pressure along the trajectory. the JPEG2000 compression algorithm corresponding to If along this line, the values are not reconstructed this bitrate value. After compressing/decompressing accurately enough, the plane may crash – and the fact the 2-D data, we get the values fe[b](x; y) which are, that on average, we get a good reconstruction, does not in general, slightly different from the original values help. f(x; y). 1.9. An appropriate criterion for data compression In the interval approach, we are interested in the L1 What we need is a compression that guarantees the error given accuracy for all pixels, i.e., that guarantees that ¯ ¯ def ¯ [b] ¯ 1 e D(b) = max ¯fe (x; y) ¡ f(x; y)¯ : the L -norm max jf(x; y; z) ¡ f(x; y; z)j is small. x;y x;y;z The larger the bitrate b, the smaller the error D(b). When the bitrate is high enough – so high that we can transmit all the data without any compression – the er- Since, in general, D(b1;:::) · De(b1;:::), the re- ror D(b) becomes 0. sulting allocation is only suboptimal with respect to Our objective is to find the smallest value b for D(b1;:::).