03-082.qxd 1/11/05 5:17 PM Page 157
Effects of JPEG2000 on the Information and Geometry Content of Aerial Photo Compression
Jung-Kuan Liu, Houn-Chien Wu, and Tian-Yuan Shih
Abstract evaluated the effects of compression on geometric accuracy. Li The standardization effort of the next ISO standard for et al., (2002) indicated that when compression ratios are less compression of the still image, JPEG2000, has recently than a factor of 10, the compressed image is near-lossless with reached International Standard (IS) status. This wavelet- JPEG. In other words, the visual quality of JPEG compressed based standard outperforms the Discrete Cosine Transform images remains excellent and the accuracy of manual image (DCT) based JPEG in terms of compression ratio, as well mensuration is, essentially, not influenced. Paola et al. (1995) as, quality. In this study, the performance of JPEG2000 is and Schmanske and Loew (2001) concentrated on the classifi- evaluated for aerial image compressions. Different com- cation accuracies of compressed images. Paola et al. (1995) pression ratios are applied to scanned aerial photos at the revealed that high quality classifications could be obtained for 1:5 000 scale. Both the image quality measurements and the images with JPEG compression ratios approaching 10:1 or even accuracy of photogrammetric point determination aspects higher. The classification result retains its overall appearance, are examined. The evaluation of image quality is based but the smoothing effect of high compression tends to elimi- on visual analysis of the objects in the scene and on the nate much of the pixel-to-pixel detail. Robinson et al. (1995) computation of numerical indices, including RMSE, entropy, and Rane & Sapiro (2001) evaluated the effect of image com- and Peak Signal-to-Noise Ratio (PSNR). The geometric quality pression on Digital Surface Model (DSM) generation. Maeder of JPEG2000 with different compression ratios is studied for (1998) investigated the compression effects on digital image some photogrammetric operations, including interior matching. Pal et al. (2002) measured the performance of orientation, relative orientation, absolute orientation, and JPEG2000 with various feature extractions and classifications of DSM generation. The objective of this study is to explore the hyperspectral images. Lee et al. (2002) reported on a combined possibility of JPEG2000 for replacing JPEG as a standard in JPEG2000 and spectral correlation method for hyperspectral photogrammetric operations. image compression. Santa-Cruz et al. (2002) compared JPEG2000 with JPEG-LS, MPEG-4 VTC, JPEG, PNG, and SPIHT. Currently, the most common form used for still image Introduction compressions is JPEG. This standard was developed by the A common characteristic of most images is that their neigh- Joint Photographic Expert Group in the late 1980s, and since boring pixels are correlated, and therefore, contain redundant then, has been the most successful and widely used image information. The foremost task then, is to find less-correlated compression technique. JPEG has been implemented by most representation of the image. Two fundamental components of the digital photogrammetric workstations (DPW) currently of compression are the reduction of both redundancy and available commercially (Li et al., 2002). However, in 1997, irrelevancy. Redundancy reduction aims at removing dupli- the JPEG Committee decided that the needs and requirements cation from the signal source (image/video). Irrelevancy of imagery applications in today’s world point to the need reduction omits parts of the signal that will be noticed by the for a new standard. This proposition brought forth the new signal receiver (Saha, 2000). Image compression research standard JPEG2000. aims at reducing the number of bits needed to represent an In a working environment, the performance of JPEG2000 image, which increases the compression ratio, by removing is evaluated for aerial image compression. Image quality the redundancies as much as possible. measurements and the accuracy of photogrammetric point Digital image compression is not only important for trans- determination (PPD) are two primary aspects to consider mission on the Internet, but also in the workflows of digital when evaluating the JPEG2000. Different compression ratios photogrammetry. For example, a standard 23 cm 23 cm are applied to scanned aerial photos of 1:5 000 scale. First, panchromatic aerial photo scanned at 20 m resolution the evaluation of image quality is based on visual analysis produces a file size of about 140 MB. The total amount of of the objects in the scene, with several indices computed storage for imagery will reach several gigabytes for an individ- with different ratios for both JPEG2000 and JPEG, e.g., RMSE, ual photogrammetric mission. Image compression has been entropy, and Peak Signal to Noise Ratio (PSNR). Second, the one important issue in photogrammetry and remote sensing in geometric quality of JPEG2000 with different compression recent years. Schiewe (1998) investigated the effect of lossy ratios is studied JPEG compression techniques on geometry and information content of satellite imagery (MOMS-02). Li et al., (2002) Photogrammetric Engineering & Remote Sensing Vol. 71, No. 2, February 2005, pp. 157–167. Department of Civil Engineering, National Chiao-Tung 0099-1112/05/7102–0157/$3.00/0 University, 1001 Ta-Hsueh Road, Hsin-Chu, Taiwan © 2005 American Society for Photogrammetry ([email protected]). and Remote Sensing
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING February 2005 157 03-082.qxd 1/11/05 5:17 PM Page 158
Overview of JPEG and JPEG2000 Additionally, JPEG2000 offers higher compression ratios for lossy compression than JPEG. According to Christopopulos JPEG et al. (2000), JPEG2000 can typically compress images any- There are several modes defined for JPEG, including baseline, where from 20 percent to 200 percent more efficiently than lossless, progressive and hierarchical. The baseline and JPEG can based on the same PSNR values. Compression effi- progress encoding methods are Discrete Cosine Transform ciency for lossy compression is usually measured based on (DCT) based on a predictive method. The hierarchical mode signal-to-noise ratios, i.e., PSNR, which will be described in encodes the image at multiple spatial resolutions using the next section. Furthermore, JPEG2000 is also able to sup- either the DCT-based compression or the lossless mode port compression ratios of about 2.5:1 for lossless compression (Wallace 1992; Li et al., 2002). (Aboufadel, 2001). The baseline mode is the most popular one and sup- Moreover, JPEG2000 is able to display images at different ports lossy coding only. In the baseline mode, the image is resolutions and sizes from the same image file, while its divided into 8 8 blocks, and each of these is transformed predecessor JPEG was only able to display images at a set by the DCT. The transformed blocks’ coefficients are quan- resolution. Since JPEG2000 is based on wavelets, the wavelet tized with a uniform scalar quantizer, zig-zag scanned, and stream can be only partially decompressed if the user only entropy coded with the Huffman coding. The quantization wants a low-resolution image, while the full resolution step size for each of the 64 DCT coefficients is specified in a image can be viewed when desired (Aboufadel, 2001). One quantization table, which remains the same for all blocks. of the major benefits of being able to access an image at The DC coefficients of all blocks are coded separately using a different resolutions is the ability to use only the amount of predictive scheme. Hereafter, this mode is simply referred to bandwidth required for any given particular level of interest. as JPEG (Santa-Cruz et al., 2002). For example, when clicking on a JPEG2000 image that has been set up for progressive-by-resolution access, the viewer JPEG2000 will be able to see the low-resolution version as soon as it JPEG2000 is a newly approved image compression stan- downloads, and can then, immediately, decide whether or dard that is intended to replace the existing JPEG standard. not to wait for higher resolution. The JPEG2000 standard is comprised of a number of parts, Another advantage of JPEG2000 is its Region of Interest including the core coding system, extension, motion (ROI) capability. The wavelets can select a particular portion JPEG2000, conformance testing, and reference software of an image to view at a high quality while leaving the rest (Adams, 2001c). In this study, the experiments are res- of the image at a lower quality. The last aspect of JPEG2000 tricted to only Part 1 of the standard, which defines the that is superior to JPEG is in the area of error resilience core system. (Adams, 2001c). Error resilience measures the ability of a JPEG2000 is based on the Discrete Wavelet Transfor- compression method to avoid letting errors introduced into mation (DWT), scalar quantization, context modeling, the image file affect the quality of the image. JPEG2000 offers arithmetic coding, and post-compression rate allocation. It significantly higher error resilience than JPEG. Therefore, handles both lossy and lossless compression using the same there is less chance that the image will be somehow cor- transform-based framework, and borrows heavily on ideas rupted and its quality sacrificed in some way. from embedded block coding with an optimized truncation (EBCOT) scheme. Quantization allows greater compression to be achieved, by representing transform coefficients with Indices of Image Quality only the minimal precision required, to obtain the desired Gonzalez and Woods (1992) stated that there are two fidelity level of image quality. Transform coefficients are quantized criteria for image processing: the objective and the subjec- using scalar quantization with a deadzone. A different tive. The subjective fidelity criteria are established based quantizer is employed for the coefficient of each sub-band, on evaluation ratings from human observers. While visual and each quantizer has only one parameter: its step size inspection is the most prominent method of subjective eval- (Adams, 2001c). Each sub-band is divided into rectangular uation, there are a number of numerical indices for objective blocks (called codeblocks in JPEG2000), typically 64 coeffi- measurements that area also used (Chen, 2001). Consider a cients wide and 64 coefficients tall, and entropy coded discrete image f(x,y) for x 1,2, ....,N and y 1,2, . . . .,M, using context modeling and bit-plane arithmetic coding. The which is regarded as a reference image, and consider a second coded data is organized in so-called layers, which act as image fˆ(x,y), of the same spatial dimensions as f(x,y), that is quality levels, using the post-compression rate allocation to be compared to the reference image. to output to the codestream in packets (Santa-Cruz et al., 2002). • Mean Square Error (MSE): JPEG2000 also supports a number of functionalities, many M 1 N 1 1 ˆ 2 2 of which are inherent from the algorithm itself. Some of erms a a [ f (x,y) f (x,y)] se . (1) MN these functionalities which outperform JPEG are described in x 0 y 0 the next section. A smaller MSE value indicates higher reconstruction fidelity. One problem with mean-square error is that it depends Comparison Between JPEG and JPEG2000 strongly on image intensity scaling. For example, an MSE of Inheriting the functionalities from coding algorithms, 100.0 for an 8-bit image looks dreadful; but an MSE of 100.0 JPEG2000 offers numerous advantages over the old JPEG for a 10-bit image is barely noticeable (Veldhuizen,1998). standard (Aboufadel, 2001; Adams, 2001c). Fundamentally, • Root Mean Square Error (RMSE): JPEG2000 supports both lossy and lossless compression of M 1 N 1 1 ˆ 2 2 single-component (e.g., grayscale) and multi-component erms a a [f(x,y) f (x,y)] se . (2) BMN (e.g., color) imagery. JPEG has a lossless compression engine x 0 y 0 (i.e., JPEG-LS), but the algorithm is independent of the DCT • Peak Signal to Noise Ratio (PSNR): algorithm used by the lossy engine, and is not used very (peck-to-peak value of the referenced image)2 often. Based on the wavelet techniques of compression, # PSNR 10 log10 2 . JPEG2000 provides a single scheme for both lossless and lossy se compressions. (3)
158 February 2005 PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING 03-082.qxd 1/11/05 5:17 PM Page 159
For a 256 level gray scale image, PSNR is defined as: the original image size by the compressed image size (Tai, 2001). The second is computing the bits per pixel after the 2 # 255 compression. Because the first is adopted as the input para- PSNR 10 log10 . (4) MSE meter in JasPER (Adams, 2001b.), the software utilized in this paper, bits per pixel is the parameter used in this study: PSNR represents radiometric degradation of the reconstructed image. A larger PSNR value indicates higher reconstruction The number of bits of the original image fidelity. Peak Signal-to-Noise Ratio (PSNR) avoids the intensity Ratio . (7) scaling problem that MSE does by scaling the MSE according The number of bits of the compressed image to the image range. For example, a MSE of 100.0 for an 8-bit image will have a PSNR of 28.1; and a MSE of 100.0 for a 10-bit image will have a PSNR of 40.2. The Experiments and the Analysis • Entropy: Each experiment has two parts. One is the computation of M 1 N 1 image quality indices proposed in the previous section; the Entropy a a P(x,y)logP(x,y), (5) other is the accuracy of photogrammetric point determina- x 0 y 0 tion. The experiments are conducted on a PC with a 2.4 GHZ where P(x,y) denotes the probability of the occurrence of the Pentium IV processor and 1024 MB of RAM (DDRRAM). pixel (x,y). The images used for these experiments are scanned from The average information per source output is called the uncertainty or entropy of the source. Entropy defines the 1:5 000 scale color aerial photos at a 20 m resolution. As average amount of information obtained by observing a shown in Figure 1, a stereo pair covering the Hsinchu area of single source output. A larger entropy value indicates more Taiwan is applied for the experiment. The area covers two uncertainty and, thus, more information is associated with main land-cover types i.e., urban area and agricultural land. the source. Although the original photo was taken in color, only the • Fidelity: grayscale image was scanned and stored as TIFF files. The m n m n typical dimension of a scanned image file is 11995 11905 # ¿ 2 Fidelity a a (gij gij ) a a g ij , (6) i 1 j 1 n i 1 j 1 pixels. All these images have a depth of 8 bits per pixel. The JasPER1.5 software was is used in this study to where gij and g ij are the gray values of the original and perform image compression with JPEG2000. JasPER is a collec- reconstructed images, respectively. Fidelity is the similarity tion of software (i.e., a library and application programs) used between the original and the reconstructed image, i.e., it is a for the coding and manipulation of images. This software measure of the geometric distortion of the reconstructed image (Li et al., 2002). is written in the C programming language. JasPER is freely available for academic uses and supports various image In lossless compressions, the fidelity is 1.0 and the PSNR formats, including .BMP (Windows Bitmap), .JPZ (JPEG2000), .JPG is infinite. The compression can be regarded as near-lossless (JPEG), .PNM (PNM/PGM/PPM), and .RAS (Sun Rasterfile) (Adams, compression when fidelity is more than 0.99 and the PSNR is 2001a, b). The current version of JasPER software is currently higher than 42.0 (Li et al., 2002). available for downloading from the JasPER project home page There are two commonly applied parameters for des- (http://www.ece.uvic.ca/ mdadams/jasper/; last date accessed: cribing the image compression ratio. The first is to divide 05 November 2004).
Figure 1. A stereo pair of aerial photographs as test images.
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING February 2005 159 03-082.qxd 1/11/05 5:17 PM Page 160
Visual Evaluation the fidelity value should not be larger than 1.0. Some of the As shown in Figure 2a, eleven different compression ratios fidelity values for JPEG images set against higher compres- ranging from 1 to 100 are performed in the experiments sion ratios are above 1.0. The reason for this is the elimina- (only 8 of them are shown). The JPEG example is also tion effect that destroys high frequency structures for JPEG. presented for comparison. For visual evaluation, three dif- To begin, Figure 4a shows the relationship between ferent objects, the road (p1), the building corner (p2), and a compression ratio and RMSE for a stereo-pair. Where JP2_L is signalized target (p3), are taken as samples and listed in the image on the left that is compressed with JPEG2000, and Figure 2a. JP2_R is the image on the right. JPEG_L is the image on the The image quality of both JPEG2000 and JPEG are compa- left that is compressed with JPEG, and JPEG_R is the image on rable up to a compression ratio 25. The images compressed the right. Because the meaning of RMSE and MSE are similar, in the JPEG format appear more blurry than JPEG2000 when only the RMSE graph is presented. compression ratio is higher than 30. This blurriness will From Figure 4a, JPEG2000 has a smaller RMSE value than likely cause errors for point measurements, such as fiducial JPEG at the same compression ratio. This trend indicates that marks, ground control points, tie points, and terrain eleva- the difference becomes more significant between the two tion points during the photogrammetric process. when the compression ratio becomes larger. In conclusion, Figure 3 shows the zoning effect for JPEG with a com- JPEG2000 has higher reconstruction fidelity than JPEG. pression ratio of 50. For manual measurement, this may Next, the relationship between compression ratio and cause pointing problems. On the other hand, the JPEG2000 entropy is shown in Figure 4b. As depicted in Figure 4b, the compression at the same rate shows a smooth image that increase in compression ratio for JPEG2000 has no effect on still allows for a sufficient interpretation of targets. the decrease of the entropy value, up to a compression ratio of 100. However, the entropy does drop with images that are Computation of Indices compressed using JPEG. Although entropy provides a meas- Five traditional numerical indices are computed and listed ure of information in the sense of the shortest average des- in Table 1 and Table 2 for left and right images, respec- cription of a set of data, this seemingly indicates that JPEG2000 tively. It takes a while to compute these indices because the preserves the information quantity better than JPEG, especially entire image (11995 11905 pixels) is used. Theoretically, at higher compression ratios.
(a)
(b) Figure 2. The visual effect and the position difference versus the compression ratio: (a) The visual effect versus compression ratio. (b) Position difference between original and JPEG2000 compressed images versus compression ratio.
160 February 2005 PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING 03-082.qxd 1/11/05 5:17 PM Page 161
Figure 3. Comparison between original and compressed imagery (with compression ratio 50).
TABLE 1. IMAGE QUALITY OF LEFT IMAGE
Compression Ratio 1 2 5 7.7 11.1 13.3 15.6 20 25 33.3 50 100
PSNR_JPEG2000 N/A 66.29 44.38 41.57 39.75 38.60 37.92 37.11 36.10 34.93 33.41 30.90 PSNR_JPEG N/A 44.70 42.80 40.93 39.12 38.27 37.42 36.22 35.20 33.71 31.64 28.31 RMSE_JPEG2000 N/A 0.12 1.54 2.13 2.62 3.00 3.24 3.56 3.99 4.57 5.44 7.27 RMSE_JPEG N/A 1.48 1.85 2.29 2.82 3.11 3.43 3.94 4.43 5.26 6.67 9.80 Entropy_JPEG2000 7.719 7.641 7.718 7.714 7.710 7.710 7.707 7.704 7.700 7.694 7.691 7.662 Entropy_JPEG 7.719 7.685 7.672 7.666 7.616 7.592 7.550 7.485 7.421 7.209 6.776 5.166 Fidelity_JPEG2000 N/A 1.000 1.000 0.999 0.999 0.998 0.998 0.998 0.997 0.998 0.997 0.997 Fidelity_JPEG N/A 0.999 0.999 0.999 0.999 1.000 1.000 0.999 0.999 0.999 1.000 1.000
TABLE 2. IMAGE QUALITY OF RIGHT IMAGE
Compression Ratio 1 2 5 7.7 11.1 13.3 15.6 20 25 33.3 50 100
PSNR_JPEG2000 N/A 67.01 44.13 41.25 39.26 38.19 37.54 36.76 35.80 34.71 33.48 31.20 PSNR_JPEG N/A 45.01 42.56 40.41 38.72 37.88 37.04 35.88 34.91 33.49 31.92 28.36 RMSE_JPEG2000 N/A 0.11 1.58 2.21 2.78 3.14 3.38 3.70 4.14 4.69 5.40 7.02 RMSE_JPEG N/A 1.43 1.90 2.43 2.96 3.26 3.58 4.10 4.58 5.40 6.47 9.74 Entropy_JPEG2000 7.670 7.584 7.666 7.662 7.659 7.655 7.653 7.650 7.650 7.643 7.642 7.625 Entropy_JPEG 7.670 7.733 7.723 7.720 7.668 7.638 7.595 7.525 7.457 7.257 6.723 5.412 Fidelity_JPEG2000 N/A 1.000 1.000 0.999 0.999 0.999 0.999 0.999 0.998 0.998 0.998 0.998 Fidelity_JPEG N/A 0.999 0.999 0.999 0.999 0.999 1.000 1.000 1.000 0.999 0.999 1.000
Figure 4c also shows that JPEG2000 always has better reference. The coordinates of the same points, determined PSNR values than JPEG for the same compression ratio. The by JPEG2000 compressed at a large spectrum of ratios, were difference becomes more significant when the compression then compared with the reference. The RMSE values are ratio is higher than 30. used to indicate the quality of PPD. This procedure follows Li et al. (2002). The Accuracy of Photogrammetric Point Determination The first step in PPD is interior orientation (IO). The IO is Besides the visual inspection and the evaluation with used to establish the relationship between the pixel and the numerical indices, Li et al. (2002) discussed an evaluation image coordinate system. The operator manually measures scheme designed for the geometric accuracy of point posi- eight fiducial marks for each image using a variety of tioning. The PCI Geomatics OrthoEngine® Module 8.2.3 (PCI, compression ratios, and the effect of JPEG2000 compression 2002) is used for evaluating the accuracy of photogrammetric on the accuracy of IO is shown in Table 3. Table 3 roughly point determination (PPD). Because PCI Geomatics does not indicates that the higher the compression ratio, the poorer support JPEG2000 files in this version, the JPEG2000 files are the accuracy of the IO. The RMSE of the IO is less than 5 m first converted into TIFF format with IrfanView (IrfanView, (0.25 pixels) with a compression ratio of up to 100. The 2003). range of RMSE values used in our study is similar to those Since this part of the experiment focuses on how described in the report by Lue (1997) that is based on the JPEG2000 compression affects PPD, the coordinates of the automatic interior orientation system. This phenomenon is points determined by the original images serve as the explained by the fact that, although the image of fiducial
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING February 2005 161 03-082.qxd 1/11/05 5:17 PM Page 162
(a)
(b)
(c) Figure 4. Indices of image quality versus compression ratio: (a) RMSE versus com- pression ratio. (b) Entropy versus compression ratio. (c) PSNR versus compression ratio.
marks become blurrier with an increase in the compression Next, the relative orientation (RO) process, i.e., the deter- ratio, the central mark can still be recognized and accurately mination of relative position and attitude of two images located with the regular “cross” shape. with respect to each other, is studied. The accuracy of RO is
162 February 2005 PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING 03-082.qxd 1/11/05 5:17 PM Page 163
TABLE 3. ACCURACY OF INTERIOR ORIENTATION VERSUS COMPRESSION RATIO
Compression Ratio 1.0 2.0 5.0 7.7 11.1 13.3 15.6 20.0 25.0 33.3 50.0 100.0
JP2_L( m) 2.75 2.65 3.38 3.16 4.34 3.87 3.09 3.07 3.28 3.77 4.23 4.55 JP2_R( m) 3.25 3.13 3.96 2.48 4.32 3.98 4.07 2.88 3.93 3.81 4.00 4.93
determined by the RMSE of all y-parallax residuals. The sample points is shown in Figure 2b. In addition, the rela- operator manually measures all control points and tie tionship of RMSE values of RO versus compression ratios is points. These control points and tie points (eighteen in all) illustrated in Figure 5a. are composed of signalized points on buildings, corner Figure 5a indicates that the accuracy of RO decreases points of some building tops, and some identifiable object when the compression ratio increases. Therefore, the increases points. A sample of these points is shown in Figure 2a. in RMSE value are a consequence of the change in the posi- The effect of JPEG2000 compression on the position of these tions of the feature points used for relative orientation.
(a)
(b)
Figure 5. Accuracy of RO/AO versus compression ratio: (a) Accuracy of relative orientation versus compression ratio. (b) Accuracy of absolute orientation ver- sus compression ratio.
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING February 2005 163 03-082.qxd 1/11/05 5:17 PM Page 164
TABLE 4. THE COMPARISON OF THIS STUDY AND LI ET AL. (2002) BASED ON many differences between the RMSE values of RO and AO RO AND AO for JPEG2000 and JPEG images at the same compression ratio. Therefore, the reason for the lower accuracy of RO Li, et al. (2002) This Study and AO in this study may be a result of the method of Image format JPEG JPEG2000 point measurement. In Li et al. (2002), the measurement Scanned pixel size 25 m 20 m is performed stereoscopically, while in this study, meas- Compression ratio 2 25 2 25 urements are done in the mono-scopic way. Besides, the Accuracy of RO( m) 1.70 3.80 2.0 4.1 quality of original images, accuracy of GCPs, photo scale, Accuracy of AO(m)-Horizontal 0.149 0.316 0.196 0.378 and processing software will also play essential roles in Accuracy of AO(m)-Vertical 0.103 0.347 0.280 0.345 determining the accuracy of RO and AO.
The Effects of JPEG2000 on Automated DSM Generation The last part of the experiment is designed to evaluate TABLE 5. COMPARISON OF PERFORMANCE WITH JPEG2000 AND JPEG BASED the effects of JPEG2000 compression on an automated DSM ON RO AND AO generation. Here, the automated DSM generation is based 2 11.1 20 on image matching techniques. There are three well- known matching methods, i.e., area-based matching, AO(m) AO(m) AO(m) feature-based matching, and relational (symbolic) match- Compression RO RO RO ing. The relationship between these three matching meth- Ratio ( m) XY Z ( m) XY Z ( m) XY Z ods and matching entities is depicted as Table 6 (Schenk, 1999). JPEG2000 2.79 0.18 0.25 2.92 0.24 0.32 3.78 0.32 0.32 In this part of the experiment, the parameters of exterior JPEG 2.03 0.22 0.28 3.67 0.25 0.33 3.87 0.36 0.34 orientation are obtained from an aero-triangulation with PAT-B® (Inpho Gmbh, 2002). For JPEG2000 and JPEG images using different compression ratios, the same parameters are applied, in order to remove the effect of manual measure- Next, the stereo model is registered to the ground by the ment. In the PCI Geomatics OrthoEngine® Module, the opera- absolute orientation (AO) procedure. This operation is ful- tor can automatically derive DSM after generating epipolar filled by using three or more ground control points that are images with known exterior orientations. The density of the well distributed over the model. The accuracy of AO is based point matching for the experiment is set to 2 pixels, about on the RMSE values of 3D coordinates on these control points. 0.2 m on the ground. The reports from DSM generation are The effect of JPEG2000 compression on the accuracy of AO is generated as the process complete, allowing the DSM correla- shown in Figure 5b. The XY_m and Z_m stand for the RMSE tion success percentage shown in Table 7 to be extracted value of AO in horizontal position and height, respectively, from these reports. in Figure 5b. There are five ground control points used in Table 7 shows the effects of JPEG2000 compression on this experiment to compute the RMSE of AO. From this figure, the DSM correlation success percentages. Where, JP2_3334 is the trend shows that the RMSE values of AO increase when the first stereo model with JPEG2000, and JP2_3435 is the the compression ratio increases, and that the RMSE in height second stereo model with JPEG2000. It is known that JPEG is larger than that in horizontal position. images cause severe destruction when compression ratios A comparison to Li et al. (2002) is shown in Table 4. are larger than 32, i.e., less than 0.25 bpp (Tai, 2001). Here, To have consistent compression ratio with Li et al. (2002), larger JPEG compression ratios are used only for comparison the compression ratio is limited from 2 to 25. Although with JPEG2000 at the same ratio. The DSM correlation success the scanned pixel size is smaller in this study, the RMSE percentage of Model_3334 is higher than that of Model_3435 values for both relative orientation and absolute orienta- for both of JPEG and JPEG2000. This phenomenon is due to tion are higher than Li et al. (2002). To make sure the more man made features existing on Model_3334 than on higher RMSE values of RO and AO in the current study are Model_3435. not coming from JPEG2000, the same procedures of mensu- Before further evaluation of the DSM generation, a brief ration are conducted on JPEG images with three different description of matching principles that PCI software uses compression ratios. As shown in Table 5, there are not follows. An area-based automated image matching method is applied to extract the elevation parallax, and produces the DSM through a comparison of the respective gray values TABLE 6. THE RELATIONSHIP BETWEEN MATCHING METHODS on each of these images. This procedure utilizes a mean AND MATCHING ENTITIES normalized cross-correlation matching method with a multi- scale strategy to match the images using the statistics col- Matching Method Similarity Measure Matching Entities lected in defined windows (called image patches). Matching is performed by considering the neighborhood surrounding a area-based correlation, least-squares gray levels given pixel in the left quasi-epipolar image (thus forming a feature-based cost function edges, region symbolic cost function symbolic description template) and moving this template within a search area to the right epipolar image, until a position is reached that
TABLE 7. DSM CORRELATION SUCCESS PERCENTAGE VERSUS COMPRESSION RATIO
Compression Ratio 1.0 2.0 5.0 7.7 11.1 13.3 15.6 20.0 25.0 33.3 50.0 100.0
JP2_3334 75.1 75.1 77.4 79.1 80.5 82.4 83.3 83.9 85.4 87.4 88.2 89.8 JP2_3435 68.8 68.8 71.0 72.8 74.0 76.0 76.9 77.2 78.7 81.9 82.7 85.2 JPEG_3334 75.1 76.5 77.4 77.5 78.7 79.7 80.8 81.4 82.8 82.9 81.0 71.5 JPEG_3435 68.8 70.3 70.9 70.6 72.2 72.9 74.3 74.8 76.5 77.0 75.6 66.0
164 February 2005 PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING 03-082.qxd 1/11/05 5:17 PM Page 165
gives the best match (Mader, 1998; Schenk, 1999; Hijazi, principles of PCI software, one reason for the improvement 2002). of match percentage may be the smoothing effect. To prove The actual matching procedure employed with PCI this assumption, the image is applied with 5 5 mean software generates correlation coefficients between 0 and 1 filters to both the original left and right images, respectively, for each match pixel, where 0 represents a complete mis- resulting in a 75 percent to 85 percent rise in the DSM cor- match and 1 represents a perfect match. A second-order relation success rate. On the other hand, can the percentage surface is then fitted around the maximum correlation of pixels that are correctly matched (PPM) represent the coefficients to find the match position of sub-pixel accu- matching accuracy? According to Maeder’s (1998) study, racy. The difference in location between the center of the both the mean absolute difference (MAD) between the dis- template and the best-matched pixel position gives the dis- tance function values at all matched locations as well as PPM parity or parallax arising from terrain relief. From which, are applied. This might be a more reasonable solution for the absolute elevation value is then computed (Hijazi, evaluating the matching accuracy. 2002). The effect of the DSM correlation success percentage The image content affects the matching accuracy greatly, might be that which is presented in Figure 6. Inspecting the and the average error in the matching accuracy grows two oval areas, one may conclude that the higher the DSM rapidly as image quality decreases (Maeder, 1998). Conse- correlation success percentage, the more pixels that are quently, the DSM correlation success percentage should be successfully matched. This is a positive influence for descending while compression ratio rises, since the image orthophoto production. If the success rate is higher, the DSM quality decreased as the compression ratio increased. On the points are distributed more evenly. Then, one can derive a contrary, reverse results are obtained. Table 7 shows that for higher quality orthophoto with a higher compression ratio JPEG2000, the increase for DSM correlation success percentage for DSM matching. But, regardless of whether the accuracy of is almost linear as compression ratio increases. Similar to DSM rises as the DSM correlation success percentage increases JPEG2000, the correlation success percentage is also increase or not, further investigation is necessary. with the compression ratio with JPEG compressed images up Another interesting issue is the relationship between the to the compression ratio of 33.3. It is known that smoothing running time for DSM matching and the JPEG2000 compres- effects remove high-frequency structures on wavelet-based sion ratio. Table 8 shows that for JPEG2000 compression, the compression (Schiewe, 1998). Based on the DSM matching decrease in running time is almost linear with an increase in
Figure 6. The orthophoto versus compression ratio.
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING February 2005 165 03-082.qxd 1/11/05 5:17 PM Page 166
TABLE 8. RUNNING TIME (MINUTES) OF DSM AUTO MATCHING VERSUS COMPRESSION RATIO
Compression Ratio 1.0 2.0 5.0 7.7 11.1 13.3 15.6 20.0 25.0 33.3 50.0 100.0
JP2_3334 239 223 228 198 185 182 176 173 170 165 165 152 JP2_3435 210 213 237 201 188 176 172 167 164 150 146 150 JPEG_3334 239 206 217 240 182 208 186 212 192 191 196 191 JPEG_3435 210 197 221 232 183 198 187 226 202 176 229 210
the compression ratio. But for JPEG compression, the running Christopoulos, C., A. Skodras, and T. Ebrahimi, 2000. The JPEG2000 time for DSM matching shows no correlation with the com- Still Image Coding: An Overview, IEEE Transactions on Con- pression ratio in Table 8. sumer Electronics, Vol. 46, No. 4, pp. 1103–1127. Gonzalez, R.C., and R.E. Woord, 1992. Digital Image Processing, Addison-Wesley. Conclusions Hijazi, J., 2002. Elevation Extraction from Satellite Data using PCI Examinations regarding effects on the indices of image quality Software, PCI Technical Paper Site (URL:http://www.pcigeomat- and geometrical accuracy have revealed that, up to a com- ics.com/tech-papers/techpapers_main.php; last date accessed: 05 November 2004). pression rate of 13.3, both JPEG and JPEG2000 are sufficient for most photogrammetric applications when the manual Inpho Gmbh, 2002 (URL:http://www.inpho.de/; last date accessed: processing method is used. The experimental results also 05 November 2004). IrfanView, 2003 (URL:http://www.irfanview.com/; last date accessed: indicate that at up to a compression ratio of 30, JPEG2000 still can be used with less restrictive demands. Furthermore, the 05 November 2004). Lee, H.S., N.H. Younan, and R.L. King, 2002. Hyperspectral Image compressed algorithm based on DWT (JPEG2000) is clearly Cube Compression Combining JPEG-2000 and Spectral Decorre- superior against the DCT (JPEG) method, especially for higher lation, IEEE International Geoscience and Remote Sensing compression ratios (larger than 50). Finally, the study of the Symposium (IGARSS), Vol.6, pp. 3317–3319. DSM automated generation demonstrated that automatic pro- Li, Z., X. Yuan, and K. Lam, 2002. Effects of JPEG Compression on cedures are more negatively influenced by lossy compression the Accuracy of Photogrammetry Point Determination, Pho- operations. togrammetric Engineering & Remote Sensing, Vol. 68, No. 8, Summarizing these results, the wavelet-based JPEG2000 is pp. 847–853. generally suitable for applications of photogrammetry and Lue, Y., 1997. One Step to a Higher Level of Automation for remote sensing. JPEG2000 enables not only a higher rate of data Softcopy Photogrammetry Automated Interior Orientation, to be compressed, but also a more flexible, multi-resolution ISPRS Journal of Photogrammetry and Remote Sensing, 52(3): offering to users. 103–109. There is no doubt that JPEG2000 will eventually be the Maeder, A.J., 1998. Lossy Compression Effects on Digital Image next international standard for image compression. While Matching, Proceedings of 14th International Conference on JPEG has been applied to aerial photo compression in various Pattern Recognition, Vol. 2, pp. 1626–1629. digital photogrammetric works, there is no softcopy stereo- Pal, M.D., C.M. Brislawn, and S.P. Brumby, 2002. Feature Extraction plotter or digital photogrammetric workstation supporting from Hyperspectral Images Compressed Using the JPEG-2000 th JPEG2000 which is likely to change. Standard, Proceedings of 5 IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI’02), pp. 168–172. Paola, J.D., and R.A. Schowengerdt, 1995. The Effect of Lossy Com- Acknowledgments pression on Image Classification, IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Vol.1, pp. 118–120. The authors wish to express their sincere thanks to Prof. Jen-Hui Chuang, Department of Computer Information PCI, 2002. PCI Geomatics V8.2 Reference Manual. Science, National Chiao-Tung University, for his valuable Rane, S., and G. Sapiro, 2001. Evaluation of JPEG-LS, the New Lossless suggestions, and the anonymous reviewers for their con- and Controlled-Lossy Still Image Compression Standard, for Com- pression of High-Resolution Elevation Data, IEEE Transactions on structive suggestions. Geoscience and Remote Sensing, Vol.39, No.10, pp. 2298–2306. Robinson, C., B. Montgomery, and C. Fraser, 1995. The Effects of Image Compression on Automated DTM Generation (URL:http:// References www.ifp.uni-stuttgart.de/publications/phowo95/Robinson.pdf; Aboufadel, E., 2001. JPEG2000: The Next Compression Standard last date accessed: 05 November 2004). using Wavelet Technology (URL:http://www.gvsu.edu/math/ Saha, S., 2000. Image Compression – from DCT to Wavelets: A wavelets/student_work/EF/; last date accessed: 05 November Review, ACM Crossroads Student Magazine (URL:http:// 2005). www.acm.org/crossroads/xrds6–3/sahaimgcoding.html; last date Adams, M.D., 2001a. The Jasper Project Homepage, Digital Sig- accessed: 05 November 2004). nal Processing Group at University of Victoria (URL:http:// Santa-Cruz, D., R. Grosbois, Touradj, E., 2002. JPEG 2000 perform- www.ece.uvic.ca/ mdadams/jasper/; last date accessed: ance evaluation and assessment, Signal Processing: Image Com- 05 November 2004). munication, Vol.17, Issue: 1, pp. 113–130. Adams, M.D., 2001b. Jasper Software Reference Manual Version Schenk, T., 1999. Digital Photogrammetry, Volume I, TerraScience, 1.500.4 (URL:http://www.ece.uvic.ca/ mdadams/jasper/; last 428 p. date accessed: 05 November 2004). Schiewe, J., 1998. Effect of Lossy Data Compression Techniques on Adams, M.D., 2001c. The JPEG-2000 Still Image Compression Stan- Geometry and Information Content of Satellite Imagery, dard, ISO/IEC JTC 1/SC 29/WG 1 N2412. International Archives of Photogrammetry and Remote Sensing, Chen, H.Y., 2002. On the Evaluation of Edge Preserving Smoothing Vol. 32, part 4, pp. 540–544. Filter, Masters Thesis, Department of Civil Engineering, National Schmanske, B.M., and M.H. Loew, 2001. Effects of Lossy Wavelet- Chiao-Tung University, Taiwan. based Compression on Classification Accuracy, IEEE Interna- Chen, P.Y., 2001. Introduction to Data Compression, A Cang Hai tional Geoscience and Remote Sensing Symposium (IGARSS), Publisher (in Chinese). Vol.1, pp. 106–108.
166 February 2005 PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING 03-082.qxd 1/11/05 5:17 PM Page 167
Tai, S.C., 2001. Image Compression, A Blue Publisher (in Chinese). Wallace, G.K., 1992. The JPEG Still Picture Compression Stan- Taubman, D.S. and M.W. Marcellin, 2002. JPEG2000, Kluwer dard, IEEE Transactions on Consumer Electronics, Vol. 38, Academic Publishers. pp. xviii–xxxiv. Veldhuizen, T.L., 1998, Grid Filters for Local Nonlinear Image Restoration (URL: http://osl.iu.edu/ tveldhui/papers/MAScThe- (Received 30 June 2003; accepted 07 November 2003; revised 02 sis/; last date accessed: 05 November 2004). February 2004)
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING February 2005 167