Lmage Compression by the JPEG Algorithm

PEER.REVIEWED

lmageGomBression bythe JPEG Algorithm

Jussi Lammi and TapaniSarjakoski

Abstract compressthe whole image in a single compressionstep. A schemefor doing this in smaller parts is proposed.The geo- Image compression is a necessityfor the utilization of large of image compressionon full-color images digital images, e.g., digitized aerial color images. The lPEc metric effect IPEG is empirically studied. We wanted to test our assumption still-picturc compression algorithm is one alternative for car- that image compressiondoes not affect the image geom- rying out the image compression task. The pnc method itself JPEG etry if small compressionratios are used. and its suitability for photogrammetric work are studied, with special attention being paid to the geometric degradation of digital images due to the compression process. In our JPEGlmage Compression Alsorithm experience, the pnc algorithm seems to be a good choice The following overview of the IPEGimage compressionalgo- for (1991) image compression. For color images, it gives a compression rithm is basedon articles by Wallace and L1ger et al. rutio of about 1:L0 without considerabledegrudation in the (1991).Those interestedin a detailed description of the IPEG visual or geometric quality of the image. algorithm are encouragedto examine the standard specifica- tions directly (at the moment of writing, the ISO/IECDIS 10s18-1(t99r) was availableto us). Introduction The /PEGstandard contains four modes of operation: se- Compressionof digital imagesis a necessityin applications quential encoding, progressiveencoding, lossless encoding, where many large images have to be archieved in a limited and hierarchical encoding.The sequentialand progressive storagespace or where digital imagesare transmitted over encodingsare based on the discretecosine transform. Both of narrow channels.The basic idea of image compressionis to these modes are lossy encoding techniques,i.e., some infor- remove redundancy from the image data. This is usually mation is lost during the compressionprocess. In the sequen- done by mapping the image to a set of coefficients.The re- tial mode, each image component is handled in a single sulting set is then quantized to a number of possiblevalues left-to-right,top-to-bottom scan while, in the progressive which are encodedby an appropriate coding method. Nowa- mode, the image is handled in multiple scans.In the lossless days, the most commonly used image compressionmethods encoding, the image is compressedso that the exact recovery are basedon the discretecosine transform (Rosenfeldand of the original image is guaranteed.The losslessmode of (Gray, Kak, lgaz; Wallace, 1991),on vector quantization Jrnc is based on a predictive method, The hierarchical mode 1984; Nasrabadiand King, 19BB),on differential pulse code encodesthe image at multiple spatial resolutionsusing either modulation (Gonzalezand Wintz, 1987),and on the use of the ncT-basedcompression or the losslessmode. image pyramids (Burt and Adelson, 1983; Mdkisara, 1991). Within the different modes of operation, a variety of en- The IPEGimage compressionalgorithm proposedby the coder/decoderpairs (so-calledcodecs) can be specified.AI- (lnnc) Joint PhotographicExperts Group offers a viable way though IPEGprovides a lot of possibilities for encoding,it of accomplishingthe image compressiontask. The JPEG also gives a "basic" compressionscheme-baseline sequen- group is an ISO/CCITT(International Standards Organization./ tial encoding-for straightforward use. In the following, we International ConsultativeCommittee for Telephone and Tel- shall concentrateon this method only. This is justified be- egraph)working group whose aim has been to develop an in- causebaseline encoding is sophisticatedenough for many ternational standardfor continuous-tonestill-picture com- applications, and it already explains the common idea be- (lso pression working group ITC1/SC2/WG10in collaboration hind most of the ;rec modes. Besides,the IPEGimplementa- with CCITTSGVIII). In the compressionmethod selectionpro- tions cunently on the market typically support only the cess-conducted by the JPEGcommittee-the discretecosine baseline sequentialencoding. (lcr)-based transform compressionwas chosen for standard The baseline encoding method of ;rnc contains three se- develooment. quential steps:forward discretecosine transform (rncr), The final ISo standard for;mc image compressionwill quantization,and Huffman coding (Figure 1), The processing be divided into two parts. Part 1 will specify the require- schemeis applied to a stream of B- by B-pixel blocks with B- ments and guidelines for the JPEGimage compression,and bits per pixel. The use of small image blocks takes into ac- Part 2 will contain the compliance teststaken. Accordins to count the fact that the correlation between adjacentpixels is the ISo Technical Program igss, the current Draft Interni- usually high in imagesof natural scenes.Decompression is tional Standardof the JPEG(DIS 109181 is expectedto be pub- achievedby following the processingsteps in the opposite lished as an International Standard US10918) before the end direction: Huffman decoding,dequantization, and inverse of year 1993. discretecosine transform (IDCT). This paper gives an overview of the IPEGimage compression algorithm, which is still quite new to the photogram- Photogrammetric Engineering & Remote Sensing, metric community. As the digital imagesused in photogram- Vol. 0t, No. 10, October 1995, pp. 1.261.-1.266. metric work tend to be very large, it is not reasonableto 0099-1112l95/61 10-1261$3.00/0 Finnish GeodeticInstitute, Departmentof Cartographyand O 1995 American Society for Photogrammetry Geoinformatics,Geodeetinrinne 2, O243OMasala, Finland. and RemoteSensing

PE&RS PEER.REVIEWED ARTICTE

DiscleteCosine Transform The B- by B-pixel block is transformedinto a set of basis-sig- nal amplitudes (64 coefficients)by the two-dimensional dis- crete cosine transform. The two-dimensional ncr of the B by B discretefunction f[x.v) is quantized F(u,v) Figure2. The coefficient are orderedinto a "zig- zag" sequence. : 1"(,r"@ Z,P- f(ry)cost*#]no] .", 1q#] (1) and the correspondinginverse is f(x,y) 1+\-, , f(zx+r)u"tl fQy+t1wrl :: L L c(u)c{v) F{ u'v)cosL 4uov-o 16 J"utL 16 J though the IpEc proposal specifiestwo entropy coding meth- (2) ods (Huffman and arithmetic coding), the baieline compressionuses Huffman coding only. where c(u), c(v) : I/t/i for u, v: 0, and c(u), c(v) : 1 for u, Colorlmate Compression Note that the DCTsimply maps the image block from one The baseline sequentialcoding is representationto another-in principle, it does not lose data. for 8-bit images,but it can be applrgd to color imagesas well. The color image compres- However, in transformation,the energy of the image block is sion is done by compressing compactedinto a few coefficients, multiple channelsone by one (non-interleavedorder) or by an approachin which all channels from the data unit (S- by S-pixel Quantization block) are com- pressedbefore proceedingto the next unit (interleaved After the FDCT,each cell in the 64-elementcoeffrcient matrix order). Although the baselinemethod is quantized to a correspondingvalue in the predetermined compressescolor images presented by any color model, it is best for quantization table. This is applied by dividing each DCTcoef- images that are in color spacessuch as YUV (y for luminance, UV for chromi- ficient (F(u,v))by the correspondingquantization element nance) in which the color components (Q(u,v))and rounding the result of division the in- are independent.As to nearest the teger:i.e., chrominancevalues need not to be consideied as fre- quently as luminance values, the spatial resolution of the U and v componentscan be decreased.This explains why Fa(u,v): (3) roundlffi] color imagescan be compressedwith a better ratio than grey- scale images. The quantization step performs the major and lossy part of Subsamplingof the chrominance channelsis done in the the compressionin the pEG algorithm. Here, the number of spatial domain, e.g.,by leaving every other pixel away in the different coefficient values is reduced and the number of line and/or column direction. If imagesare -ompressedin zero value coefficientsis increased. the interleaved order, the data units have to be combined to quantization, After the quantized coefficient for the zero such groups that the encoding can be done independently- for frequency in both dimensions (so-calledDC coefficient) is en- each group. Grouping is necessarybecause chrominance coded as the differencefrom the DCterm of the previous channelshave to be composedof B by B blocks also after block. Finally, all coefficientsare ordered into a "zig-zag" se- subsampling.Next, the baselineencoding schemeis applied quence (Figure 2). This ordering makes entropy coding easier to all channels. by placing the low-frequency coefficientsbefore the high-frequency coefffcients. Compressionof Large Digital lma(es Decompressionof large digital imagesis time-consuming. EntropyCoding Large imagesare also rather cumbersometo handle as a The final step in the lcr-based compressionis entropy cod- whole. It is also true that, in many applications, only part of ing. Here, the quantized coefficientsare encoded to a more the image is required immediately. An approach in which compact form by using the statisticalstructure of data. Al- the original image is divided into tiles "large enough" to be

lompre.ssed t$'ffi!fiff- {[-J rquannanonr I utroons I rmsedara Figure1. BaselineJpEc compression scheme.

L262 PE&RS PEER.REVIEWED ARIICTE

used as compressionunits is useful when we try to over- come these problems. A catalogindicating where each variable size block begins in the file is needed for quick accessto l*-*"",.il the image parts (Figure 3). l\ In piecewise compression,only those parts of the image are decompressedthat are really needed.The time required ,-'i;;"'== to decompressa few tiles is moderatecomoared to decom- pressingthe whole image. Note that the time used for compression of large digital imagesis not so crucial. This is quite obvious becausecompression is typically done only once,and becausethe whole compressionprocess is sepa- rated from the actual use of images. Piecewisecompression seems to be a suitable schemeto be used, e.g.,in ph-otogrammetricworkstations. It is also ex- pected to be commonly used in other application areas;e.g., Tiled image the current version of the well-known TIFFimage file format Variable siz€ blocks of Aldus Corporation has a capability to representlpEG com- pressedimages in a tiled form. Figure3. Piecewisecompression of a largedigital image. Testingthe Geometric Effects of theIPEG Geometric degradationof digital images due to the compression processis especially interesting in photogrammetry,becausethe geometricquality of the imagesis important. The with a Sharp JX-600 desktop scanner in full-color mode and geometricquality of an image can be degradedin a compres- with a resolution of 600 dpi. We compressed the scanned sion processin two ways: [1) by degradingthe radiometric image (Onglnol) according to the baseline IPEGimage com- quality of an image after compressionsuch that measured pression scheme, using Storm Technology's PicturePress soft- targetsare blurred or ambiguousand, therefore,pointing be- ware with a Micron Xceed ICDP-II Picture Accelerator. The comes inaccurateor impossible; and (2) by shifting objectsin image was compressed into three different levels (Excel1enf, line or column for bothJ directions. Both of these degrada- High, and Fard, with compression ratios of about 1:7, 1:15, tions can be global or local by nature. and 1:66, respectively. A portion of the test imagery is To a considerableextent, geometric degradationdepends shown in Plate 1. on the compressionmethod used, but also on the contents of The visual quality of the image Excellent was very good. the compressedimage and the amount of compression.It is The image High was also quite good under visual examina- clear that, if the compressionmethod is a losslessone, there tion although some compression effects were visible. The vi- is no danger of geometric degradation.On the other hand, all sual quality of the image Fol'r was poor-the size of the lossy methods are a possible source of geometric degradation compression blocks (s by B pixels) was clearly visible, and of type one. Some of these methods might affect shifts as all edges were heavily smoothed. Despite this degradation in well-this applies particularly to methods that use global visual quality, we used image Fori in the test as an example knowledge for compression.As compressionmethods usu- of a case in which the compression has become too large. ally have difficulties with spatial objectswith large intensity For the test, we chose 50 linear features from the center variations, it is reasonableto expect geometric degradation area of the images. The features selected were larger than the with these targets. compression blocks and were situated in areas where geo- The researchof Mikhail et al. (1,984)is an example of a metric degradations were likely to occur fedges with large in- study where geometric degradationin the compressionproc- tensity variations). They were similar in the sense that all ess is examined. In their work, the authors studied the geo- were about 70 pixels long and represented the edges of roofs metric effect of a discretecosine transform-basedimase (see Original in Plate 1). compression.It was found that the nct-based compreision We repeatedly measured the test set in all images using moved the measuredcross targets as much as 0.5 pixels manual pointing-20 times with the uncompressed image when the compressionratio was about L:16 on B-bit images. and ten times with the compressed images. Sub-pixel point- As assumed,the error decreasedas the compressionde- ing was achieved by zooming the viewed image area so that creased. every pixel on the image was doubled before pointing. Draw- ing routines from the available graphics library were applied TestAnangement instead of sub-pixel graphics {Sarjakoski and Lammi, tg93), We studied empirically the geometric effect of pnc image because sub-pixel drawing was not yet implemented in our compressionon full-color images.We wanted to test our as- mensuration software at the time of this test. sumption that JPEGimage compressiondoes not affect image The root-mean-square errors (RMSE)for perpendicular geometryif small compressionratios are used. We also differences between the endpoints of linear features were cal- wanted to know what happens when the amount of compres- culated (Figure 4). These root-mean-square errors were com- sion becomeslarge. It was consideredimportant to examine bined to represent the pointing precision of the linear two factors empirically: precision and accuracy.Precision is features in each image. A set of average features for each test a measureof internal quality or repeatability of the measure- set was formed by calculating averages from the endpoint co- ments. On the other hand, the term accuracyis used to de- ordinates of the linear features measured. To test the point- note the external (or global) quality of the measurements ing accuracy of the linear features in the test images, we kept comoared to some referenceor "truth." the set of average features from the Original data as a refer- The test image was an aerial color diapositive digitalized ence when we calculated the RMSEfor perpendicular differ-

PE&RS l

PEER.REVIEWED ARTICI"I

{ .,7 t s

l q .\ -q { ..?

ffi :,rq ffilt -!4 r

Org n.r il\11(l i'ir

a ; .t .l q

fl ry d*'$ +

Higlr fr,lr

.t Platr, 1. { ltcrt-liOrOf tlta'trst nrit!('r\ slrrri.,irl, ilra rrri,rgi'(,1/.rtr/r,r1 1|1j 1ir ,,i,l,rr r, .i1 . 1i il I1l.llr.F.\ce/iclti 1.7r. f/rglii1.15r lrrl irir l.(l{r,i.I tlll, r.lr.t!l(,()rlqrl,;' -.ri.r Lr ! | lr - , , ,1 slrowr'rlr rccl.

f!F_&RS PEEN.REVIEWED ARIICTE

TaeLe1. PotrurncPREcrsroN or LtnenRFentuREs rN TEST IMAGEs Pnesexreo sv MAxrvuN4ERRoR (MAx vy) AND Roor-MEAtt-Seunnr Ennon (RMSE). Tue VaruesAne Expnrsseo ru PrxeLs.Decnees op FReroovFoR DTFFERENT TEST Srrs AnEnLso PRrssNrEo (u). Test sets (compressionratio) max vy RMSE Original 0.90 + o.2B 1842 Excellent (t:z) 0.84 + o.27 886 Higfi (l:rs) 0.88 + 0.26 878 Fair (r:06) 1.15 + 0.39 864

o Endpoint of a linear featur€

I ee"pe"ai""t"" distanc from the TleLe 2. Slvpre VnRterucESBETWEEN rHr Test Stt Oatatut nno oruen Tesr features StentncaNcE oN { averageoflinear SETSARE Cotl,tpnReoav Usrr,rcrre F-Dtsrntaurton. THr LEVEL Figure4. Perpendiculardifferences between the end- wHlcHTHE AppRopnrme Nulr- Hvpotnests Ho CnruBr RetEcreols St-towt'tttl (Ho rHAT Hrvr rxe Snue Venteruce).THE TEST pointsof linearfeatures. PeRcem Exeecrs DrsrRrBuTrorus ls BrsEo or ruE VnLuEsPRESENTED tt{ TneLe 1. Excellent(t:7) High (r:75) Fojr (1:66) encesin the compressedtest sets.We also directly compared 8O.2"/o 99.70/o 99.S% the compressed sets of average features with the set of average features from the Original data. Again, root-mean-square errors for perpendicular differencesbetween the endpoints set and Ihe Original set. However, the finding that the set were calculated. High is even more precise than the set Original is somewhat surprising. One explanation is that the staircaseeffect on the PointingPrecision test imagesis smoothedthrough image compression,and so The pointing precision of the linear featuresin the test im- the pointing is easierto repeat.The pointing precision in the ages was calculated according to the test arrangement. The set Forr is clearly poorer than the precision in the set Origr- test setswere considerednormally distributed. Featureswith no1. According to the F-test, the sets Foir and Original almost a perpendicular difference greater than 3 times the root- surely originated from a different data generation process. mean-square error of the corresponding data set were kept as The compressionused in this caseis simply too large-the grosserrors and were reiected.The pointing precisions of the pointing of linear features is no longer unambiguous. linear featuresin the test imagesare presentedin Table 1. The compressedtest setswere compared with the set PointingAccuncy Original one by one. For comparison ofvariances (D,o,,n,,.,, The set of averagefeatures from the Original image was used 6'co-p,""""a),the hypothesisHo is written as as a reference when the RMSerrors for perpendicular differ- enceswere determined.The results from these calculations Ho: : (4) Ero.,rr,u, 6rco-p.u""ud are presentedin Table 3. The set of averagefeatures from the and H, as compressed images was also directly compared to the Origi- no/ set of averagefeatures. Results from these comparisons H.: Ero.rr,,u,* 6r"o_o.."",a. (s) are shown in the Table 4. Comparisonof the results in the tables shows that the The F-test was used test the to statistical significanceof set Excellent is very close to the set ariginal. The set High is hypothesis The the above. ratio between the sample vari- also quite close to Original. However, the values in Tables 3 (s2orisi,ul,s2co*p.""""d) used (zo): ances was as a test parameter and 4 indicate that there may be errors of a systematicna- 1.e,, ture in some features of High. In Fair the number of large errors has grown so much that the difference from Origrnal is z.: j!::si-L. (6) obvious. 5-Compressed After the test, we checkedthe featuresthat causedthe The hypothesis Ho is rejectedwith the risk crif largest residuals in the set Foir. The linear features having the four largestmisplacements in a perpendicular direction ) z, F, o/z(uoriginal, uco*pr""".a) (7) were causedby the fact that the image compressionratio was too large for those objects.The original edge either had dis- appearedtotally or had been smoothed so badly that the pointing was more like guesswork.In two of the cases,the zr3 (B) disappearededge had a similar kind of linear feature in the ('"o-p.""""a, F. -or, uorigi'ut) close neighborhood.This coincidence causeda systematic where vo.,r,."r,uco-p.."sed are the degreesof freedom and F rep- error in the pointing. resentsthe F-distribution. The significancelevels (i.e., 1-ct/ 2) on which the appropriate null hypothesisHo can be re- Conclusions jected are presentedin Table 2. We have empirically studied the geometriceffect of Jrnc im- The pointing precision in the set Excellent does not dif- age compressionon full-color images.The visual quality of fer statistically significantly from that in the set Original Bfi imageExcellent (compressionratio 1:7) was very good, and in the set High therc is a difference between the compressed no noticeable degradationin the geometric quality of this im-

PE&RS 1265 PEER.REVIEWED ARIICTE

TneLe3. Tne RMS ERRoRSFoR PrRprruorcuLlR DrrrEReNces rn ruE ISO/IEC DIS 10918-1, TggL Digital Compression and Coding of Con- CoN4pREssEDTrsr Sers Cnrcuureo ev UsrNcrse Ser or AvERnceFEATURES tinuous-Tone Still Images, ISO/IEC JTCllSC2/WG10. FRoMrHE Sg Oneu,ut As A REFERENcE(on "Ment"). Mnxrvult Ennons (vnx L6ger, A., T. Omachi, and G.K. Wallace, 1991. JPEG still picture (RMSE) v") nruoRoor-Menr-Squnnr Ennons AREPRrsrnrEo. Vrlurs AnE compression algorithm, Optical Engineering, 30(7):947-954. EXPRCSSTOIruPIxErS. DEGREES or FReEooIr,IroR DIrrEnEnTTEST SETS ARE ALSo Mikhail, E.M., M.L. Akey, and O.R. Mitchell, 1984. Detection and Pneserureo(v). sub-pixel location of photogrammetric targets in digital images, Test sets Photogrammetrio, 39(3) :63-83. Icompressionratio) max v" RMSE Mdkisara, K., 1991. Adaptive Laplacian pyramid compression of remote sensing images, Proceedings of the 1991 International Excellent (7:7) 1.00 + 0.30 886 Geoscience and Remote Sensing Symposium (IGA&SS'91), Hel- High (1:"15) "1.40 + 0.31 87a sinki, Finland, 3:7439-1,442. Foir-(1:66) 3.32 + 0.65 864 Nasrabadi, N.M., and R.A. King, 1988. Image coding using vector quantization: a review, IEEE Transactions on Communications, 36(81:957-971. TaeLe4. THg DtrrrRrncr BETwEENrHr Ser or AveRncr Frnrunes FRoMTHE Rosenfeld, A., and A.C. Kak, 1982. Digital Picture Processing, Vol. 1, TESTSET Oner,tn ANDTHE Ser or Avrnnce FEATURESFRoM THE ColpREsseo Second Edition, Academic Press, Orlando, Florida, 435 p. TEsr Srrs. MexrpruvEnnon (ulr vy)AND Roor-MEAN-SeuARE ERRoRS (RMSE) Sariakoski, T., and Lammi, 1993. Requirements of a stereo work- ARe PResenreo,Vlrurs ARr ExpResseorn PrxeLs.Decnrrs oF FREEDoMFoR f. station for the GIS environment, of Visual Languages DrrrenErt Trsr Sers ARellso PRESENTED(u). Iournal and Computing, 4(2):127-742. Test sets WaIIace, G.K., 1991. The JPEG still picture compression standard, (compressionratio) InAX V\ Communications of the ACM, 34(4)3|-aa.

Excellent (t:z) o.27 t o.12 99 [Received 26 February L993; revised and accepted 14 January 1994; High (715) o.73 + 0.18 s9 revised 22 ApfiI \gg4) Fair [1:66] z-/b + 0.50 99 Tapani Sariakoski Tapani Sarjakoskigraduated from the Helsinki age was found. The visual quality of image High (1:15)had Universityof Technologyin 1978with an M.Sc. slightly deterioratedwhen comparedwith Ihe Original. A degreein surveying engineering.After that he small geometric degradationeffect in the caseHigh was worked as a researcherin analytical photogram- found in the test-some of the linear featureswere mis- mehy at the Technical ResearchCentre of Fin- placed. In summary, baselineJrEG image compressiondoes land. From 1984to 19BBhe receiveda position from the not have a geometric effect on the image geometrywhen Academy of Finland for postgraduatestudies. During 1984- compressionratios of about L:10are used.Some geometric 1986 he studied artificial intelligence and expert system degradationeffect may occur with higher compressionratios. technology at Purdue University as applied to photogramme- Our examination was basedon the use of visual pointing try, and he received the Doctor of Technology degreefrom instead of numerical feature extraction methods.We believe the Helsinki Universityof Technologyin 1988.Since 19BB that the results are applicable when digital imagesare used he has been the head of the department of cartographyand for visual, interactive mensuration in workstations.The use geoinformaticsat the Finnish GeodeticInstitute. During this of ;enc image compressionmight have varying effectson nu- period, he has led researchin computer based cartography merical feature extraction and mensuration methods.This and geoinformatics.Spatial datamodels,knowledge based gives a topic for further research. methods, and integration of digital stereoimagery into geo- graphical databaseshave currently been his special interest.

References |ussi Lammi Burt, P.J.,and E.H. Adelson, 1983. The Laplacian pyramid as a com- JussiLammi graduatedfrom the Helsinki Uni- pact image code, IEEE Transactions on Communicafions, 31(4J: versity of Technology 1,992with an M.Sc. de- 532-540. gree in surveying engineering.He has since Gonzalez, R.C., and P. Wintz, "1987. Digital Picture Processing, Sec- started his postgraduatestudies in the field of ond Edition, Addison-Wesley, Reading, Massachusetts, 503 p. -- digital photogrammetry.Cunently he is working Gray, R.M., 1984. Vector quantization, IEEE ASSP Magazine, 1.(2):4- as a researchscientist at the Finnish GeodeticInstitute in the 29. department of cartography and geoinformatics.

Havinga meeting?

Help promote ASPRS by displaying one of our tabletop exhibits. We'll also send along an assortmentof literafure about ASPRSincluding: membership applications, society and conferencebrochures,and publications catalogs. We have five of these tabletop displays in rotation, so g€t on our schedule today! Contact: fulie HilI at ASPRSHeadquarters, 5410 GrosvenorLane, Suite 210,Bethesda, MD 2O874-276O; phone 301-493-0290,ext. 21 or fax to 301-493-0208;email: [email protected].

PE&RS