Human Visual System Inspired Color Space Transform in Lossy JPEG 2000 and JPEG XR Compression Roman Starosolski Institute of Informatics, Silesian University of Technology, Akademicka 16, 44-100 Gliwice, Poland [email protected] Abstract. In this paper, we present a very simple color space trans- form HVSCT inspired by an actual analog transform performed by the human visual system. We evaluate the applicability of the transform to lossy image compression by comparing it, in the cases of JPEG 2000 and JPEG-XR coding, to the ICT/YCbCr and YCoCg transforms for 3 sets of test images. The presented transform is competitive, especially for high-quality or near-lossless compression. In general, while the HVSCT transform results in PSNR close to YCoCg and better than the most commonly used YCbCr transform, at the highest bitrates it is in many cases the best among the tested transforms. The HVSCT applicability reaches beyond the compressed image storage; as its components are closer to the components transmitted to the human brain via the optic nerve than the components of traditional transforms, it may be effec- tive for algorithms aimed at mimicking the effects of processing done by the human visual system, e.g., for image recognition, retrieval, or image analysis for data mining. Keywords: image processing, color space transform, human visual sys- tem, bio-inspired computations, lossy image compression, ICT, YCbCr, YCoCg, LDgEb, image compression standards, JPEG 2000, JPEG XR 1 Introduction For natural images, the correlation of the RGB color space primary color com- ponents red (R), green (G), and blue (B) is high [18]. Correlation results from the typical characteristic of RGB images and reflects that the same informa- tion is contained in two or all three components. For example, an image area which is bright in one component usually is also bright in others. Computer generated images also share such characteristic, since artificial images mostly are made to resemble natural ones. Recent image compression standards: JPEG 2000 [28,9] (as well as the DICOM incorporating JPEG 2000 [15]) and JPEG NOTICE: this is the author's version of a work that was subsequently published in S. Kozielski et al. (Eds.): BDAS 2017, CCIS 716, pp. 564-575, 2017. The final publi- cation is available at Springer via http://dx.doi.org/10.1007/978-3-319-58274-0 44. 2 Roman Starosolski XR [4,10] compress independently the components obtained from an RGB image by using a transform to a less correlated color space. Although the independent compression of transformed components is not the only method for color image compression, it is the most frequently used one. It allows to construct a color image compression algorithm based on a simpler grayscale image compression algorithm. As compared to compressing the untransformed components, by ap- plying the color space transform we improve the image reconstruction quality or the lossless compression ratio (for lossy and lossless algorithms, respectively), since without the transform the same information would be independently en- coded more than one time. However, alternative approaches are known that take advantage of inter-component correlations while encoding of untransformed or transformed components [1,6,7]. In this paper we present a human visual system inspired color space transform (HVSCT) for lossy image compression. We evaluate this transform by comparing it for 3 sets of test images and 2 image compression standards (JPEG 2000 and JPEG-XR) to transforms ICT/YCbCr and YCoCg. The reminder of this paper is organized as follows. In Section2 we discuss properties of irreversible color space transforms and present the ICT/YCbCr and YCoCg transforms used then for comparison with HVSCT. Section3 in- troduces the new transform. Section4 contains experimental procedure, results, and discussion; Section5 summarizes the research. 2 Color space transforms The Karhunen-Lo`eve transform (KLT) is an image-dependent transform that for a specific image is constructed by using the Principal Component Analysis (PCA), it optimally decorrelates the image [16,18]. The computational time com- plexity of PCA/KLT is in practice too high to compute it each time an image gets compressed. Instead, fixed transforms are constructed based on PCA/KLT by performing PCA on a set of typical images. Then, assuming that the set is suf- ficiently representative also for other images, which were not included in the set, we use the obtained fixed KLT transform variant for all images. The frequently used color space transforms, for example, the YCbCr color space transform de- scribed below, are fixed transforms constructed based on PCA/KLT; however, there are algorithms constructing a color space transform for the specific image. An adaptive selection of the transform from a large family of 60 simple trans- forms was proposed by Strutz [26]; performing the selection slightly increases the overall cost of the lossless color image compression algorithm. In [27], an even larger family of 108 simple transforms is presented; adaptive transform se- lection is performed for the entire image or for separate image regions, however, the latter approach leads to only a small further ratio improvement. Singh and Kumar [19] presented an image adaptive method of constructing a color space transform based on the Singular Value Decomposition. Although this method is of significantly greater computational time complexity, than a method which directly selects a transform from a family of simple transforms, it is still simpler than computing PCA/KLT for a given image. Human Visual System Inspired Color Space Transform 3 The probably most commonly used color space, but the RGB space, is YCbCr. It was constructed using PCA/KLT, but with an additional requirement: the transform should contain a component that approximates the luminance per- ception of the human visual system [14]. YCbCr contains the Y component that represents the luminance and two chrominance components: Cb and Cr. YCbCr was constructed decades ago for video data and nowadays is used both for video and for still image compression. There are many variants of the transform be- tween RGB and YCbCr (resulting in respective variants of the YCbCr color space). Below we present one of them, ICT (Eq.1), with inverse (Eq.2): 2Y 3 2 0:29900 0:58700 0:114003 2R3 4Cb5 = 4−0:16875 −0:33126 0:500005 4G5 ; (1) Cr 0:50000 −0:41869 −0:08131 B 2R3 21:00000 0:00000 1:402003 2Y 3 4G5 = 41:00000 −0:34413 −0:714145 4Cb5 : (2) B 1:00000 1:77200 0:00000 Cr ICT is defined in the JPEG 2000 standard for lossy compression [10]. Note, that if the transformed components are to be stored using integer num- bers, then the transform is not exactly reversible|we say that it is irreversible or not integer-reversible. It is not a problem in a typical case of lossy coding, where distortions introduced by forward and inverse transform are much smaller than distortions caused by lossy compression and decompression. However, in the case of the very high quality coding, the color space transform may limit the obtainable reconstruction quality. The integer-reversible variants of ICT and of other transforms are constructed using the lifting scheme [2]. The reversibil- ity is obtained at the cost of the dynamic range expansion of the transformed chrominance components by 1 bit (the dynamic range of a component is de- fined as a number of bits required to store pixel intensities of this component). The dynamic range expansion affects the transform applicability, since certain algorithms and implementations either do not allow or do not process efficiently images of depths greater than, e.g., 8 bits per component. Expansion may be avoided by the use of modular arithmetic (as in the RCT transform in the JPEG- LS extended standard [8]), however, such transform introduces sharp edges to transformed components, that worsen the lossy compression effects. In this re- search we focus on typical transforms for lossy coding|not using the modular arithmetic and not expanding the dynamic range of transformed components. A recent YCoCg transform is an another interesting transform (forward in Eq.3 and inverse in Eq.4): 2Y 3 2 1=4 1=2 1=43 2R3 4Co5 = 4 1=2 0 −1=25 4G5 ; (3) Cg −1=4 1=2 −1=4 B 2R3 21 1 −13 2Y 3 4G5 = 41 0 15 4Co5 : (4) B 1 −1 −1 Cg 4 Roman Starosolski It was obtained based on PCA/KLT constructed for a Kodak image-set (see section 4.1 for the Kodak set description); YCoCg is an irreversible variant of a YCoCg-R transform included in the JPEG-XR standard [10,14]. The YCoCg transform is significantly simpler to compute, than ICT. The former requires 15 simple floating point operations (additions, subtractions, multiplications) for forward and 8 operations for inverse transform. The YCoCg forward transform may be computed in 6 integer operations (add, subtract, and bit-shift; the latter denoted by >>): t = (R + B)>>1; Y = (G + t)>>1; Co = R − t; Cg = Y − t; inverse in 4 additions and subtractions only: G = Y + Cg; t = Y − Cg; R = t + Co; B = t − Co; 3 New transform inspired by human visual system We described in detail previously [21] the following interesting fact. A color space transform that results in a single luminance and 2 chrominance components is performed by our (i.e., human) visual system. There are three types of cone cells in our retinas that are most sensitive to three light wavelengths, these are S-cones (short wavelength with sensitivity peak in violet), M-cones (middle wavelength, sensitivity peak in green), and L-cones (long wavelength, peak in yellow). According to the common opinion, the cones simply respond to blue (S-cones), green (M-cones), and red (L-cones) light.