IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 29, NO. 5, MAY 2019 1559 Letters Efficient Chroma Sub-Sampling and Luma Modification for Color Image Compression Shuyuan Zhu , Member, IEEE, Chang Cui, Ruiqin Xiong , Member, IEEE, Yuanfang Guo , Member, IEEE, and Bing Zeng , Fellow, IEEE Abstract— In color image compression, the chroma components are by a factor of 2 in both the horizonal and vertical directions in the often sub-sampled before compression and up-sampled after compression. pixel domain, leading to the YUV 4:2:0 format. Although sub-sampling the chroma components saves the bit-cost for Unfortunately, sub-sampling of the chroma components usually compression, it often induces extra color distortions in the compressed images. In this letter, we propose two approaches to tackle this problem. induces extra distortions to the compressed RGB images. These First, we propose a sub-sampling method in the transform domain and distortions may be reduced by applying the interpolation dependent apply it to both chroma components. Then, based on this sub-sampling, sub-sampling [2]–[5] to each individual chroma component. Note we propose a novel method to modify the luma component. In our that these techniques [2]–[5] must be carried out in the YUV space. proposed luma modification algorithm, the distortions that occurred in the two chroma components can be coupled together and utilized to In practice, they cannot guarantee a low RGB distortion for the modify the luma component. With our proposed chroma sub-sampling compressed color image. To solve this problem, the RGB distortion and luma modification algorithms, we can achieve a low RGB distortion constrained chroma sub-sampling is proposed in [6], where sub- in practical image coding. The experimental results demonstrate that our sampling of the chroma components is optimized to achieve a lower proposed methods offer more significant coding gains compared with the RGB distortion. The solution in [6] has demonstrated some promising state-of-the-art methods for the compression of color images. results when it is employed in the 4:2:0 format coding. Index Terms— Color image, compression, distortion, sub-sampling, Besides of the optimization of the sub-sampling operation for modification. chroma components, the luma component may also be modified I. INTRODUCTION to achieve a high-quality color image compression [7]. In [7], OLOR image compression typically happens in the YUV space, the traditional spatial sub-sampling is performed on two chroma Cwhere the input RGB signal is converted into the YUV signal components and a low RGB distortion is guaranteed by modifying the which is composed by one luma (Y) component and two chroma luma component according to the sub-sampled chroma components. (U and V) components. The chroma component is usually more This method is named as the joint chroma sub-sampling and distor- smooth than the luma component, and the latter one contains plenty tion minimization-based luma modification (JCSLM). Unfortunately, of textures. After the color space conversion, the luma and chroma JCSLM has to be carried out before compression. When it is adopted components are divided into image blocks. Then, the discrete cosine in practical coding schemes, the compressed color image still suffers transform (DCT), quantization and entropy coding are accordingly from certain compression distortion, which accordingly limits the performed on these blocks to achieve the compression. coding efficiency. In general, the RGB-to-YUV space conversion removes the redun- In this letter, we firstly propose a transform domain sub-sampling dancies between color channels [1], and the compression of the scheme for both the chroma components. Our proposed method YUV signal can achieve a high compression ratio more easily. targets a low RGB distortion and yields a comparable sub-sampling In practical YUV based image compression, the chroma sub-sampling performance to the YUV 4:2:0 format. Specifically, this proposed is employed very frequently to reduce the image data-size. This sub- scheme converts each N × N chroma block into an (N/2) × (N/2) sampling saves a considerable bit-cost, and thus potentially improves coefficient block, which accordingly induces a low bit-cost for the coding efficiency. Currently, the most popular chroma sub- compression. Then, based on this sub-sampling scheme, we propose sampling scheme down-samples two chroma components separately a novel method to modify the luma component. When our proposed modification algorithm is adopted in practical image coding, the Manuscript received November 1, 2018; revised January 14, 2019; accepted distortions, including both the sub-sampling and compression distor- January 23, 2019. Date of publication January 29, 2019; date of cur- rent version May 3, 2019. This work was supported in part by the tions, occurred on two chroma components can be coupled together National Natural Science Foundation of China under Grant 61672134, and utilized to modify the luma component. At last, we can achieve Grant 61772041, Grant 61802391, and Grant 61720106004, in part by the state-of-the-art performance, i.e., a low RGB distortion as well the Fundamental Research Funds for Central Universities of China under as a high coding efficiency, for the compression of color images. Grant ZYGX2016J038, and in part by the 111 Projects under Grant B17008. This letter was recommended by Associate Editor S. Liu. (Corresponding authors: Yuanfang Guo; Bing Zeng.) II. RGB COLOR DISTORTION S. Zhu, C. Cui, and B. Zeng are with the School of Information and Com- The RGB-to-YUV space conversion is implemented by perform- munication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China (e-mail: [email protected]). ing a transform on the red (R), green (G) and blue (B) pixels { , , } R. Xiong is with the Institute of Digital Media, School of Electronic Engi- xRi xGi xBi to produce the corresponding Y, U and V pixels { , , } neering and Computer Science, Peking University, Beijing 100871, China. xYi xUi xVi as Y. Guo is with the Laboratory of Intelligent Recognition and Image ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ Processing, School of Computer Science and Engineering, Beihang University, xYi xRi 16 Beijing 100072, China (e-mail: [email protected]). ⎣ ⎦ = ⎣ ⎦ + ⎣ ⎦, xUi A xGi 128 (1) Color versions of one or more of the figures in this letter are available xV xB 128 online at http://ieeexplore.ieee.org. i i Digital Object Identifier 10.1109/TCSVT.2019.2895840 where A is the color transform matrix. 1051-8215 © 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. 1560 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 29, NO. 5, MAY 2019 2 2 2 If the degradation, such as the sub-sampling or the compression, where DL is an N × M matrix and DH is an N × (N − M) happens to the Y, U and V pixels, the reconstructed R, G and matrix. If we remove XH and only reserve XL for further processing, {ˆ , ˆ , ˆ } Bpixels xRi xGi xBi should be determined by the degraded the transform domain sub-sampling for X is accordingly achieved. {ˆ , ˆ , ˆ } ˆ = Y, U a nd V pi xe l s xYi xUi xVi based on the inverse transform With the sub-sampled X, i.e., XL , x can be rebuilt as x DL XL. { , , } of (1). Then, the color distortion measured on xRi xGi xBi is However, due to the removal of the high-frequency components, calculated as rebuilding x directly from XL tends to induce a high distortion. When ⎡ ⎤ ⎡ ⎤ T the compression is performed on X , the reconstruction of x from x −ˆx x −ˆx L Ri Ri Ri Ri the compressed X will suffer from a more serious distortion. d = ⎣x −ˆx ⎦ ⎣x −ˆx ⎦ L RGBi Gi Gi Gi Gi To achieve a high-quality reconstruction, we propose to preserve xB −ˆxB xB −ˆxB ⎡ i i ⎤ i i ⎡ ⎤ the high-frequency information as much as possible in the remaining −ˆ T −ˆ xYi xYi xYi xYi low-frequency coefficients. Specifically, we produce XL with the = ⎣ −ˆ ⎦ ( −1)T ( −1) ⎣ −ˆ ⎦. xUi xUi A A xUi xUi (2) minimal reconstruction distortion as xV −ˆxV xV −ˆxV i i i i X˜ = arg min x − D X 2. (6) =[ ] =[ ] =[ ] L L L 2 Let us use xY xYi K ×1, xU xUi K ×1 and xV xVi K ×1 XL to represent the K-point column-vectors for the Y, U and V compo- Then, the optimal XL is generated as nents, respectively. When the degradation happens, with the degraded − ˆ =[ˆ ] ˆ =[ˆ ] X˜ = (DT D ) 1(DT x). (7) luma and chroma components, i.e., xY xYi K ×1, xU xUi K ×1 L L L L and xˆ =[ˆx ] × , we calculate the distortions of the Y, U and V V Vi K 1 According to (7), a sub-sampled coefficient vector with M coef- components as e = x −ˆx , e = x −ˆx and e = x −ˆx , Y Y Y U U U V V V ficients is produced and these coefficients will be further exploited respectively. Then, based on e , e and e , the YUV error vector Y U V to reconstruct x, leading to our proposed upward-conversion guided can be constructed as e =[eT eT eT ]T . Similarly, in the RGB YUV Y U V transform domain sub-sampling (UGTS). space, the corresponding RGB error vector eRGB can be obtained. − The proposed UGTS can be applied to both the chroma compo- By assuming that λ , (m, n = 0, 1, 2) is the element of A 1, m n nents for color image compression. Specifically, we specify M = we can exploit λ , to form a color conversion matrix such that m n N2/4 to yield a comparable sub-sampling performance to the YUV × eRGB = eYUV.