Received: 29 June 2017 Revised: 7 September 2018 Accepted: 9 September 2018 DOI: 10.1002/col.22291 RESEARCH REVIEW Review and evaluation of color spaces for image/video compression Samruddhi Y. Kahu1 | Rajesh B. Raut2 | Kishor M. Bhurchandi1 1Visvesvaraya National Institute of Technology, Nagpur, India Abstract 2Shri Ramdeobaba College of Engineering and A color space plays an important role in color image processing and color vision Management, Nagpur, India applications. While compressing images/videos, properties of the human visual Correspondence system are used to remove image details unperceivable by the human eye, appro- Samruddhi Y. Kahu, Visvesvaraya National priately called psychovisual redundancies. This is where the effect of the color Institute of Technology, South Ambazari Road, Nagpur, 440010, India. spaces' properties on compression efficiency is introduced. In this work, we study Email: [email protected] the suitability of various color spaces for compression of images and videos. This review work is undertaken in two stages. Initially, a comprehensive review of the published color spaces is done. These color spaces are classified and their advan- tages, limitations, and applications are also highlighted. Next, the color spaces are quantitatively analyzed and benchmarked in the perspective of image and video compression algorithms, to identify and evaluate crucial color space parameters for image and video compression algorithms. KEYWORDS CIE L*a*b*, color spaces, color spaces' review and benchmarking, image and video compression, imaging, quantitative analysis 1 | INTRODUCTION approximately represent the wavelengths of red, green, and blue colors.2,3 A complex color “Ci” perceived as spatial inte- As rightly said, “An image is worth thousand words,” and as gration of incident wavelengths on the three cone sensors by known a video is a long sequence of images. Images and human brain is given by Equation (1)1; videos are used to capture a scene and display it at any later λmaxð instant and place, may be repeatedly, without loss of percep- ¼ ðÞλ ðÞλ λ ¼ ð Þ tual information. Colors are important components of any Ci Si f d i 1,2,3, 1 λ scene. Thus, for efficient and effective capture and display min of color images, a thorough study of the human visual sys- where Si(λ) denotes the sensitivity of the ith type of cones tem (HVS) was carried out and a few color spaces like Red that is, S, M, and L, f(λ) = S(λ), M(λ) or L(λ), and [λmin, 1 1 Green Blue (RGB) and Hue Saturation Intensity (HSI) λmax] denotes the interval of wavelengths outside which the mimicking it were devised. sensitivities are zero. Typically in air or vacuum, visible Human eye contains light sensors spatially and nonuni- region of the electromagnetic spectrum is specified by wave- 3 formly distributed all over the retina. There are mainly two length region between λmin = 360 nm and λmax = 830 nm. types of light sensors; (1) “Rods”—responsible for perception However, some sources state that the effective range is from of gray level or achromatic intensities of light primarily under 400 nm to 700 nm.4 Responses of the S, M, and L cones in very low luminance levels. At very low intensities of light, no this wavelength region are presented in Figure 1. colors can be perceived as only shades of gray are seen. The three primary/fundamental color theory, also known (2) “Cones”—responsible for perception of different colors, as the “trichromatic theory of color vision” was developed are of three types; namely, S, M, and L sensitive to short, based on the works of Maxwell, Young, and Helmholtz.2,5 medium and long wavelengths of light. These wavelengths They proposed the presence of three receptors in the human 8 © 2018 Wiley Periodicals, Inc. wileyonlinelibrary.com/journal/col Color Res Appl. 2019;44:8–33. KAHU ET AL. 9 visual contents using a color space suitable for minimizing psychovisual redundancies. Thus, selection of a color space S M L 1 is a crucial issue for designing compression algorithms. Figure 2 shows flowchart for a generalized compression 0.8 algorithm. Color space conversion is the first step as seen in Figure 2. 0.6 Today, YCbCr color space is widely used by most of the state-of-the-art compression algorithms such as JPEG,9,10 0.4 JPEG 2000,11 H.264,12 HEVC,13 and so forth, as it gives Responsivity high decorrelation. As HVS is more sensitive to luminance, 0.2 separation of luminance, and chrominance components allows use of chrominance subsampling14 resulting in high 0 compression ratios (CR). Most of the research in this field is 400 450 500 550 600 Wavelength (nm) done using the above mentioned decorrelation-based approach, leading to the widespread use of YCbCr. How- 2 FIGURE 1 Responsivity of LMS cones in human retina ever, a small number of compression algorithms have also come up which use the correlation between R, G, and retina approximately sensitive to red, green, and blue wave- – B components effectively.15 17 Hence, these algorithms are lengths of the visible light. Some studies even claim that the called correlation-based approaches. Analysis of the advan- theory of three fundamental colors was developed by tages and disadvantages of the correlation and decorrelation- Thomas Young based on the work of an English glassmaker based approaches is presented by Gershikov and Porat.18 A G. Palmer.5 Thomas Young neglected the fact that fourth few color spaces have also been designed especially for type of sensors are also available in the HVS that are respon- compression algorithms.19,20 sible for gray vision and also affect the saturation of a com- This work initially reviews all the major published color plex color. Accordingly, many other systems use four – spaces, their advantages, limitations, and applications. The components to represent a complete color space.6 8 Hering, main objective of this article is to analyze their suitability for in mid-1870s proposed the idea of four primary colors, compression algorithms. We have also identified and com- namely, yellow, red, blue, and green. This is called Hering's puted nine parameters for benchmarking different color “opponent-colors theory of vision”, which is based on the spaces for compression algorithms. Thus, 38 color spaces observation that certain colors were never perceived to occur are surveyed, analyzed, and benchmarked using the parame- together. He also proposed the presence of three types of receptors in the human eye with bipolar responses to light- ters to establish their suitability for the said purpose. This dark, red-green, and yellow-blue colors.2,5 Aubert first rec- study helps us find the reason behind the widespread use of ognized that Hering's opponent color theory and Young, YCbCr space for compression. Although YCbCr color space Helmholtz, and Maxwell's primary color theory are not con- is being used conventionally for compression, other spaces tradictory.5 Based on the stage theory developed by Jameson may be more suitable due to their properties like linearity and Hurvich, it was established that there are indeed three and perceptual uniformity. Thus, survey of the published types of trichromatic receptors in the human eye as proposed color spaces and their quantitative analysis for suitability for by Young, Maxwell, and Helmholtz. However, the output of compression algorithms are the two goals of this work. the three receptors is not sent as it is to the human brain. It is Rest of the article is organized as follows. Section 2 pre- used to form the opponent red-green and blue-yellow signals sents classification of different color spaces and different as proposed by Hering.2,3 factors affecting their performance in compression applica- In the due course of time, different color spaces were tions. Section 3 reviews all the major published color spaces, introduced for several applications like, color printing and their features, advantages, limitations, and applications. display, image and video transmission, and storage, etc. In Important parameters for quantitative analysis of the color the recent mobile communication and computing revolution, space have been identified and defined in section 4. Quanti- multimedia handling, transmission, and display have gained tative analysis of all these color spaces in terms of the identi- popularity and subsequent prime importance. Obviously, fied parameters has been presented in section 5. Section 6 image and video compression algorithms need to express the concludes the article. FIGURE 2 Generalized compression scheme 10 KAHU ET AL. FIGURE 3 Classification of color spaces 2 | FACTORS AFFECTING complex color. New family of calibrated color spaces intro- CLASSIFICATION OF COLOR SPACES duced for such applications is called device independent color spaces.21 Many color spaces have been introduced so far. On the basis of their definition, we classify them in three categories: (1) Fundamental (2) Derived, and (3) Application specific. 3 | COLOR SPACES' REVIEW Most color spaces use three components to represent a complex color. A few spaces can describe a complex color Color spaces can be classified in many ways as already dis- in terms of a chromaticity diagram using only two compo- cussed in section 2. However, to study their attribute impor- nents. Third component can be expressed as one minus addi- tance and characteristics relevant to compression, we group tion of the other two components. A few other color spaces the different color spaces based on the purpose for which use more than three attributes to describe complex colors. they were originally proposed as shown in Figure 3. Thus, Thus, color spaces can also be categorized in two categories; the color spaces are broadly classified into sensor based, dis- (1) Euclidian color spaces with three or less attributes and play based, color matching based, color equivalency based, (2) Riemannian color spaces with more than three attributes.