1 A color temperature-based high-speed decolorization: an empirical approach for tone mapping applications Prasoon Ambalathankandy ID˙ , Yafei Ou ID˙ , and Masayuki Ikebe ID˙ Abstract—Grayscale images are fundamental to many image into global and local methods. Global methods can define processing applications like data compression, feature extraction, only one conversion function for all pixels, and most of printing and tone mapping. However, some image information these methods use all pixels in the image to determine the is lost when converting from color to grayscale. In this paper, we propose a light-weight and high-speed image decolorization function. On the other hand, local ones process the target method based on human perception of color temperatures. from neighboring pixels in the same way as a spatial filter, Chromatic aberration results from differential refraction of light the function is different for each pixel. However, both types depending on its wavelength. It causes some rays corresponding of methods face the issue of calculation cost, which comes to cooler colors (like blue, green) to converge before the warmer from optimization iterations or spatial filter processing. colors (like red, orange). This phenomena creates a perception of warm colors “advancing” toward the eye, while the cool colors We have developed a fast decolorization method that reflects to be “receding” away. In this proposed color to gray conversion the perception of warm and cool colors which is well known model, we implement a weighted blending function to combine in psychophysics studies [1]. Colors are arranged according to red (perceived warm) and blue (perceived cool) channel. Our their wavelengths on the color wheel, the ones with longest main contribution is threefold: First, we implement a high-speed wavelengths are on the right side of the wheel and are known color processing method using exact pixel by pixel processing, and we report a 5:7× speed up when compared to other as warm colors, as they evoke warmth. These hues include new algorithms. Second, our optimal color conversion method shades of red, yellow, and orange. On the other hand, green, produces luminance in images that are comparable to other blue and violet which have shorter wavelengths are placed state of the art methods which we quantified using the objective on the left side of the color wheel, and are perceived as cool metrics (E-score and C2G-SSIM) and a subjective user study. colors. The color of an object in a scene affects our perception Third, we demonstrate that an effective luminance distribution can be achieved using our algorithm by using global and local of its apparent depth and this phenomenon has been exploited tone mapping applications. by many artists. This optical illusion has been studied by psychologists, and early researchers explored color-depth re- Index Terms—Color temperature, chromatic aberration, de- colorization, luminance, pre-processing, RGB, tonemap. lationship. One of the widely accepted theory explains this phenomenon is due to fact that shorter wavelengths of visible light are refracted more than longer wavelengths [2]. In other I. INTRODUCTION words, an equidistant source of different wavelengths cannot Grayscale channels, which reflect image luminance, are be focused simultaneously onto our retina. This phenomena used for various applications such as printing, tone mapping, is called as chromatic aberration and we discuss it in detail data compression, and feature extraction. Thus, obtaining lu- in section III-A. In our decolorization method, we combine minance along with human perception has a key role for decol- the idea of warm/cool colors with the Helmholtz-Kohlrausch orization, which converts RGB channels to high-quality gray effect [3]. On that account, we make two assumptions, which arXiv:2108.13656v1 [eess.IV] 31 Aug 2021 ones. For example, High Dynamic Range (HDR) compression are: (i) warm colors (mainly including R) are lighter than Y is ideally performed by tone mapping the luminance channel of YCbCr and (ii) mixed colors are darker than the Y or L of for the lower computational and memory cost. However, CIE with the same luminance. To satisfy these assumptions, applying well-known luminance channels such as Y of YCbCr we use a weighted blending of RGB channels and remap them or V of HSV does not guarantee appropriate tone mapping, to warm/cool colors on the luminance channel. Following are as these channels do not reflect human perceptions. Therefore, our main contributions: decolorization has gathered considerable attention and various • We achieve high-speed color mapping by exact pixel-by- sophisticated methods to achieve perceptual decolorization pixel processing. have recently been proposed. These methods can be classified • We obtain luminance comparable to that of optimization- Prasoon Ambalathankandy and Yafei Ou are with the Research Center based methods. For Integrated Quantum Electronics, Hokkaido University, Sapporo 060- 0813, Japan, and also with the Graduate School of Information Science and • We demonstrate effective luminance distribution by per- Technology, Hokkaido University, Sapporo 060-0814, Japan. forming objective evaluations and a subjective user study. Masayuki Ikebe is with the Research Center For Integrated Quan- tum Electronics, Hokkaido University, Sapporo 060-0813, Japan (e-mail: There are many well defined methods to convert any color [email protected]). image to a grayscale image. An effortless procedure is to 2 cool warm Luminance advance HVS based recede Original Image color mapping This work YCbCr: Lum. Fig. 1: Main concept of our decolorization method: Human perception of warm and cool colors. Warm colors “advance” toward the eye, while cool colors “recede”. In this work we are able to accurately reflect the human perception of color temperatures, whereas this phenomenon is non-existent in conventional YCbCr color space. In the global methods category, Gooch et al. [4] proposed a global decolorization algorithm that can be implemented by solving the optimization problem for all image pixels. Then, Kim et al. [7] aimed at high-speed processing by simplifying Gooch’s method. Smith et al. [6] used unsharp masking and the Original Image Y of YCbCr CIE Lab This work Helmholtz-Kohlrausch effect (H-K effect) model of Nayatani et al. [3]. Nayatani’s model [3] is merely an experimental Fig. 2: Comparison of luminance components obtained using model for the effect of the CIELUV chrominance component YCbCr, CIELAB and our proposed method. for human perception. On the other hand, the method of Lu et al. [8] is focused on converting a color image into a gray image with high contrast. The main advantage of these assign different weights to color channels, in order to have the methods is transformation consistency, i.e., the same color is same luminance in the grayscale image as the original color converted to the same grayscale. However, speed remains a image. For example, in the MATLAB function rgb2gray, problem for these methods. Most of the local methods are it converts any RGB values to grayscale (Gray) values by aimed at speeding up the method of Lu et al [8]. To enhance forming a weighted sum of the R, G, and B components as image contrast, Ancuti et al. [10] adopted the strategy of Gray = 0:2989×R+0:5870×G+0:1140×B. This function using a Laplacian filter and Song et al. [12] used a Sobel operates under an assumption that human visual system is filter. Although local methods are effective in terms of contrast more sensitive to green color. When operating with CIELab emphasis, they are disadvantageous in terms of tone mapping and YUV color spaces, one could directly obtain luminance because conversion consistency is not maintained and it differs channel as the grayscale version of the color image as they from human perception. consider the luminance and color channel to be independent. Recently, some machine learning-based techniques have But, such crude approaches will fail to preserve image contrast also been proposed for image decolorization [13]–[17]. Cai as shown in these examples (see Fig. 1 and Fig. 2). et al. [14] proposed a method, that used the perceptual loss In several real-world image/video processing applications function to pretrain VGG-Net [18]. However, it is difficult like detail enhancement, image matching, and segmentation to control and many of their output images are far from under different illumintion a 1-D grayscale image has to be human perception additionally, the computational cost are obtained from its corresponding 3-D color image. However, high. Processing an image of 256 × 256 size it requires mapping the 3-D color information onto a 1-D grayscale roughly 30 seconds on a single Nvidia GeForceGTX 1080 image while retaining the original contrast and fine details is a GPU. Zhang et al., proposed a CNN framework that combines challenging problem. Additionally, implementing decoloriza- local and global image features [17]. However, their network tion algorithms with a reasonable computational efficiency is framework do not account for exposure features [13]. Lin pivotal for realising their real-time applications. Many studies et al.’s method [16] by utilizing a database of 50 images have been carried out to develop novel decolorization methods. from the Corel dataset produced 50 grayscale images using These mapping methods can be categorized into global [4]– the Color2Gray algorithm [4]. With these 50 input/output [8] and local methods [9]–[12]. In local mapping methods, the image pairs as training examples for their partial differential same color pixel within an image could be mapped into differ- equations-based (PDE)learning system, the learn color2gray ent grayscale values depending on its spatial location. Ideally mapping. The PDE system produced images of comparable this undesirable as such output images may be perceived as quality to [4]. However, for an input image of size n × n their unnatural.