Discriminating Colors with Smartphone Camera for Deco-Film Pvc Sheet

ICIC Express Letters ICIC International ⃝c 2018 ISSN 1881-803X Volume 12, Number 11, November 2018 pp. 1137–1143

DISCRIMINATING COLORS WITH SMARTPHONE CAMERA FOR DECO-FILM PVC SHEET

Hoyoun Im1 and Minsoo Kim2,∗ 1Graduate School of Management of Technology 2Division of Systems Management and Engineering Pukyong National University Daeyeon Campus, 45 Yongso-ro, Nam-gu, Busan 48513, Korea aﬁ[email protected]; ∗Corresponding author: [email protected] Received April 2018; accepted July 2018

Abstract. With the continuous enhancement of smartphones, the image quality and resolution of smartphone cameras have also reached almost the level of professional digital cameras. If these improved smartphone cameras can be used in production processes instead of conventional industrial equipment, then companies will be able to reduce huge investments for buying such expensive equipment. Of course, before them to be put into the industrial applications, comparative experiments on the performance of using the smartphone cameras over the conventional method should precede. Through this comparative study, the smartphone camera should show the performance within the tolerance required in the original process and then it can be utilized in the real production process. In this comparative study, authors have evaluated the color discrimination performance on Deco-Film PVC sheet using smart phone camera instead of industrial spectrophotometer. Tested smartphone camera showed high correlation on color difference values with industrial spectrophotometer. Additional experiment comparing the visual inspection results of human experts who are in charge of color discrimination in the actual production process with the smartphone camera has shown a much more stable result when the smartphone camera is used instead. In conclusion, authors have confirmed that smartphone camera can sufficiently replace visual inspection task for color discrimination of Deco-Film PVC sheet. If the results are applied to similar industrial processes, companies will be able to design more economical production environments by replacing expensive industrial equipment. Keywords: Smartphone camera, Spectrophotometer, Color difference, Deco-Film PVC

1. Introduction. According to GSMA Intelligence, there are 5 billion mobile subscribers globally [1], and Statista.com estimates that the number of smartphone users worldwide exceeds 2.5 billion [2]. Smartphones are now closely linked to everyday life and are used in businesses in a variety of ways across public and private sectors. With the spread of smartphones, the functions installed on smartphones are continuously improving and expanding. In particular, there are many smartphone cameras that already have the performance of professional digital cameras, though they depend on the model. Though the use of smartphone cameras is widely accepted in the consumer field, it has not extended to the manufacturing domain yet. There are still many chances to use smartphone cameras in the production field except for some high-end applications. Of course, sufficient experiment and research should be preceded for such use. Many visual inspection tasks which are important for the quality control occur generally at the production system [3,4]. Companies prefer automated testing to have consistent and robust results, but this re- quires a huge investment due to the introduction of expensive equipment. Because of this, many small and medium-sized enterprises (SMEs) that do not have enough resources rely mainly on labor for inspection tasks. The production of Deco-Film is one of such areas

DOI: 10.24507/icicel.12.11.1137

1137 1138 H. IM AND M. KIM that SMEs usually carry out and that require color inspection for PVC sheets. Keeping color difference between PVC sheets within a specified range is often performed by man- ual inspection, and an expensive spectrophotometer is used for high quality consistency. If smartphone cameras can be used for these tasks, then the investment can be reduced while retaining reasonable level of quality. In this study, authors have discussed how to use smartphone camera for color discrimination of Deco-Film PVC sheet. The composition of this paper is as follows. In the above introduction section, authors have briefly explained the motivation of this research. In the second section, authors have examined existing researches on the color discrimination using smartphone cameras. In Section 3, basic theories related to color representation and transformation are briefly summarized. Section 4 introduces the Deco-Film production process and explains how to use a smartphone camera for color discrimination of PVC sheet. Authors have compared the performance of spectrophotometer with that of smartphone camera, and finally compared the result of smartphone camera with visual inspection. In the last section, the conclusion of this study is briefly summarized with some future research issues. 2. Color Discrimination Using Smartphone Camera. There have been many at- tempts to use smartphone cameras for commercial purposes, such as in the infotainment area, but the use of them in production environments for color discrimination is relatively small. Amongst them, the cases that are published in academic papers are even harder to find. This section introduces some of the pioneering research for using smartphone camera to discriminate colors in the production environment. S. K. Vashist et al. developed an SBCR (Smartphone-Based Colorimetric Reader) using a smartphone (Samsung Galaxy S3 mini) camera to reduce cost, miniaturize and be lightweight by replacing commercial MTPR (Microtiter Plate Reader). For this purpose, a custom-made dark hood and base holder assembly was developed. The smartphone equipped with a back camera (5 megapixels’ resolution) was used for colorimetric imaging via the hood and base-holder assembly. With several experiments, they showed their SBCR is more compact, lighter, inexpensive and enriched with more features than conventional commercial MTPR [5]. S. Singhal et al. provided a medical image processing system based on colorimetric detection particularly for diabetic patients that measure the BGL (Blood Glucose Levels) by scanning an image of the visual glucose test strip using a smartphone. Their experiment results have shown that such technique has the promising capability to measure BGL in humans [6]. In their study, the Sony Xperia Arc LT15i model was used and the results were compared to the commercial Accu-Chek Active blood glucose meter. Y. Wang et al. used a smartphone-based colorimetric reader coupled with a remote server for rapid on-site analysis of catechol. The applicability of their method to a real-life scenario was confirmed by the on-site analysis of various catechols from a water sample of the Yangtze River. Their result showed that the method was able to identify the catechol with 100% accuracy and could predict the concentra- tion to within 0.706-2.240 standard deviation [7]. The color discrimination utilizing the improved performance of smartphone camera is gradually expanding its usage in chemistry and medical field like AssayColor app [8] and Enzo ELISA Plate Reader [9]. If the smartphone’s computing power, network connectivity and enhanced camera capabilities are properly combined, color discrimination task can be conducted as effectively as expensive industrial equipment in some applications. Conventional color discrimination tasks based on the naked eyes of human experts are neither efficient nor stable because the performance is significantly influenced by the situation of the workers [3,4], and thus they are gradually being replaced by the method of using the equipment like digital camera and industrial colorimeter. ICIC EXPRESS LETTERS, VOL.12, NO.11, 2018 1139 3. Color Representation and Transformation. The color space consists of a set of mathematically described colors and algorithms that can precisely interpret the individual color elements. A color model is an abstract mathematical representation scheme that the colors can be interpreted as tuples of numbers. Commonly used color models are CIE (International Commission on Illumination) XYZ, RGB (Red, Green, and Blue), CMYK (Cyan, Magenta, Yellow, and Black) and HSV (Hue, Saturation, and Value). Since various devices may use different color spaces and color models to represent colors, this section briefly talks over representative color spaces and color conversion between them. The CIE XYZ color model is one of the first mathematically defined color spaces based on the study of human color perception and it is the basis for almost all other color spaces. The widely known CIE RGB color space is also linearly related to this color space. The RGB color model is mainly used in a display device, and is typically configured to have a value ranging from 0 to 255 for each of R, G, and B dimension. The CMYK color model is mainly used for printers and the color density of each of C, M, Y, and K is expressed in a range of 0 to 100%. Specifically, K is added for color reproduction of a dark color. In the 1920s, Wright and Guild presented theories that underlie CIE XYZ model through experiments on human vision, and the CIE RGB color space was created from this study [10-14]. The International Commission on Illumination defined the CIE XYZ color space (CIE 1931) in 1931 based on these results [13,14]. The CIE LAB, actually CIE L∗a∗b∗ is a color space in which L is brightness, and A(a∗) and B(b∗) are the color-opponent dimensions that correspond to green vs. red opponent and blue vs. yellow opponent, respectively. According to the result that human color perception is nonlinear, the CIE LAB color space also has a nonlinear relationship with the actual light wavelength and is designed so that the distance of two different colors in the LAB color space is proportional to the actual color difference of human sensation [15,16]. The LAB color space, first published by Hunter in 1948, is obtained by transforming the CIE XYZ color space through a quadratic function. In modern times, the CIE L∗a∗b∗ color spaces are used mainly by converting through cubic function. Images taken by a smartphone camera are basically saved as RGB color values while industrial spectrophotometer usually adopts LAB color model. Thus, it is necessary to convert RGB colors to LAB colors to compare the smartphone camera captured image to spectrophotometer result. The color space conversion used in this study is summarized in Figure 1. It can be seen that CIE XYZ space is used temporarily for conversion from RGB color space to CIE L∗a∗b∗ color space. In the CIE L∗a∗b∗ color space, the difference between two colors can be obtained by the Euclidean distance between two points (L0a0b0 and L1a1b1) in three-dimensional space given in Equation (1). √ 2 2 2 ∆E = (L0 − L1) + (a0 − a1) + (b0 − b1) (1)

Figure 1. Conversions from RGB color space to CIE L∗a∗b∗ color space 1140 H. IM AND M. KIM 4. Comparison Experiment. Typically, Deco-Films are made by laminating PVC films and additional steel sheets on PET films. Since lamination is irreversible, color inspection of PVC film is made before that process. The entire process is briefly shown in Figure 2. Of course, it is desirable to use a colorimeter to strictly check the colors of input PVC film, but because of cost and inconvenience, it is done by the visual inspection of the operator in the actual process. It is highly possible to improve process consistency and product quality through a smartphone camera that is easier to use than the industrial colorimeter and that is more accurate than the human eye. This section describes two comparative experiments conducted to check the feasibility of smartphone cameras for Deco-Film production. The overall flow of the comparative experiments performed in this study is summarized in Figure 3. It can be briefed into two major experiments. First, samples’ color values are captured through the industrial spectrophotometer and smartphone camera. After converting the RGB values read from the smartphone camera into LAB values, the color difference from the standard sample can be calculated through Equation (1). By comparing ∆E1 and ∆E2, the color difference values of the industrial spectrophotometer and the smartphone camera, it is possible to grasp the color capturing capability of the two devices and their relationship. In the second experiment, visual inspections by human experts are conducted to sort out the samples in the order of increasing color difference values (∆E3) from the standard sample. This order of manually sorted samples is compared to that of increasing color difference values measured by smartphone camera. Since the task of letting the human operator estimate the LAB value of the sample is very error prone, authors have made a comparison to sample’s position by simply ordering the color differences from the standard sample. The details and results of the experiments are explained further.

Figure 2. Typical Deco-Film production process

Figure 3. Overall flow of the comparative experiments ICIC EXPRESS LETTERS, VOL.12, NO.11, 2018 1141 4.1. Spectrophotometer versus smartphone camera. From the various PVC color sheets, one color sample sheet is selected for each lot. For this sampled sheet, a snapshot is taken by using smartphone camera (Samsung Galaxy S7) in a Day light Box (Bo Teck Super Light). The average L2a2b2 value of the captured sample is calculated by using the center pixels (500 × 500) of the captured image. By using this average value, the color difference ∆E2 from the standard sample’s preset L0a0b0 value can be calculated. In the same way, the color difference ∆E1 can be calculated by using the captured color value L1a1b1 via spectrophotometer (Minolta 3600D) for the same sample. By inspecting these ∆E1 and ∆E2 values over multiple lot samples, we can see the correlation between the color capturing capabilities of above two devices. Figure 4(a) compares the ∆E2 values obtained from the smartphone camera by taking 30 times of snapshots for the same lot sample, to the ∆E1 value of the spectrophotometer. From these results, it can be seen that the color differences obtained from the two devices are consistently retained over multiple lots. Figure 4(b) shows that the color difference values obtained from the two devices actually have a very high correlation with the quadratic function. The adjusted R2 value in the quadratic regression equation has reached up to 88.95%, which also has shown statistical significance for p = 0.05 value. The actual regression equation may vary depending on the types of equipment used and test setups. However, judging based on the results obtained from this study, the color difference value of the spectrophotometer can be quite precisely estimated by the color difference value of the smartphone camera.

(a) Color diﬀerences: ∆E1 and ∆E2 (b) Regression result between two devices

Figure 4. Color capturing capabilities of spectrophotometer and smartphone camera

4.2. Smartphone camera versus visual inspection. In case of visual inspection, it is very difficult for the workers to estimate the LAB color values. So, the experiment is set to compare the relative color differences by sorting out the samples. First, the samples are arranged in the order of increasing color difference values measured by spectrophotometer, and then the samples with the color difference less than 0.05 are bound into one group. The reason for setting the reference value to 0.05 is that it is much smaller than the level that can be identified by the human experts (0.2 ∼ 0.4) but large enough to be detected by calibrated colorimeters. As a result, we could create 7 groups ranging from A to G as shown in the ‘Group’ column of Figure 5. By comparing the positional differences between the groups of captured color samples, we can easily calculate the capability differences between the two devices. The ‘Positional Difference’ column of Figure 5 shows that smartphone camera has total of 6 positional differences over spectrophotometer. By 1142 H. IM AND M. KIM comparing the positional differences, it is possible to compare the color discrimination capability of the worker to the other devices. In this study, 12 subjects who have ex- periences of color inspection work for more than 5 years are experimented. Workers are asked to rank the sample PVC sheet of the lot in accordance with the color difference from the standard sample, and their results are summarized in Table 1. It shows that the average of positional difference is 8.833, which is larger than the result obtained from the smartphone camera. That is to say, the discrimination of color using smartphone camera is more accurate than workers’ visual inspection, and it is more stable when considering the standard deviation of 2.480 caused by the differences in performance of the workers.

Figure 5. Comparison of color discrimination capability using group ranking

Table 1. Visual inspection result of 12 workers

Sample Worker Group Lot 1 2 3 4 5 6 7 8 9 10 11 12 G 2 G G G G G F F G G G G F G 9 F F F G F G G F F G F G F 7 G G G F E G G G G F G G E 1 E E C E E D C E E C C E E 11 D C D E G E C D D D D D D 13 E E E D D E E E E C E E C 12 C C E B B C E C C C E C C 8 C D C C C B C B C C C C C 6 C C C C C C C C B E B C C 3 B C B C A C B C B B B B B 4 C B C C C C B C C B C A B 10 B A A A C A D A C E C C A 5 A B B B B B A B A A A B 6 8 12 4 10 8 12 8 8 10 12 8 Positional Average Positional Difference: 8.833 Difference St. Dev. of Positional Difference: 2.480 ICIC EXPRESS LETTERS, VOL.12, NO.11, 2018 1143 5. Conclusion and Future Research. In this study, authors have tried to answer the question whether the industrial spectrophotometer can be replaced by smartphone camera. To verify this, the authors have conducted a comparative experiment on the capabilities of the two devices and compared the results using the smartphone camera with those of the visual inspectors over the Deco-Film PVC color sheet. Experimental results show that the captured color difference value of smartphone camera from the standard sample is larger than that of industrial spectrophotometer, but the difference is somewhat consistently retained across the test samples. In addition, the color difference values of the smartphone camera and those of spectrophotometer have a very high correlation and can be regressed through a quadratic equation. This means that it is possible to very closely predict the spectrophotometer’s measurement result by using the color difference value of the smartphone camera. Smartphone camera has also shown that color discrimination is more accurate than visual inspection. Based on the above experiment results, it seems that the smartphone camera can replace the industrial spectrophotometer. Identifying colors can be greatly affected by the characteristics of the device. Therefore, depending on the combination of devices to be tested, the results of the experiment may vary considerably. To actually take advantage of the smartphone camera in the manufacturing process, additional experiments should be performed to accurately account for the characteristics of the devices used by each company.

REFERENCES [1] GSMA Intelligence, Number of Mobile Subscribers Worldwide Hits 5 Billion, https://www.gsma.com /newsroom/press-release/number-mobile-subscribers-worldwide-hits-5-billion/, 2017. [2] Statista.com, Number of smartphone users worldwide from 2014 to 2020, https://www.statista.com /statistics/330695/number-of-smartphone-users-worldwide/, 2018. [3] J. Beyerer, F. P. Leon and C. Frese, Machine Vision: Automated Visual Inspection: Theory, Practice and Applications, 1st Edition, Springer, 2016. [4] D. Grace, The Eﬀectiveness of Performance Appraisal Systems: Employee Relations and Human Resource Management, Anchor Academic Publishing, 2014. [5] S. K. Vashist, T. van Oordt, E. M. Schneider, R. Zengerle, F. von Stetten and J. H. T. Luong, A smartphone-based colorimetric reader for bioanalytical applications using the screen-based bottom illumination provided by gadgets, Biosensors and Bioelectronics, vol.67, pp.248-255, 2015. [6] S. Singhal, P. Ralhan and N. Jatana, Smartphone-based colorimetric detection to measure blood glucose levels, The 8th International Conference on Contemporary Computing (IC3), 2015. [7] Y. Wang, Y. Li, X. Bao, J. Han, J. Xia, X. Tian and L. Ni, A smartphone-based colorimetric reader coupled with a remote server for rapid on-site catechols analysis, Talanta, no.160, pp.194-204, 2016. [8] Alidans.com, AssayColor, http://www.alidans.com/assaycolor/2.aspx. [9] EnzoLifeSciences.com, Immunoassays Kits, http://www.enzolifesciences.com/platforms/immunoass ay-and-assay-development/immunoassays-elisa-kits/. [10] T. C. Stubbings and H. Hutter, Peer previewed: Combining multispectral image information using color, Analytical Chemistry, vol.72, no.2, pp.282-288, 2000. [11] G. Sharma and R. Bala, Digital Color Imaging Handbook, CRC Press, 2017. [12] A. Koschan and M. Abidi, Digital Color Image Processing, 1st Edition, Wiley-Interscience, 2018. [13] C. Oleari, Standard Colorimetry: Deﬁnitions, Algorithms, and Software, Wiley, 2016. [14] J. Schanda, Colorimetry: Understanding the CIE System, Wiley-Interscience, 2007. [15] M. D. Fairchild, Color Appearance Models, 3rd Edition, John Wiley & Sons, 2013. [16] D. Margulis, Photoshop LAB Color: The Canyon Conundrum and Other Adventures in the Most Powerful Colorspace, 2nd Edition, Peachpit Press, 2015.