sign in sign up
<<
Home , Chroma subsampling

WSEAS TRANSACTIONS on COMPUTERS Pavel Pokorny

Lossy Compression in the Chroma Subsampling Process

PAVEL POKORNY Department of Computer and Communication Systems Faculty of Applied Informatics, Tomas Bata University in Zlín Nad Stráněmi 4511, 760 05 Zlín CZECH REPUBLIC [email protected] http://fai.utb.cz

Abstract: - Nowadays, compression algorithms are frequently used in the computer software field. The reason is the large amounts of data that is processed by applications. These compression algorithms, in different forms, are deployed in many raster image processing cases. The reasons for this are the quite high compression ratios and also, the usually high compression/decompression speed (the success of these two major parameters of the compression algorithms depends on the image content). In addition, the images can often allow one to use algorithms which cause a substantially greater reduction of the image data size. The JPEG graphic format is the most widely used. Here, the loss rate is determined in two ways by the subsampling components and the quantization of the coefficients that are obtained by the Discrete Cosine Transform calculation. In this paper, attention is focused on the subsampling process of the color components. The content compares the most frequently used sampling techniques - mainly to the quality and size of the processed image.

Key-Words: - , Lossy Compression, Image Quality, Sampling,

1 Introduction schemes to reduce file size by eliminating is the art or science of redundancy. Many files and graphic formats use this representing information in a compact form. We principle with different modifications. create these compact representations by identifying Lossy compression techniques involve some loss and using structures that exist in the data. Data can of information, and data that has been compressed be characters in a text file, numbers that are samples using lossy techniques generally cannot be of speech or image waveforms, or sequences of recovered or reconstructed exactly. In return for numbers that are generated by other processes. The accepting this distortion in the reconstruction, we reason we need data compression is that more and can generally obtain much higher compression more of the information that we generate and use is ratios than is possible with [1]. in digital form - in the form of numbers represented Lossy data compression schemes are designed by by bytes of data. And the number of bytes required research on how people perceive the data in to represent multimedia data can be huge. For questions. For example, the human eye is more example, in order to digitally represent 1 second of sensitive to subtle variations in luminance than it is without compression (using the CCIR 601 to the variations in color. JPEG image compression format), we need more than 20 megabytes, or 160 or work in part by rounding off megabits [1]. nonessential bits of information [2]. Compression can be either lossy or lossless. Programs using simple or complex graphics are Lossless compression reduces bits by identifying appearing in virtually every area of computing and eliminating statistical redundancy. No applications: e.g. games, education, desktop information is lost in lossless compression i.e. the publishing or graphical design, just to mention a original data can be recovered exactly from the few. These programs all have one factor in common. compressed data. Lossless compression is possible The images they use consume prodigious amounts because most real-world data have statistical of memory or disk storage space. The raster redundancy. For example, an image may have areas representation method is very often used for images of color that do not change over several ; (a display mode for the vast majority of hardware). instead of coding "red , red pixel ..." the data The amount of image data depends on the resolution may be encoded as "279 red pixels". This is a basic and in this case. For example, a single example of run-length encoding; there are many

E-ISSN: 2224-2872 76 Volume 15, 2016 WSEAS TRANSACTIONS on COMPUTERS Pavel Pokorny

image with a resolution 800 x 600 pixel and 224 of CCD cameras. For this reason, the color depth consumes over 1.4 MB of memory. It first step is to separate the brightness (sometimes isn’t hard to imagine applications that would require called “”) and color components. This literally hundreds of these images to be accessed conversion process usually means a transformation [9]. to one of the models used in color analog television Compression offers the solution to this problem. techniques e.g. YIQ (the PAL TV standard), YUV In images, its objective is to reduce the irrelevance (the NTSC TV standard) or YCbCr (the SECAM and redundancy of the image data in order to be able TV standard). to store or transmit data in an efficient form. Lossy This transformation between two color spaces is lossless, i.e. there is no possibility of losing any methods are especially suitable for natural images image information. So, if one converts an image into such as photographs in applications where minor a second one with any color space used in TV (sometimes imperceptible) loss of fidelity is techniques, it is possible to convert it back to the acceptable in order to achieve a substantial RGB or CMYK color model at any time and we reduction in . always get a result identical to the original image. A range of different lossy compression methods [10] are used currently. One of them is to reduce color The transformation equation between RGB and space to the most common colors in the image (i.e. YIQ is: the selected colors are specified in the color palette that is included in the image file). Another method  0.587R0.299 + Y G=  B0.114+ is based on . This is the most 0.275GR0.596 - I =  B0.321 + (1) commonly used method and forms the core of the images [3][11][8]. 0.523R0.212 - QG = B0.311- The third lossy compression method is chroma subsampling, which takes advantage of the fact that The relation between RGB and YIQ is given by the human eye perceives spatial changes of the equation: brightness more sharply than those of color [4][5]. This paper is focused on this (third) compression Y =  R0.299 + 0.587  G +  B0.114 method. The second chapter describes the theory of U = - R0.147 - 0.289  G +  B0.436 (2) the subsampling process. The first section of the second chapter presents the color transformations V = R0.615 - 0.515  G -  B0.100 that are necessary in order to separate brightness and color components. The second section lists the most commonly used subsampling methods at present. The third chapter shows the created application and describes both the user’s and programmer’s views. The fourth chapter provides information about the results that these created applications allowed us to obtain and a discussion about data-saving and decreasing image quality.

2 Chroma Subsampling The Chroma subsampling process requires color transformations. The following text also begins by describing these, and then goes on to delineate chroma subsampling methods.

2.1 Color Transformation Fig. 1 Top left: Original image; Top right: The Y As mentioned above, the subsampling algorithms component; Bottom left: The Cb component; concern only the color components of the picture. Bottom right: The Cr component [12] The images themselves are mostly represented by the RGB or CMYK color models, or by the internal

E-ISSN: 2224-2872 77 Volume 15, 2016 WSEAS TRANSACTIONS on COMPUTERS Pavel Pokorny

The RGB and YCbCr transformation equation is: which are subsampled). Usually, this value is equal to 4. 0.587 + R0.299 = Y = R0.299 + 0.587  G +B0.114 • The second number is the horizontal factor of the Cb and Cr components (i.e. number of Cb  0.3313G R0.1687 -  = +B0.5 - + 128 (3) samples in the rows of the processed area).  0.4187 =Cr R0.5 -  G  B0.0813 + 128 - • The third number describes the vertical factor of the Cb and Cr components (i.e. number of The main concern of this article is the last- chrominance samples in the columns of the mentioned color model (YCbCr), because it is used processed area). very often, for example, in the JPEG graphic format. • The fourth number is like the first number. The The reverse conversion is performed according to difference is, it represents the horizontal sampling the following formula: reference for the alpha (i.e. transparency) in the case where the image supports it. This number is 128)-(Cr1.402 + Y = R =  Y + 128)-(Cr1.402 not used without the transparency. G  0.34414 0.71414=128)-(Cb Y 128)-(Cr - - (4) The most commonly used leading digit of 4 is a 128)-(Cb1.772 + Y = B =  Y + 128)-(Cb1.772 historical reference to a sample rate roughly four times the NTSC or PAL color subcarrier frequency; Figure 1 shows the original image and the same the notation originated when subcarrier-locked image with the separated components Y, Cb and Cr. sampling was under discussion for It is possible to clearly see that the most significant [7]. The most common chroma subsamplings are: component is precisely Y in this picture. In contrast, the Cb and Cr components reported low intensity. • 4:4:4 – without subsampling. Each of the three For this reason, the subsampling algorithms refer YCbCr components has the same sample rate, like only to these components. the input resolution. This scheme is commonly used in cinematic post-production and a different compression method is usually used in the following 2.2 Subsampling steps. There are several ways how to make the chroma • 4:2:2 - both chroma components are each subsampling process. It is possible to find them in subsampled by a factor of 2 horizontally; their all analog color television standards (SECAM, PAL, effective positions are coincident (co-sited) with NTSC) in a certain form. Usually, two subsampling alternate brightness. Many formats and methods are performed - these methods are often interfaces use this scheme. used in the JPEG graphic format. The first method • 4:1:1 - in this scheme, the Cb and Cr components calculates the ratio of the color of the neighboring are each subsampled by a factor of 4 horizontally pixels that are placed on the screen in a row. The and are coincident with every fourth brightness second algorithm calculates the average color of the sample. Currently, this subsampling is used for low- four neighboring pixels (these pixels are placed in end and consumer applications. the two adjacent rows and columns on the screen). • 4:2:0 – the Cb and Cr components are each Both methods are lossy; when the algorithm subsampled by a factor of 2 horizontally and a calculates the final color component, in the first factor of 2 vertically. This scheme is used in case, the size of the raster data is reduced by about JPEG/JFIF still-frames in computing, in H.261 (for 33%, in the second case by about 50%. [6][12] video-conferencing), in MPEG-1, in consumer In practice, subsampling is designated by a string 576i25 DV25, and in most variants of MPEG-2. of 3 (or sometimes 4) integers separated by colons, Different technologies support different variants of e.g. 4:2:2:4. The relationship among the integers 4:2:0 subsampling. In JPEG/JFIF, H.261, and denotes the degree of vertical and horizontal MPEG-1, Cb and Cr are sited interstitially, subsampling. At the outset of digital video, horizontally halfway between alternate brightness subsampling notation was logical; unfortunately, samples. In MPEG-2, Cb and Cr are co-sited technology outgrew the notation. [7] Today’s horizontally. notation means: • 4:1:0 – This ratio uses half of the vertical and one- fourth of the horizontal color resolutions, with only • The first number represents the horizontal one-eighth of the bandwidth of the maximum color sampling reference (the width of the area in pixels, resolutions used. An uncompressed video in this format with 8-bit quantization uses 10 bytes for

E-ISSN: 2224-2872 78 Volume 15, 2016 WSEAS TRANSACTIONS on COMPUTERS Pavel Pokorny

every macro-pixel (which is 4 x 2 pixels). Some Javascript language, which ensures the subsampling support this ratio - but it is not widely used. calculation according to the selected method and • 3:1:1 – the Cb and Cr components are each then displays its result in the right window. The last subsampled by a factor of 3 horizontally. The button, called “Reset Output”, erases subsampling chroma samples are then divided by every third results and clears the right window. brightness sample. 36 bytes of the R, G and B components are also reduced to 20, effecting approximately 2:1 compression. [7]

The above described differences among chroma subsamplings 4:4:4, 4:2:2, 4:1:1 and 4:2:0 are shown in Figure 2.

Fig. 3 User interface of application to test the image subsampling

3.2. Programmer’s View The design of the created application is organized into two files - index.html and script.js. The Fig. 2 Top left: Subsampling 4:4:4; Top right: index.html file contains the whole HTML code and Subsampling 4:2:2; Bottom left: Subsampling 4:1:1; also all properties that are set by the help of Bottom right: Subsampling 4:2:0 Cascading Style Sheets (CSS). The HTML code is normally determined by tags. Each of them has its own identifier (the "id" attribute). The buttons (tag 3 A Testing Application ) have additional "value" properties, which In order to evaluate the results of the subsampled contain a text description of each button. The drop- color components, an application based on HTML5 down menu for selecting the type of color and the Javascript programming language was subsampling is designed by the tag is application can be run in any web browser. formed by the

E-ISSN: 2224-2872 79 Volume 15, 2016 WSEAS TRANSACTIONS on COMPUTERS Pavel Pokorny

parameter) and all edges including borders the "onclick" method. This method can run any ("margin", "border", "border-radius", and "padding" function that can do whatever a programmer wants. properties). For mutual placement of the windows, Clicking the Load Image button runs the function the left canvas has the property "float: left" set that ensures loading the selected image from a file. additionally. The loaded image is stored in the image object and it is immediately drawn on the canvas with the help 3.2.1 Javascript of the "drawImage" method. This method allows The whole program code is divided into individual drawing in different forms – direct draw, image functions. Global initializations represent only scaling before drawing or image clipping in the exceptions from this. These initializations include rectangle form or scaling and drawing on the canvas the canvasses specifications, access to user interface in the defined position. buttons and declarations of global variables. The Reset Output button causes the calling of a The user can draw on the canvas using its simple function, where only one command deletes primary rendering context, which can be obtained the output canvas window. This command is on the basis of an object handle. This handle can be performed by the help of the "clearRect" method. derived by calling the "document.getElementById" The function that is called by clicking on the function, which contains a single parameter (object Calculate Button is the most complex. It has to identifier). In this case, this means the id attribute of perform several operations in the correct order: relevant canvas. Access to the context is then given by calling the "getContext" method, see below:  "image" to "ImageData" conversion  RGB to YCbCr transformation var canvas1 = document.getElementById ("Can1"),  chroma subsampling based on selected method canvas1Context = canvas1.getContext ("2d");  YCbCr to RGB transformation  "ImageData" to "image" conversion Using this, it is possible to generate the contexts of both canvasses. These contexts provide one with The "image" to "ImageData" conversion is a large number of methods that programmers can performed by the "getImageData" method which use to draw many graphic elements - like rectangles, transfers pixel values from the input canvas into the complex paths or images, to set customizing "ImageData" object. The reverse process at the end drawing styles, or use various transformations. of the operation list is performed by the In order to communicate with the user through "putImageData" method. The color transformation the application interface, it is also necessary to processes and chroma subsampling calculation are obtain the handles of whole command elements. programmed based on the algorithms described in These elements include buttons and the drop-down the previous chapters of this paper. menu, as mentioned above. To do this, we use the "document.getElementById" function again. The application also contains only two global 4 Results and Discussion variables (objects) – "image" and "imageData". The To test the created application, several different "image" variable represents an image object. Its images were used. The commonly used jpg graphic declaration is performed by the command: format was avoided because it already includes

lossy compressed data and the results would be image = new Image(); inaccurate. That is the reason why images were used

and why were rendered in the Blender software After this initialization, the programmer can set [15]. The rendered images were exported to the the image source and draw it on the selected canvas. PNG graphic format because this format uses the The "imageData" object represents pure data of an lossless compression algorithm. Figure 4 shows a image, i.e. typically R(ed), G(reen), B(lue) and the zoomed selected part of one image - a roof with a A(lpha) values of each pixel in the given image. chimney in 64 x 64 pixel resolution. The original This is necessary so as to have direct access to the image is in the top left corner, the other images whole range of color and transparency components represent outputs created using the 4: 2: 2, 4: 1: 1 and to calculate the subsampling process. and 4: 2: 0 subsampling algorithms. The differences When the user clicks on the selected button are not very noticeable in the full resolution of these (Load Image, Calculate or Reset Output), this action images. So, this zoomed part of the image was causes an event that can be detected by the help of created in order to see more visible differences in

E-ISSN: 2224-2872 80 Volume 15, 2016 WSEAS TRANSACTIONS on COMPUTERS Pavel Pokorny the various types of the subsampling algorithms. 5 Conclusion The most significant changes are logically evident Subsampling algorithms offer a very interesting after using the 4: 1: 1 and 4: 2: 0 subsampling. [14] option to reduce the amount of image data. Due to The subsampling process causes merging the fact that the loss concerning image information (usually by using the averaging calculation) of the is the least noticeable to the human eye, this process color components of the neighboring pixels. In is preceded by the conversion process, where essence, this leads to the blurring effect that is more separated brightness and color components are pronounced with the increasing number of merged separated. Then, subsampling is only used for these pixels. This implies that this method is particularly color components. suitable for photographs or rendered images which Currently, several different types of subsampling don’t contain sharp edges or color gradients. In algorithms are used in the image processing process these cases, the subsampling of the color and - depending on its type, this leads to the components is the most visible. corresponding compression results. Practical Table 1 shows the data-saving for each type of experience shows that the subsampling process is subsampling. This is the total savings of the original suitable for images with high color depths and larger data, i.e. the percentage numbers include all three resolutions; and lost colors are more visible in components. images that contain sharp edges or color gradients. In order to test the subsampling algorithms, the Tab. 1 Data-saving using different subsampling application in HTML5 in combination with the methods JavaScript programming language was created. This application allows one to subsample components by Subsampling 4:4:4 4:2:2 4:1:1 4:2:0 using 4:4:4, 4:2:2, 4:1:1 and 4:0:0 methods. method Currently, these standards are often used in some graphics images and many video formats. This Save 0% 33% 50% 50% application that was created could be further extended in the future. This extension can contains further subsampling algorithms including A comparison of Figure 4 and Table 1 evokes the transparency or the support of different color idea of seeking a compromise between image representations (e.g. YUV, YIQ). In addition, this quality and the quantity of image data. If the highest application could support other user-friendly image quality is preferred, avoiding the losing features such as zoom in/out and navigation of the compression algorithms including the subsampling images, or save the output into a file. processes is the best solution. The second thing is, if The increased compression ratios can be further the images are intended for common user purposes, reached by using it in combination with other some loss rate can usually be tolerated, because the compression algorithms - just as the JPEG standards image quality can be decreased very slightly. And, do. Future work will also proceed in this direction. Table 1 shows that the user can obtain very interesting memory storage savings. References: There is a problem to define the term “acceptable [1] K. Sayood, Introduction to Data Compression, image quality”. How much image information can 3rd edition, Elsevier, 2006, 680 p., ISBN be lost so one ever could say that image still 9780126208627. provides sufficiently high quality image [2] A. Khasman and K. Dimililer, Haar Image information? Classical methods that identify an Compression Using a Neutral Network, image quality (expressed errors) are not applicable ACC’08 Proceedings of the WSEAS because their solution offers strange results. For International Conference on Applied example, a comparison of the differences between Computing Conference, 2008, pp. 412-417, the original image and the image with the ISBN 9606766671. subsampled color components can often give the [3] M. Petrou and C. Petrou, Image Processing: result that none of the pixels have the same value. The Fundamentals, A John Wiley and Sons. So, the typical method for error measurement (i.e. Ltd., 2010, 794 p., ISBN 9780470745861. the sum of the squares of the differences of color [4] H. Chen, S. Mingzhe and Eckehard Steinbach, pixel values) can return high values, even where the Compression of Bayer-pattern video sequences differences between these images are almost using adjusted chroma subsampling. IEEE unnoticeable. [11]

E-ISSN: 2224-2872 81 Volume 15, 2016 WSEAS TRANSACTIONS on COMPUTERS Pavel Pokorny

Transactions on Circuits and Systems for Video [11] P. Tisnovsky (2006). Lossy compression with Technology, vol. 19, 2009, pp. 1891-1896. JPEG [Online]. Available: [5] Ch. Glenn, Toward Better Chroma http://www.root.cz/clanky/ztratova-komprese- Subsampling Recipient of the 2007 SMPTE obrazovych-dat-pomoci-jpeg/#ic=serial- Student Paper Award, SMPTE Motion Imaging box&icc=text-title Journal, vol. 117, 2008, pp. 39-45. [12] P. Tisnovsky (2006). Programming JPEG – [6] C. Hass (2008). JPEG Chroma Subsampling color transformation and subsampling [Online]. [Online]. Available: Available: http://www.impulseadventure.com/photo/chro http://www.root.cz/clanky/programujeme-jpeg- ma-subsampling.html transformace-a-podvzorkovani- [7] Ch. Poynton (2008). Chroma Subsampling barev/#ic=serial-box&icc=text-title notation [Online]. Available: [13] W3C contributors (2014). HTML5 - a http://w.poynton.com/PDFs/Chroma_subsampli vocabulary and associated APIs for HTML and ng_notation.pdf XHTML [Online]. Available: [8] Prepressure contributors (2015). The JPEG file http://www.w3.org/TR/html5/ format [Online]. Available: [14] Red contributors (2015). Chroma subsampling http://www.prepressure.com/library/compressio techniques [Online]. Available: n-algorithm/jpeg http://www.red.com/learn/red-101/video- [9] M. Nelson, Lossy Graphics Compression, The chroma-subsampling Data Compression Book, 2 edition, Willey, [15] Blender Foundation (2015). Blender – Free and Cambridge, UK, 1995, pp 216-255. Open 3D Creation Software [Online]. [10] Y. Wiseman (2013). The still image lossy Available: http://www.blender.org/ compression standard – JPEG [Online]. Available: http://u.cs.biu.ac.il/~wiseman/jpeg2.pdf

Top left: Original image; Top right: The 4:2:2 image subsampling; Bottom left: The 4:1:1 image subsampling; Bottom right: The 4:2:0 image subsampling

E-ISSN: 2224-2872 82 Volume 15, 2016