See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/228394656 Adaptive Homogeneity-Directed Demosaicing Algorithm Article · July 2007 CITATIONS READS 11 2,602 1 author: Thorsten Frommen RWTH Aachen University 4 PUBLICATIONS 46 CITATIONS SEE PROFILE All content following this page was uploaded by Thorsten Frommen on 18 March 2018. The user has requested enhancement of the downloaded file. Adaptive Homogeneity-Directed Demosaicing Algorithm Thorsten Frommen Contents 1 Introduction 2 2 Digital Cameras 2 2.1DesignandFunctionality.................................... 2 2.2ColorFilters........................................... 3 2.2.1 BayerFilter........................................ 3 2.2.2 RGBEFilter....................................... 3 2.2.3 CYGMFilter....................................... 3 2.2.4 OtherColorFilters.................................... 3 2.3ImageSensors........................................... 4 2.3.1 MosaicFilterSensor................................... 4 2.3.2 FoveonX3......................................... 4 2.3.3 3CCDSensor....................................... 4 2.3.4 OtherImageSensors................................... 4 3Demosaicing 5 3.1ImageTheory........................................... 6 3.2Interpolating........................................... 6 4 Artifacts and their Dealing with 7 4.1Aliasing.............................................. 8 4.1.1 GreenImageInterpolation................................ 8 4.1.2 Red(Blue)ImageInterpolation............................. 8 4.2MisguidanceColorArtifacts................................... 9 4.3InterpolationArtifacts...................................... 10 5 The Algorithm 10 5.1Homogeneity-DirectedDemosaicingAlgorithm........................ 10 5.2AdaptiveParametrization.................................... 10 6 Currently used Demosaicing Techniques 11 7 Use in Medical Image Processing 11 7.1DigitalMedicalPhotography.................................. 11 7.2DigitalMicroscopy........................................ 12 7.3DigitalFluorescenceMicroscopy................................ 12 8Conclusion 12 9 Implementing the Algorithm 13 9.1AnImplementation........................................ 13 9.2ComputedResults........................................ 13 Abstract Digital cameras use in large part only one CCD (charge-coupled device) with a color filter array as image sensor. Thus, the output images feature a certain kind of pattern in which every pixel has only single- color information, for instance red or cyan. The process of modifying the raw (image) material and reconstructing an image with full RGB color information at each pixel is called demosaicing. In this paper, one particular way of demosaicing will be explained and analyzed. Concluding, the importance and the use of demosaicing regarding medical image processing will be considered. Keywords: demosaicing algorithm, color artifact, interpolation, Bayer pattern, metric neighborhood model. 1 Introduction This seminar paper is primarily based on Adaptive homogeneity-directed demosaicing algorithm, written by Hirakawa and Parks in 2005 [1] and an earlier version from 2003 [2]. In general, a digital still camera uses one sensor chip with a preceded color filter array. Since the most common type of digital cameras features a mosaic filter sensor (see Section 2.3) in combination with a Bayer filter (see Section 2.2), Hirakawa and Parks focus on this specific camera type. Its raw images have, corresponding to the filter design, at each pixel only single-color information. In order to acquire RGB images while not suffering losses regarding the actual resolution at the same time, certain aspects need to be thought over right-mindedly. These considerations emerge not only in respect of digital still cameras, but in respect of any digital optical apparatuses that use an insufficient amount of mosaic filter sensors; regarding medical image processing, there is more than just one application, as we will see in Section 7. 2 Digital Cameras To understand the topic properly, first we glimpse at the design and the functionality of a typical digital camera. At this, we will tend to the inside of the camera—in particular, the color filter array and the sensor chip. Having the basics made plain to us, we will begin looking into the subject of demosaicing and all its stumbling blocks. 2.1 Design and Functionality A common digital camera features a mosaic filter sensor that consists of a color filter array and a single CCD as image sensor. After passing the ocular, the light beams will encounter the filter complex, illustrated in Figure 1. Depending on the filter type, the light will be split into specific components Figure 1: Inner design of a typical digital camera. which will be transmitted to the chip that will convert the amount of incoming light into electric current. Finally, these electric signals will be processed into image data, typically in JPEG compression. In addition to this very common design of a digital camera, there are a lot of other ones. Some cameras, for example, use a CMOS (complementary metal oxide semiconductor) in place of a CCD. Other cameras don’t contain a mosaic filter while doing the filtering in some different way. Yet another camera type splits the light via a prism assembly and thus features three CCDs instead of only one. 2.2 Color Filters Figure 2: Bayer Filter. Figure 3: RGBE Filter. Figure 4: CYGM Filter. 2.2.1 Bayer Filter A very common color filter is the Bayer filter or so-called Bayer pattern [3]. It consists of three distinct filter types where the first one is sensitive to the green region of the spectrum, the second type transmits for the red spectral range and the third type is a blue filter. The filter types are arranged in a repeated 2 × 2 matrix (quincunx grid) that has one red and blue component, respectively, and, due to the fact that the human eye is more sensitive to green compared to red and blue, two green components. Furthermore, a better luminance representation is provided. Figure 2 shows the original array of the definition given in the US patent that has, in each quincunx grid, the two green elements placed in the diagonal from the upper left to the lower right. Often, patterns with other valid arrangements of these three filter types are called Bayer filters, too. 2.2.2 RGBE Filter The RGBE filter contains four distinct regions. In actual fact, it extends the Bayer filter by replacing in each quincunx grid one green component with one that transmits for the emerald (cyan) spectral range, illustrated in Figure 3. According to Sony Corporation, using this filter reduces the color reproduction error dramatically [4]. The RGBE filter, too, may occur with different arrangements of its components by having green and cyan in one diagonal, though. 2.2.3 CYGM Filter To be seen in Figure 4, the CYGM filter uses—in contrast to the two aforementioned filter types that feature the primary colors red, green and blue—the subtractive or typesetting colors cyan, yellow and magenta plus green. This four-color array is said to provide more accurate luminance information while it is less accurate regarding color information. 2.2.4 Other Color Filters Apart from the three introduced color filters, there are diverse other ones in use. The Eastman Kodak Company, for example, is currently working on new color filter patterns that feature panchromatic or “clear” pixels in order to collect a higher amount of light and thus deliver higher quality photos under low-light conditions [5]. Each and every mosaic filter has its own selection of color components and their specific arrangement; consequently, it needs its own demosaicing algorithm. 2.3 Image Sensors Figure 5: Mosaic filter sensor. Figure 6: Foveon X3. Figure 7: 3CCD sensor. 2.3.1 Mosaic Filter Sensor The most common image sensor is the mosaic filter sensor, shown in Figure 5. It consists of a mosaic pattern color filter, depicted in Section 2.2), and one chip. Each pixel element collects only single-color information, so the image material resembles the mosaic pattern of the filter. To obtain a full RGB color image, the collected data needs to be modified in the form of attaining the missing color information (see Section 3). 2.3.2 Foveon X3 The Foveon X3, like the mosaic filter sensors, is made up of one chip. A major difference lies within the arrangement of its components, as the Foveon X3 has three layers of pixels that are placed one upon the other within the silicon (see Figure 6). This technique takes advantage of the fact that red, green and blue light penetrate the silicon to different depths [6]. Since every pixel is covered by all of the three distinct filter layers, the collected data represents an image with full RGB color information at each pixel. 2.3.3 3CCD Sensor In contrast to the two above-listed sensors, the 3CCD sensor contains, as the name implies, three CCDs (see Figure 7). The prism setup in the center of the array refracts the light and directs the appropriate wavelength ranges to their respective CCDs [7]. The collected data will be processed to generate a RGB color image. Due to the spatial demands and the high expenses of the prism assembly, this sensor type is not as common as a single-chip sensor—regarding digital still cameras, at least. The main field of application of 3CCD sensors is digital video cameras, because size does not matter at this. 2.3.4 Other Image Sensors There are more than just these three sensor types in use—for example, the Super CCD sensor that has octagonal